DEMYSTIFYING EVALUATION

Contents

List of figures

List of examples

Acknowledgements

Preface

Glossary of selected evaluation terms

1. Introduction

Why ‘demystifying’ evaluation?

Who is the book for?

So what is evaluation?

Why evaluate?

When to evaluate?

Using the rest of this book

2. Compilation: setting the right foundations

Introduction

Types of evaluation

Measurement vs understanding

Objective-setting and evaluation rationales

Avoiding unrealistic expectations

The moral dimension

Resourcing, capability and independence

Engagement in effective evaluation

3. Composition:designing for needs

Introduction

Scaling and scope

Information needs and sourcing: secondary analysis

Primary evidence and gap analysis

Hybrid designs and their use

Proportionality in design

Data protection and security

Anticipating the analysis

4. Conducting process evaluation

Introduction

Different types of process evaluation

Process evaluation, self-evaluation and action research

Evaluating achievements

Evaluating effectiveness

Evaluating quality

Methodological options and choice

Avoiding pitfalls in process evaluation

5. Conducting economic evaluation

Introduction

Understanding the economic perspective

Cost evaluations

Cost-effectiveness evaluations

Cost-utility evaluations

Cost-benefit evaluations

Sensitivity analysis

Avoiding pitfalls in economic evaluation

6. Conducting impact evaluation

Introduction

Unpicking the jargon

Chasing the counterfactual

Fully experimental methods and RCTs

Quasi-experimental methods

Non-experimental evaluation

Qualitative methods and assessing impact

Avoiding pitfalls in impact evaluation

7. Analysis, reporting and communications

Introduction

Reporting and informing decision-making

Deliverables and reporting outputs

Demonstrating reliability and validity

Getting the report right

Going beyond reporting

8. Emerging challenges for evaluation and evaluators

Introduction

Integrating development and evaluation design

Rising expectations, diminishing resources

Timeframes and real-time analysis

Confidence, credibility and engagement in evaluation

Relativism and proportionality

Next steps

References

Annex A: The ROTUR framework for managing evaluation expectations

Annex B: Ready reckoner guide to experimentation choices in impact evaluation

Index