Evaluation and Monitoring Definitions
Monitoring is an ongoing assessment of policies and programmes undertaken during the implementation phase of a policy, often using administrative data, to assess whether the delivery of the policy is as planned, is reaching the intended target population and is using the allocated resources efficiently. Formative or process evaluation is a formal process of monitoring as it assesses how, why and under what conditions a policy is working, what is happening as the policy is been implemented, if it is been implemented in line with the original design and for whom the policy is working (or not working). Systematic monitoring of policies during the implementation phase contributes important data which is essential to effective evaluation.
Evaluation is a systematic assessment of the operation and/or the outcomes of a programme or policy, compared to a set of explicit or implicit standards, as a means of contributing to the improvement of the programme or policy.1
There are five key elements in this definition:
- Systematic assessment (either qualitative or quantitative, or both);
- Operation (study of process);
- Outcomes (study of the affect or impact - short-term and/or long-term);
- Standards of comparison (comparing findings against a set of expectations / indicators);
- Contribution to improving the programme or policy, as the evaluation findings are incorporated into the policy cycle.
The Policy Cycle
This policy cycle, referred to in the Green Book as ROAMEF, follows a logical sequence of policy formation:
- Rationale: What is the reason for developing a new policy? Is it necessary? Is the rationale for intervention clear? Can it be assumed that intervention will be cost-effective?
- Objectives: If it is established that there is a reason for intervention then what are the objectives of the proposed new policy or programme? What are the intended outcomes, outputs and targets?
- Appraisal: Policy options are appraised to develop a value-for-money solution that meets the objectives of the particular government action, within the allocated budget. Which option will deliver a cost effective solution and, at the same time, achieve the goals of the policy?
- Monitoring: When the agreed policy or programme is developed, an ongoing monitoring of its implementation will track its success in achieving the objectives and in ensuring the expected benefits. This involves a systematic collection of data relating to the management and outcomes of the policy during implementation. To be fully effective, plans for monitoring must form part of the initial planning of a policy or programme.
- Evaluation: When a policy or programme is completed or has advanced to a pre-determined degree, it should undergo a comprehensive evaluation. Evaluation examines the outturn or a policy against what was expected and is designed to ensure that the lessons learned are fed back into the decision-making process. This ensures that the public policy process is continually refined to reflect what best achieves objectives and promotes the public interest.2
There is a range of actors, each with different but complementary roles at each stage of the policy cycle, who would be the focus of an evaluation investigation.
- At the political level there are policy-makers who initiate and, possibly, design the policy but are not involved in implementation and delivery. In the political process, policies are not arrived at in isolation but are part of an overall strategy and are often complementary to each other;
- At the administration / management level are the managers who undertake the detailed design of the structures and organisation for the implementation of the policy and the project management;
- The service providers who are the front line delivery personnel;
- The target groups, individuals and/or representative organisations that the policy is designed to benefit.
1 Evaluation C H Weiss, Pearson, USA (1998)
2The Green Book: Appraisal and Evaluation in Central Government HM Treasury, London, TSO (2003) - available on www.hm-treasury.gov.uk/media/785/27/green_book_03.pdf