The Evidence-based Policymaking Commission strongly recommends that agencies use learning agendas for evaluation plans to build the base of evidence that can be used to make program and government decisions. They result in a structured approach for setting priorities for conducting rigorous empirical research about “what works” and what works “best.” Learning agendas can also include plans for needed research and analysis relevant to a whole range of evidence activities, including statistical performance analysis. This can help promote a culture of evidence.
Performance Management vs. Program Evaluation
Program evaluation and performance management are closely linked, but each serves a unique purpose. The unique roles thus typically require the two sets of activities to be organizationally separate. The overlapping roles, however, require that they coordinate to build evidence that can be used to improve program results.
Performance analysis and program evaluation provide different types of evidence about the results of programs and services. Evaluations are typically done by outside third party researchers; use a variety of data sources, determined based on the study question; and often estimate net impacts—what would happen without the intervention--that is compared to a counterfactual. Performance management is done in-house, uses administrative program data, and tracks progress towards management goals and objectives that managers and staff can use, ideally in real time, but without a counterfactual comparison.
Performance Management in Learning Agendas
Since performance management and program evaluation each produce information about the results of programs, it can be useful to include some performance-relevant analysis in an agency’s learning agenda can be very useful, complementing the regular and ongoing performance management analysis conducted by the performance management staff. The U.S. Department of Labor provides an example of the types of performance-related studies that can be incorporated into an agency’s learning agenda.
The Department of Labor’s Chief Evaluation Officer coordinates the learning agenda process. In collaboration with evaluation specialists in the Chief Evaluation Office (CEO), each operating agency in the department prepares an annual learning agenda that reflects their priority research topics and questions, including studies that are related to performance measures and outcomes that could provide evidence about how to improve performance:
The Performance Management Center (PMC) leads the Labor Department’s performance management activities, but the CEO coordinates with PMC in several ways. Rigorous evaluations help policy makers and administrators understand why public programs may or may not be meeting their goals, the relative effectiveness of different strategies to achieve goals, and how informed evidence can help identify what needs to change to improve results. In fact, evaluations contribute evidence that feeds into the performance management process. Examples of CEO studies that address performance measures and measurement are:
Culture of Evidence
Moving from a strict compliance culture to an evidence culture requires collecting, maintaining and analyzing data and information to address evaluation and performance questions of interest. A strong evidence culture is promoted by having a systematic, active, empirically based, and comprehensive program of research and evaluation, consciously coordinated with management and operations through strategic planning and performance management.
Demetra Nightingale, Institute Fellow, Urban Institute and Professorial Lecturer, Trachtenberg School of Public Policy and Public Administration, George Washington University; Former position with U.S. Government: Chief Evaluation Officer, U.S. Department of Labor.