Excelling at Criterion 06: Strengthen State and Local Capacity
Historically, the U.S. Agency for International Development (USAID) often partnered with external parties based in the U.S. to conduct evaluations of USAID’s work abroad. However, learning what works, for whom and under what circumstances requires a greater understanding of context. That’s why USAID has made “localization” a key part of its approach to evaluation. The agency has successfully engaged local institutions in these evaluations based in the countries where USAID was working. But it recognized an unfulfilled remaining need — few were in leadership roles. Most local groups’ involvement was simply collecting data, for example.
USAID created the Local Evaluation and Evidence Support (LEES) mechanism this past year to help realize that opportunity. Awarded in September 2023, with four years of work launched the following month, LEES is building evaluation capacity on the ground in other countries by training local organizations to receive direct USAID funding to lead on USAID evaluation work in the future. “We want to ensure we are working with experts in the countries,” explained Dr. Winston Allen, Agency Evaluation Officer in the Office of Learning, Evaluation and Research at USAID’s Bureau for Policy, Planning and Learning. LEES will operate in four to six African countries, including Zambia, where 25 firms have already been selected and have completed training as of Summer 2024. LEES has also identified 25 such organizations in Ghana, which started training in October 2024. Importantly, some organizations have been involved in USAID evaluation work before, but some have never been.
USAID launched LEES through its 2020-2025 Evaluation, Monitoring, and Learning Services IDIQ (Indefinite Delivery, Indefinite Quantity) contract, created in 2010 when USAID established its Office of Learning, Evaluation, and Research. Capped at $655 million, this IDIQ allows USAID missions easy access to a pool of 19 firms engaged in evaluation work, easing the process of tapping evaluation capacity for the work of building evidence that USAID requires. The IDIQ vehicle helps account for the relative speed with which USAID has been able to advance this initiative.
We want to ensure we are working with experts in the countries.
Local feedback so far has pointed to significant enthusiasm for the initiative. In fact, in both Zambia and Ghana, more organizations sought to participate than were available in the spots. To be responsive to this demand, USAID is already working on making the training materials available through the USAID Learning Lab website so that as many organizations as interested can access training to take on increasing roles in evaluations of work in their own countries.
USAID first featured in the Federal Standard of Excellence in 2013. Its FY 2024 Discretionary Budget was $28.948 billion, the fifth-largest such budget of the 11 agencies in the 2024 Federal Standard of Excellence.
25
10.
USAID employs a “Pay-for-Results (PFR)” approach to development, in which the agency makes payments when implementing organizations achieve specific milestones or development results. PFR brings all stakeholders together from the beginning of the development activity to set performance metrics, helping to ensure that allocated funds achieve measurable results as cost effectively as possible. USAID has developed a guide offering guidance on setting, pricing and administering performance metrics in PFR programming; a PFR primer is also available here.
USAID also has Notices of Funding Opportunity (NOFOs) that requires procurement contracts to prioritize proposals addressing the needs of people experiencing unfavorable outcomes. An example is the USAID/Ghana NOFO, Teaching and Learning Materials and Classroom Furniture (TLM+) Activity. Its evaluation criteria state that USAID/Ghana will evaluate an applicant’s technical approach and year one work plan outline based on the extent to which they are sound, evidence-based and likely to meet the objectives and purpose set forth in the NOFO, including applying a gender, youth and environmental protection lens; using a behavior-led approach; and incorporating opportunities for learning and adapting.
USAID has a chief evaluation officer who sits on the agency’s Data Board and meets with the Operations Council and Privacy Council on an ad hoc basis. The chief data officer (CDO), chief evaluation officer, statistical official and others have standing weekly/monthly meetings. The CDO’s team and leadership from the Office of Learning, Evaluation and Research, which manages agency requirements related to performance monitoring, evaluation and organizational learning, also have a standing meeting. The CDO’s team maintains an internal dashboard that is shared with the evaluation officer and statistical official to help track progress against milestones on an ongoing basis.
One important way of encouraging evidence use in the federal government is to show leading examples of where this practice already is happening and how it contributes to positive impacts both for federal agencies and for the people they serve. To do so, agencies must have strong evaluation leadership, with an evaluation officer capable of advising on the design and implementation of evaluations, interpretation of results, and integration of findings into action—both inside and outside of their agencies. USAID demonstrated how an evaluation office can exercise this kind of leadership when they launched their new “Evidence to Action Briefs” in June 2023. (Read more on p. 28 of The Power of Evidence to Drive America’s Progress.)
The chief data officer (CDO) manages the USAID Data Services team, which focuses on improving the usage of data and information to ensure that the agency’s development outcomes are supported and enhanced by evidence. In September 2019, USAID established a Data Administration and Technical Advisory (DATA) Board. The DATA Board acts as USAID’s data governance body. It serves as a central venue for seeking input from agency stakeholders regarding data-related priorities and best practices to support agency objectives. It informs data-related policies, procedures and standards at the agency. The chief data officer, the chief evaluation officer, and the statistical official meet regularly (weekly/monthly). Additionally, the agency established a standing meeting with the chief data officer’s team and leadership from the Office of Learning, Evaluation and Research, which manages agency requirements related to performance monitoring, evaluation and organizational learning.
USAID invested at least $270.6 million in FY 2022, FY 2023 and prior year money on evaluations completed in FY 2023, evaluations that are ongoing during FY 2024, evaluation technical assistance, and evaluation capacity-building, representing 1.3% of the agency’s $20.6 billion FY 2023 budget.
USAID has an agency-wide Evaluation Policy that ensures compliance with the Foreign Aid Transparency and Accountability Act and the Evidence Act. The agency also has an annual evaluation plan and just released a Principles for Evidence Use resource, which outlines the key considerations USAID employs for generating and applying evidence. Finally, USAID has an agency Learning Agenda that articulates USAID’s priorities for learning and evidence to inform agency decision-making.
In order to strengthen state and local capacity through integrated data systems, USAID allows for data capacity and systems integration through multiple mechanisms including some government-to-government agreements. USAID’s government-to-government agreements include dedicated partner monitoring and evaluation capacity development efforts, such as training, mentoring and technical assistance, when improved partner capacity is needed to achieve development results. Additional information can be found here. USAID requires some form of performance evaluation in all of its projects (see Core Tools for Program Effectiveness). Additionally, USAID funded improving institutional capacities and increasing the sustainability of civil society organizations in Mexico. The program included capacity-building assistance with monitoring, evaluation and learning.
To further support and strengthen local capacity for integrated data systems, USAID is working to support and improve local evaluation in multiple ways, including via a range of technical support and assistance for evaluation. Some examples of USAID’s technical assistance are documented here. USAID also provides training to grantees to use evidence in designing and making implementation decisions. An example can be found here. In addition, USAID recently awarded a four-year Local Evaluation and Evidence Support (LEES) contract that identifies networks of host country research and evaluation entities that USAID, other international organizations, and local governments can engage for evaluation and learning activities, strengthening their technical and organizational capacity; helping them market their services to potential clients; and supporting them as they begin to conduct evaluations and learning activities.
USAID engages communities, particularly those that are historically underserved, in various ways. First, USAID’s Policy on Promoting the Rights of Indigenous Peoples aims to strengthen the agency’s engagement with indigenous communities to improve the impact and sustainability of our development programs. Second, USAID’s Knowledge Management and Organizational Learning (KMOL) Policy, which was updated in 2024, emphasizes community engagement and locally-led development. Third, Section III of USAID’s Learning Agenda, “Consulting Stakeholders to Define New Learning Questions,” describes how stakeholders are engaged to develop the Learning Agenda. For example, through a series of over 20 consultative sessions and subsequent question validation discussions, internal stakeholders provided input on learning priorities and critical evidence needs representing views from across the sectors and geographies in which USAID works. Finally, the agency has developed a guide for collecting feedback from beneficiaries of USAID programs.
Another way in which the agency has engaged communities is through its technology and data plans. USAID has an Information Technology Strategic Plan (ITSP) that was developed with a variety of stakeholders, including NGOs serving communities “on the ground” who have experienced unfavorable outcomes. The ITSP describes the process of including stakeholders in the development of the plan. In addition, USAID recently revised its Public Access Plan (PAP), which supports open data through public access to federally funded research. Throughout the revision process, USAID participated in the National Science Technology Council’s Subcommittee on Open Science to incorporate indigenous knowledge and tribal sovereignty considerations into PAP and other plans and policies related to open science and open data. Finally, USAID seeks to engage groups served by the program, implementing partners and other stakeholders in evaluation designs. To aid in this process, the agency has developed a resource operating units can use for identifying and engaging stakeholders in the evaluation process. USAID also uses Developmental Evaluation (DE) as one of its performance evaluation approaches. DE includes the sustained inclusion, participation and investment of a broad cross-section of stakeholders affected by an intervention. For example, USAID used DE to evaluate Jalin, an activity in Indonesia designed to catalyze local solutions to improve maternal and newborn health outcomes.