Excelling at Criterion 04: Evaluation Policy, Plan and Learning Agenda
Evidence-based policymaking is only possible after building evidence. Thanks in part to the Evidence Act, the 24 federal agencies named in the CFO Act of 1990 now have an evaluation officer charged with one side of that equation. However, federal agencies typically do not have an equivalent position for taking those findings from evaluations and putting them to work — and ultimately shifting taxpayer dollars to initiatives the research shows improve outcomes. That leaves this essential project to the creativity and persistence of other agency leaders not formally tasked with translating evidence into action.
One such leader with that foresight and fortitude is Dr. Mary Hyde, Director of Research and Evaluation at AmeriCorps. Under her leadership, in March 2024, AmeriCorps published its Strategic Learning and Evidence Building Plan: Fiscal Years 2022-2026. This blueprint lays out ways the agency will increasingly apply the findings from evaluations about its work to make its work even more effective.
Each Friday, AmeriCorps staff from its budget, evaluation and strategic planning teams meet to discuss how these three areas can be better integrated.
“This document created the opportunity to have a more intentional and specific conversation” about how AmeriCorps can tie its learning agenda to strategic planning, Dr. Hyde notes. Since the publication went live, a new Core Project team within a new Strategic Plan Steering Committee at AmeriCorps meets weekly. Each Friday, AmeriCorps staff from its budget, evaluation and strategic planning teams meet to discuss how these three areas can be better integrated — including how evidence can inform budgeting, strategic planning and grantmaking to achieve the resulting strategic goals. As a result of these standing meetings, the agency’s Budget Director has begun a new practice of asking AmeriCorps offices to identify the strategic goals their requested FY 2025 funds will advance, considering the learning that has flowed from prior evaluations. AmeriCorps is again walking the walk, operationalizing its commitment to use what it’s learning to benefit all Americans.
AmeriCorps first featured in the Federal Standard of Excellence in 2014. Its FY 2024 Discretionary Budget was $1.269 billion, the 10th-largest such budget of the 11 agencies in the 2024 Federal Standard of Excellence.
71
06.
AmeriCorps provides guidance and resources to support state and local grantees to evaluate programs and build the necessary data systems and capacity. An important way that it does so is through funding opportunities. The FY 2024 AmeriCorps State and National Competitive Grants Notice of Funding Opportunity (NOFO), among others, includes evaluation expectations. The agency also provides an evaluation plan template and expectations on the type of evidence to be collected. The FY 2025 NOFO similarly does so (see p. 14). Additionally, the agency’s FY 2025 RSVP NOFO demonstrates how AmeriCorps scores NOFOs based on use of evidence. AmeriCorps also encourages the use of evidence-based models in program designs. Finally, AmeriCorps provides technical assistance on evaluation capacity, such as around data collection, through a contractor and a range of standing resources provided to grantees.
As AmeriCorps’ designated evaluation officer, the director of the Office of Research and Evaluation (ORE) coordinates evaluation policy and the use of findings. The agency has a Research and Evaluation Council that meets monthly to assess the agency’s learning agenda and evaluation plan. Members of the Council include the director of ORE, the chief information officer/chief data officer, the chief of staff, the chief of program operations, and the chief operating officer.
AmeriCorps hired its first chief data officer (CDO) in FY 2021 to improve data governance, collection, sharing and analysis. Since then, the CDO has tackled longstanding data management priorities, including enhancing the agency’s data analytics capacity and establishing a structure to coordinate data integrity, performance and research efforts across AmeriCorps.
AmeriCorps has implemented a robust Data Governance Framework to guide its data management strategy, using a tiered and tailored approach. The Research and Evaluation Council and Strategic Advisory Board support this framework. The Strategic Advisory Board, led by senior executives, including the chief data officer, chief of staff, chief operating officer, chief modernization officer, chief program officer, and director of the Office of Research and Evaluation (ORE), provides leadership and guidance to advance data-driven decision-making. This board ensures AmeriCorps’ data initiatives align with the organization’s goals and stakeholder needs.
The Data Governance Council (DGC) serves as the collaborative core of the Data Governance Framework, co-chaired by the CDO and the Director of ORE. The DGC unites business units across AmeriCorps to foster consistent data definitions and manage data as “enterprise assets” that benefit the entire organization rather than remaining siloed within individual departments.
To further its evaluation agenda, AmeriCorps has invested 1.2% of its $975 million FY 2024 operating budget — about $12 million — in evaluation, evaluation technical assistance and evaluation capacity-building.
AmeriCorps has developed a Strategic Learning and Evidence Building Plan to define its learning agenda for FY 2022-26, reinforcing its commitment to using evaluation to shape policy and practice. This agenda addresses priority research questions through approximately 13 contracts, which support dozens of evaluations, and over 20 research grants.
In addition, AmeriCorps established an evaluation policy built on five guiding principles: rigor, relevance, transparency, independence and ethics. To enhance collaboration, staff from the agency’s budget, evaluation and strategic planning teams meet weekly to discuss strategies for better integrating these areas.
AmeriCorps has two grant programs that both define and prioritize the use of evidence of effectiveness, totaling $391 million and representing 45.4% of their grant program funding. This analysis includes all competitive grant programs and the five largest (in dollar amount) noncompetitive grant programs. These grants direct funds for evidence-based programs focused on a wide array of state priorities such as after-school programs, disaster relief, health services and building affordable housing.
Many of the remaining grants encourage or intend to build evidence. Practices that have not yet been documented to be effective can become evidence-based after an impact evaluation is conducted. Adding a definition of evidence and a mechanism to prioritize evidence in the grant can direct funds to those programs. Programs can benefit from a combination of well-established EBPs and innovative programming intentionally designed to meet emerging community needs.
AmeriCorps employs performance management practices to foster a culture of accountability focused on achieving equitable outcomes. The agency’s first goal in its FY 2022–26 Strategic Learning and Evidence Building Plan is to “partner with communities to alleviate poverty and advance racial equity.”
Reporting to the chief of staff, AmeriCorps’ strategic planning team oversees performance management. This team — which includes a member from the Office of Research and Evaluation as well as enterprise risk management specialist, chief data officer governance specialist, chief operating officer, and Office of Budget Management and Program analyst — meets quarterly with agency leadership to review progress toward strategic goals. Performance outcomes guide agency decisions, and the FY 2022–26 strategic plan, approved by the White House Office of Management and Budget, includes performance indicators. Additionally, the agency has developed an internal implementation tracker to assess progress toward its goals and objectives.
AmeriCorps demonstrates how performance data supports outcomes in underserved communities by analyzing how programmatic changes, like increased living allowances from the American Rescue Plan Act, impact participation in national service programs. By assessing the redesign of service experiences and using performance data, the agency can refine its strategies to build a diverse pipeline of service members. Additionally, evidence from grantmaking models shows that removing barriers, such as waiving cash match requirements, can improve access for community-based organizations in underserved areas. AmeriCorps will use this learning to improve its programs.