Substance Abuse and Mental Health Services Administration (SAMHSA) at HHS

SAMHSA has two leaders coordinating evaluation policy and findings: the director of the Center for Behavioral Health Statistics and Quality as the evaluation lead and the director of the Office of Evaluation (OE) as the evaluation officer. The OE is central to SAMHSA’s evaluation governance, overseeing program evaluations in partnership with program centers or offices. OE also works collaboratively with the National Mental Health and Substance Use Policy Laboratory to provide support for SAMHSA’s evaluations by overseeing the identification performance indicators to monitor SAMHSA programs. OE similarly developed and maintains an evaluation repository and supports Evidence Act initiatives, including the development of a learning agenda, an evaluation repository, and an agency Capacity Assessment.

Recent notable achievements include an Evaluation of SAMHSA Programs and Policies, as well as Evaluation Plans for FY 2023 and FY 2024. Additionally, SAMHSA established the SAMHSA Evidence and Evaluation Board (SEEB), complete with a charter and confirmed voting members, and updated the Evaluation of SAMHSA Programs and Policies guidance document.

SEEB’s purpose is twofold. It serves as SAMHSA’s principal evaluation and evidence forum for managing its evaluation portfolio and its evaluation and evidence data. It is a strategic asset to support the agency in meeting its mission and agency priorities, including implementation of the Evidence Act. SEEB coordinates activities of the evaluation officer, chief data officer, statistical officer and performance improvement officers (in all centers and offices) and provides a structured environment to pursue alignment with the framework offered by the Evidence Act. SEEB also serves as the mechanism to both generate and disseminate knowledge and best practices, while providing a forum for the agency to reach consensus on issues pertaining to evaluation and evidence.

The director of the agency’s Center for Behavioral Health Statistics and Quality (CBHSQ) serves as the chief data officer, overseeing SAMHSA’s survey and surveillance datasets and playing a key role on the SAMHSA Evidence and Evaluation Board (SEEB).

SAMHSA’s 2023-2026 Data Strategy outlines the creation of a Data Governance Board (DGB) to implement and monitor the success of the strategy. Led by CBHSQ, the DGB includes experts from each of SAMHSA’s Centers and Offices and aligns with SAMHSA’s Strategic Plan, the U.S. Department of Health and Human Services’ (HHS) Strategic Plan and Data Strategy, as well as federal laws and executive orders. The DGB will aims to improve the accessibility, quality and reliability of SAMHSA’s data, with a core focus on fostering intra-agency collaboration and innovation in data use and investment.

SAMHSA invested $133.7 million in activities related to evaluation and data in FY 2024, which represented 1.7% of the agency’s $7.4 billion budget.

The agency has an evaluation policy, an annual evaluation plan and a learning agenda to support the building and use of evidence. In FY 2022, SAMHSA developed and approved an evaluation policy document that provides guidance for evaluations of programs and policies. Evaluation reports are publicly available.

SAMHSA also contributes to the U.S. Department of Health and Human Services’ (HHS’) Evaluation Plan. The FY 2024 HHS Evaluation Plan includes evaluations that are planned to continue or start in FY 2024. Although they do not represent all of the evaluations expected to be conducted by HHS, each evaluation contributes to HHS’ ability to answer the priority questions presented in the current Evidence Building Plan, which ties directly to the current HHS Strategic Plan. HHS uses diverse data sources, methodological approaches and dissemination plans to address the complex and varied health and human services needs of the populations it serves.

A SAMHSA-specific learning agenda is forthcoming. Additionally, as an agency within HHS and an active participant in the HHS Evidence and Evaluation Policy Council, SAMHSA contributed to the HHS learning agenda, participated in monthly meetings and contributed to developing the Evidence Building Plan.

SAMHSA is dedicated to making agency data open and available to the public, balanced with the need to respect and ensure privacy. Data collections featured are DAWN (Drug Abuse Warning Network), N-MHSS (National Mental Health Services Survey), TEDS (Treatment Episode Data Set), MH-CLD (Mental Health Client-Level Data), N-SSATS (National Survey of Substance Abuse Treatment Services), NSDUH (National Survey on Drug Use and Health) and URS (Uniform Reporting System). As a Federal Statistical Unit, SAMHSA adheres to the publicly available Statement of Commitment to Scientific Integrity by Principal Statistical Agencies. To promote ethical use of sensitive SAMHSA data, restricted-use data are available via Research Data Gov, a web portal shared by Federal Statistical Agencies for discovering and requesting access to restricted microdata.

SAMHSA’s 2023-2026 Data Strategy focuses on equity and aligns with the agency’s 2023-2026 Strategic Plan. (Two of the four core principles in the Strategic Plan are Equity and Commitment to Data and Evidence, which are integrated throughout the agency’s activities.) The Office of Evaluation (OE) within the Center for Behavioral Statistics and Quality (CBHSQ) developed the Advancing SAMHSA Programs in Reaching Equity (ASPIRE) dashboard to provide senior leaders with easily accessible data and feedback on reaching equity around SAMHSA program participants. The U.S. Department of Health and Human Services (HHS) evaluation officer included the ASPIRE dashboard as a case study in an upcoming HHS toolkit. The ASPIRE case study is a collaboration between the Division of Evidence, Evaluation and Data Policy (DEEDP) in the Office of the Assistant Secretary for Evaluation and Planning and the ASPIRE team members from SAMHSA’s OBHE and CBHSQ.

The agency is committed to making its data publicly accessible. The Substance Abuse and Mental Health Data Archive (SAMHDA) is a one-stop shop for SAMHSA public use data, including online analytic capabilities and downloadable datasets. The agency also has an online data analysis system dedicated to its highest-impact studies. Finally, SAMHSA has established standardized data-sharing processes. Its data-sharing agreement, available upon request from the SAMHSA data privacy officer, is used with contractors who work with or have access to sensitive client-level data. Along with other Federal Statistical Agencies, SAMHSA is currently working to further streamline and standardize public data access processes. The Evidence Act requires the establishment of a Standard Application Process. SAMHSA is diligently working with peer agencies to implement this requirement. Currently, selected restricted-use data are available upon completion of an application and review.

SAMHSA has policy documents and Notices of Funding Opportunity (NOFOs) stating that grant funds can be used to improve data system integration and infrastructure. For example, this FY 2023 NOFO (p. 25). The agency allows grantees to allocate funds to enhance data capacity or integrate data systems, in line with award terms and the Negotiated Indirect Costs Rate Agreement.

Program evaluation is explicitly permissible in FY 2023 NOFOs such as the Cooperative Agreement for the Refugee and Migrant Behavioral Health Technical Assistance Center and is a requirement of other NOFOs, such as this. SAMHSA’s Center for Mental Health Services (CMHS) provides Community Mental Health Services Block grant recipients with technical assistance (TA) to support evaluation, data collection and reporting, and the use of evidence-based programs. The agency’s Evidence-Based Practices Resource Center (EBPRC) offers free TA resources that provide guidance on improving data capacity and integrating data systems. Based on internal agency deliberations, informal community feedback and responses to a Request for Information released on September 7, 2023, the EBPRC is adding new features, such as a program review module and implementation and capacity-building resources. SAMHSA also provides TA on evaluation capacity through entities such as the Service Members, Veterans and their Families Technical Assistance (SMVF TA) Center and the Disaster Technical Assistance Center (DTAC).

All the agency’s grant programs have access to TA, including assistance related to data capacity and integrating data systems, and evaluation and performance management for each specific grant program. Depending on the program, TA is provided through a dedicated TA center (an example is here) and/or the grants project officers for that program.

SAMHSA has 44 grant programs that both define and prioritize the use of evidence of effectiveness, totaling $2.6 billion and representing 51.8% of their grant programs. This analysis includes all competitive grant programs and the five largest (in dollar amount) non-competitive grant programs. These grants direct funds to evidence-based interventions in areas such as opioid use, children’s mental health and behavioral health more broadly.

Many of the remaining grants encourage or intend to build evidence. Practices that have not yet been documented to be effective can become evidence-based after an impact evaluation is conducted. Adding a definition of evidence and a mechanism to prioritize evidence in the grant can direct funds to those programs. Programs can benefit from a combination of well-established EBPs and innovative programming intentionally designed to meet emerging community needs.