Report 2017-117 Recommendation 10 Responses

Report 2017-117: Mental Health Services Act: The State Could Better Ensure the Effective Use of Mental Health Services Act Funding (Release Date: February 2018)

Recommendation #10 To: Mental Health Services Oversight and Accountability Commission

To ensure that the MHSA-funded triage grants are effective, the Oversight Commission should require that local mental health agencies uniformly report data on their uses of triage grants. It should also establish statewide metrics to evaluate the impact of triage grants by July 2018.

Annual Follow-Up Agency Response From October 2020

In May 2018, the Commission awarded thirty triage grants through three sets of Requests for Applications (RFA). Each of the grants require the local mental health agencies to collect and report data as part of the statewide evaluation of the triage grants.

In January 2019, the Commission contracted with the Behavioral Health Centers of Excellence (BHCE) at UC Davis and UCLA to perform the statewide evaluation of the triage grants. In July 2020, the BHCEs submitted to the Commission a Statewide strategy for metrics to evaluate the impact of triage grants. Over the next year, the Commission staff in collaboration with the BHCOEs will continue to update the strategy to ensure the feasibility and sustainability of the proposed metrics.

California State Auditor's Assessment of Annual Follow-Up Status: Partially Implemented

In July 2020, UC Davis and UCLA developed metrics for the Oversight Commission to evaluate impacts of triage grants. However, as the Oversight Commission indicated in its response, it will take the next year to evaluate the feasibility and sustainability of the proposed metrics. We look forward to reviewing the Oversight Commission's review and evaluation of the effectiveness of the proposed metrics.

Annual Follow-Up Agency Response From October 2019

In May 2018 the Commission awarded thirty triage grants through three sets of Request for Applications (RFA). Each of the grants require the local mental health agencies to collect and report data as part of the statewide evaluation of the triage grants.

The Commission has contracted with the Behavioral Health Centers of Excellence (BHCOE) at UC Davis and UCLA to perform the statewide evaluation of the triage grants. Under the contracts the BHCOE, prior to the end of 2019, are required to establish a clear strategy to evaluate the impact of the triage grants. Additionally, the BHCOE are to submit a draft evaluation plan in January 2020 setting forth the statewide metrics to evaluate the impact of the triage grants.

California State Auditor's Assessment of Annual Follow-Up Status: Pending

1-Year Agency Response

Recognizing the need for standardized data collection and evaluation, in January 2018, the Commission authorized $10 million to contract with a third party to do statewide evaluation of the triage grants. The evaluator will work closely with grantees and Commission staff to devise evaluation strategies including establishing statewide metrics.

The Commission awarded thirty triage grants through three sets of Request for Applications (RFA). Each of the grants require collaboration with the third party evaluator in collecting and reporting required data as part of the statewide evaluation of the triage grants.

In 2018 the Commission entered into negotiations with the Behavioral Health Centers of Excellence (BHCOE) at UC Davis and UCLA to perform the statewide evaluation of the triage grants and had anticipated finalizing the contract before July 1, 2018. However, as a result of the $12 million per year reduction in funding for Triage programs in the 2018/19 enacted budget the Commission, in July 2018, reduced the funding of each of the grants and the statewide evaluation contracts. This reduction in funding required further negotiations with the BHCOE. The Commission executed the contracts with the BHCOE in January 2019. Under the contracts the BHCOE, prior to the end of 2019, are required to establish a clear strategy to evaluate the impact of the triage grants.

California State Auditor's Assessment of 1-Year Status: Pending

6-Month Agency Response

Recognizing the need for standardized data collection and evaluation, in January 2018, the Commission authorized $10 million to contract with a third party to do statewide evaluation of the triage grants. The evaluator will work closely with grantees and Commission staff to devise evaluation strategies including establishing statewide metrics.

In May 2018 the Commission completed awarding thirty Triage grants through three sets of Request for Applications (RFA). Each of the grants require collaboration with the third party evaluator in collecting and reporting required data as part of the statewide evaluation of the triage grants.

Earlier this year the Commission entered into negotiations with the California Behavioral Health Centers of Excellence (CBHCE) at UC Davis and UCLA to perform the statewide evaluation of the triage grants and had anticipated finalizing the contract before July 1, 2018. However, as a result of the $12 million per year reduction in funding for Triage programs in the 2018/19 enacted budget the Commission, in July 2018, reduced the funding of each of the grants and the statewide evaluation contracts. This reduction in funding requires further negotiations with the CBHCE. The Commission anticipates finalizing the contracts by October or November 2018. Commission anticipates a clear evaluation strategy will be in place by July 2019.

California State Auditor's Assessment of 6-Month Status: Pending

60-Day Agency Response

During the initial triage grant phase the Commission required counties to propose an evaluation strategy as part of the application to ensure counties could evaluate their progress while also ensuring counties could tailor the program to their local priority needs. All grantees were required to submit periodic reports on hiring, encounters with mental health services, evaluation efforts, and expenditures. This approach allowed the counties to propose a triage program that is responsive to the unique priorities and could be integrated into their continuum of care. Recognizing the need for standardized data collection and evaluation, in January 2018, the Commission authorized $10 million to contract with a third party to do statewide evaluation of the triage grants. The evaluator will work closely with grantees and Commission staff to devise evaluation strategies including establishing statewide metrics.

The Commission issued three sets of Requests for Applications (RFA) for the triage grants. Each of the RFAs requires the grantee collaborate with the third party evaluator in collecting and reporting required data. On March 22, 2018 the Commission awarded triage grants for adult services and is scheduled to award the triage grants for children and transitional age youth services at its April 2018 meeting. The final set of grants which is for triage services in partnership with schools will be awarded in early May. The Commission anticipates the contracts for all grants will be fully executed in June 2018.

The Commission is in negotiations with the California Behavioral Health Centers of Excellence (CBHCE) at UC Davis and UCLA to perform the statewide evaluation of the triage grants and anticipates finalizing the contract by June 30, 2018. Commission anticipates a clear evaluation strategy will be in place prior to July 2019.

California State Auditor's Assessment of 60-Day Status: Pending

All Recommendations in 2017-117

Agency responses received are posted verbatim.