Report 2020-114 Recommendation 11 Responses

Report 2020-114: California Air Resources Board: Improved Program Measurement Would Help California Work More Strategically to Meet Its Climate Change Goals (Release Date: February 2021)

Recommendation #11 To: Air Resources Board, State

To better demonstrate the socioeconomic benefits that its incentive programs achieve, by February 2022 CARB should do the following:

- Identify clear and measurable metrics it will use to assess each of the socioeconomic benefits it intends its programs to achieve.
- Develop a process to collect data, or use existing data, to measure and report on each metric.
- In its funding plans and annual reports, CARB should report to the Legislature and its board on the metrics.

Annual Follow-Up Agency Response From October 2023

In the FY 2022-23 Funding Plan for Clean Transportation Incentives, CARB identified metrics for projects mentioned in Recommendation 10. Since then, CARB and its grantees have developed standardized data collection processes including participant surveys, user testimonials, and vehicle telematics. The metrics CARB identified include:

Clean Cars 4 All (CC4A) - Increase vehicle reliability measured by future disruptions of service based on the model year of the retired vehicle and that of the incentivized vehicle, and improve access to employment, goods, and services based on user surveys and testimonials.

Financing Assistance for Lower-Income Consumers - Improve ability of low-income or disadvantaged community members to purchase a clean vehicle using a low-interest loan, measured by the number of clean vehicles purchased by low-income consumers and the number of loans facilitated by the program.

Clean Mobility Options - Improve access to goods, services, and workforce training and development measured through participant surveys and vehicle telematics data.

Clean Mobility in Schools - Improve students' access to schools and transportation cost savings made possible by investments.

Sustainable Transportation Equity Project (STEP) - Improve priority populations' access to key destinations and transportation cost savings made possible by investments.

All active projects are now collecting data. The new Financing Assistance and Statewide CC4A program launches in early 2024 as a needs-based model and new metrics will be added. Staff plan to report on the analysis of these metrics as part of the FY 2024-25 Funding Plan process where robust data is available.

California State Auditor's Assessment of Annual Follow-Up Status: Partially Implemented

Annual Follow-Up Agency Response From October 2022

CARB continues to implement changes in response to this recommendation. In the proposed FY 2022-23 Funding Plan for Clean Transportation Incentives, CARB identified clear and measurable metrics for projects mentioned in Recommendation 10. CARB and project grantees are developing standardized data collection processes through participant surveys, user testimonials, and vehicle telematics. Metrics for each project identified in CARB's response to Recommendation 10 include:

Clean Cars 4 All - Increased vehicle reliability measured by future disruptions of service based on the model year of the retired vehicle and that of the incentivized vehicle, and improved access to employment, goods, and services based on user surveys and testimonials.

Financing Assistance for Lower-Income Consumers - Improved ability of low-income or disadvantaged community member to purchase a clean vehicle, using a low-interest loan, without having a vehicle to retire.

Clean Mobility Options - Improved access to goods, services, and workforce training and development opportunities measured through participant surveys and vehicle telematics data.

Clean Mobility in Schools - Improve students' access to schools and transportation cost savings made possible by investments.

Sustainable Transportation Equity Project - Improve priority populations' access to key destinations and transportation cost savings made possible by investments.

In future Funding Plans, starting with the FY 2023-24 Funding Plan, staff will report on the analysis of these metrics for each project where robust data is available.

California State Auditor's Assessment of Annual Follow-Up Status: Partially Implemented

We agree that CARB has identified the expected benefits for the programs in question under recommendation #10. We will continue to work with CARB to assess the data it collects from these programs and the conclusions it draws from such data about the programs' effectiveness.

1-Year Agency Response

In the FY 2021-22 Funding Plan for Clean Transportation Incentives published on October 8, 2021 and approved by CARB's Board on November 19, 2021, CARB identified clear and measurable metrics for projects mentioned in Recommendation 10. CARB and project grantees are developing standardized socioeconomic benefit metrics and implementing data collection processes through participant surveys and testimonials and vehicle telematics. Examples of metrics for each project identified in CARB's response to Recommendation 10 include:

Clean Cars 4 All - increased vehicle reliability measured by future disruptions of service based on the model year of the retired vehicle and that of the incentivized vehicle, and improved access to employment and goods and services based on user surveys and testimonials.

Financing Assistance for Lower-Income Consumers - increased number of clean vehicles in disadvantaged communities based on the increased number of cleaner vehicles purchased by disadvantaged community residents utilizing the program.

Clean Mobility Options - improved access to goods and services and workforce training and development opportunities measured through participant surveys and vehicle telematics data.

Clean Mobility in Schools - improved visibility and acceptance of zero-emission mobility options based on school district staff feedback and vehicle telematics.

Sustainable Transportation Equity Project - travel cost savings based on number of subsidies, value of each subsidy, and average fare of each clean transportation service.

These are described further in the FY 2021-22 Funding Plan. In future Funding Plans starting with the FY 2022-23 Funding Plan, staff will report on the analysis of these metrics for each project.

California State Auditor's Assessment of 1-Year Status: Partially Implemented

Please see our assessment of Recommendation #10, which also concerns the metrics discussed in this recommendation.

6-Month Agency Response

CARB staff has identified clear and measurable metrics for each of the projects mentioned in Recommendation 10 and will include those in the FY 2021-22 Funding Plan which will be published in October 2021. The FY 2021-22 Funding Plan will also include the process developed for each project to collect additional data or use existing data to measure and report on each metric that has been identified. In future Funding Plans, staff will report on the analysis of these metrics for each of the projects.

California State Auditor's Assessment of 6-Month Status: Partially Implemented

CARB provided evidence that it has begun identifying metrics by which it plans to measure the socioeconomic benefits its programs are intended to achieve. We will continue to assess CARB's progress in implementing this recommendation when CARB submits its one-year response.

60-Day Agency Response

As mentioned above, CARB staff is currently evaluating incentive programs that provide socioeconomic benefits as cobenefits. CARB staff will review current program data that is collected and will use a public process for the Funding Plan to help identify clear and measurable metrics including how the data will continue to be collected, analyzed, and measured. CARB will include information on these metrics in future funding plans and/or annual reports, as appropriate.

California State Auditor's Assessment of 60-Day Status: Pending

All Recommendations in 2020-114

Agency responses received are posted verbatim.