Report 2020-114 Recommendation 1 Responses

Report 2020-114: California Air Resources Board: Improved Program Measurement Would Help California Work More Strategically to Meet Its Climate Change Goals (Release Date: February 2021)

Recommendation #1 To: Air Resources Board, State

To improve its ability to isolate each of its incentive programs' additional GHG reductions, by February 2022 CARB should establish a process to formally identify its incentive programs' overlap with other programs that share the same objectives. As part of that process, CARB should document how it will account for the overlap to allow the most accurate program measurement possible.

Annual Follow-Up Agency Response From October 2023

For over a decade, CARB has used multiple policy approaches, including incentives and regulations, to reduce greenhouse gas (GHG) emissions and transition to zero-emission vehicles. To respond to CSA's recommendation to separate GHG reductions of individual strategies, CARB executed a contract with researchers at the University of California specializing in environmental economics and policy. During the contract period, CARB met with the researchers biweekly to provide necessary program data for their investigation. Findings published in June 2023 concluded that both regulations and incentives led to significant and direct GHG reductions. However, participant survey data was insufficient to distinguish isolated incentive program impacts. Moreover, the report highlighted how incentive programs provided "second order" benefits beyond the direct benefits of additional zero-emission vehicle sales. For instance, incentives increased consumer demand, which spurred automaker investment, and by 2022 enabled CARB to set ambitious zero-emission sales targets that aligned with the Governor's Executive Order N-79-20 to phase-out combustion vehicle sales by 2035. Consequently, the report highlighted, "because the various ZEV programs work in concert, the role of any one policy is impossible to disentangle with precision from among various consumer and business decisions to purchase or lease ZEVs." CARB is thus unable to further refine its methodologies to isolate GHG reductions from its complementary programs. Nevertheless, if methods to approximate the individual contributions of incentive programs become available, moving forward, CARB will explore opportunities to further refine GHG quantification methodologies.

California State Auditor's Assessment of Annual Follow-Up Status: Partially Implemented

Although CARB contracted with UC Berkeley to attempt to identify the impact of overlapping regulatory and incentive programs, CARB acknowledges that it was unable to distinguish between incentive and regulatory impacts on emissions. Further, the report produced by UC Berkeley indicates that the research team is currently reviewing available data on ZEV purchases to better estimate the number of ZEVs that are purchased or leased as a direct result of regulatory and incentive programs.

The report notes that the incentive programs overlap with regulations, but does not indicate how CARB will account for the overlap to allow the most accurate program measurement possible and isolate the reductions attributable to each program.

We acknowledge that CARB has taken efforts to be responsive to the recommendation, but in the absence of a formal process for identifying and accounting for overlap, CARB has not addressed all aspects of the recommendation.

Annual Follow-Up Agency Response From October 2022

Meeting California's ambitious climate targets requires a mix of policy approaches, including regulatory mandates, incentives, consumer education, and infrastructure investment. Together, these complementary policies interact and influence consumer behavior and the vehicle market to achieve GHG emission reductions greater than single policies would achieve alone. In addition, the combination of policies that achieve GHG reductions also advance environmental justice and multiple environmental and social goals. Therefore, it is challenging to attribute emissions reductions to individual programs. Addressing the recommendation to establish a formal process will require additional data and analysis to estimate the impacts of individual components of CARB's integrated policy approach.

As mentioned in the One Year Update, to obtain the additional data and analyses that are needed to address CSA's recommendation, CARB executed a contract with a team of researchers at the University of California who specialize in technological innovation systems theory, behavioral research, and environmental policy. The contract was executed in December 2021 and will take approximately two years to complete. Since January 2022, CARB has been meeting with the researchers on a biweekly basis as they review California's portfolio policy approach related to transportation and assess the effectiveness of incentives in changing consumer behavior, achieving further GHG emissions reductions, and advancing social equity. CARB expects to receive an interim report in December 2022. In 2023, the researchers will develop quantification methods for existing consumer-based incentive programs and provide recommendations for long-term data collection methods and analysis tools to quantify the emission benefits of individual incentive programs.

California State Auditor's Assessment of Annual Follow-Up Status: Partially Implemented

1-Year Agency Response

Meeting California's ambitious climate targets requires a mix of policy approaches, including mandates, incentives, consumer education, and infrastructure investment. These policies induce further GHG emission reductions, advance environmental justice, and concurrently address multiple environmental and social goals. However, their interrelated operation and the complexity of consumer behaviors make it challenging to attribute emissions reductions to individual programs. Addressing this challenge requires additional data and analysis to separate the impacts of CARB's existing incentives on consumer behavior from regulations and other programs.

To address CSA's recommendation, CARB executed a contract with a team of researchers at the University of California who specialize in technological innovation systems theory, behavioral research, and environmental policy. The objective of this contract is to better define GHG emission benefits of individual CARB incentive programs. The researchers are reviewing California's portfolio policy approach related to transportation and assess the effectiveness of incentives in changing consumer behavior, achieving further GHG emissions reductions, and advancing social equity. The researchers will also develop quantification methods for existing consumer-based incentive programs and provide recommendations for long-term data collection methods and analysis tools to quantify the emission benefits of incentive programs. The contract was executed in December 2021 and will take approximately 2 years to complete. CARB began meeting with the researchers biweekly in January 2022 to discuss CARB's programs and provide regulatory and incentive program data necessary for the researchers' analysis. CARB expects to receive interim results in Summer 2022.

California State Auditor's Assessment of 1-Year Status: Partially Implemented

In addition to the the contract CARB references in its response, CARB provided evidence that it has been meeting with the researchers to provide them the information necessary to conduct their work. Therefore, although the substantive portions of this recommendation are still outstanding, CARB has demonstrated partial implementation.

6-Month Agency Response

Meeting California's ambitious climate targets requires a mix of policy approaches, including mandates, incentives, consumer education, and infrastructure investment. The complementary nature of these policies has the benefit of inducing further GHG emission reductions, advancing environmental justice in California's underserved communities, and concurrently addressing multiple environmental and social goals. However, this complementary nature and the complexity of consumer behaviors make it very challenging to attribute emissions reductions to each individual program. Addressing this challenge requires additional data collection and analysis on the impacts of CARB's existing incentives on consumer behavior aside from those resulted from regulations and other programs.

CARB is developing a contract with university researchers to review California's portfolio policy approach related to transportation and its effectiveness. The contract is intended to inform development of long-term data collection methods and analysis tools that can assess the effectiveness of incentives in changing consumers' behaviors and disentangle the emission benefits of incentives and other programs. Results from this contract will be used to develop methods that quantify the benefits of CARB's light- and heavy-duty consumer-focused ZEV incentive programs. The contract will likely take approximately 2 years to complete, and CARB expects to receive interim results and report the researchers' findings to CSA starting in Summer 2022.

California State Auditor's Assessment of 6-Month Status: Pending

60-Day Agency Response

Meeting California's ambitious climate targets requires a mix of policy approaches, including technology forcing measures like the ZEV mandate, incentives, consumer education, and infrastructure investment. The complementary nature of these policies has the benefit of inducing further GHG emission reductions that concurrently address multiple environmental and social goals, including providing benefits to the overall economy. To meet Recommendation #1, we will conduct further investigation through further data collection and analysis. We are committed to establishing a process to refine how we analyze the effect of each policy.

As part of this implementation, CARB plans to formally define objectives for each of its incentive programs and identify quantifiable metrics to assess progress towards these objectives. CARB plans to designate or hire additional staff with expertise in economics and data science to further study and evaluate potential overlaps among various incentive and regulatory programs. CARB also intends to commission an extramural contract to collect necessary information needed to attribute greenhouse gas emission and socio-economic benefits to each individual program. CARB staff will utilize data collected through this contract to inform the process and methodologies implemented by the agency moving forward. CARB anticipates needing additional resources to further enhance this process.

California State Auditor's Assessment of 60-Day Status: Pending

Fully implementing this recommendation may require CARB to collect data that is relevant to its program design that it does not currently collect. However, the primary purpose of the recommendation is to ensure that CARB recognizes, and accounts for, overlap among the key objectives of its incentive and regulatory programs. Our work identified many of these key objectives for the programs we reviewed, including where existing overlap is clear. Therefore, CARB should first focus on coordinating among its programs and divisions to identify and account for the objectives its programs are already pursuing. Our review did not identify any reason this process would require additional resources beyond those CARB already has in place to manage the design and implementation of its programs. If CARB believes such resources are necessary, it must clearly demonstrate why the process of implementing this recommendation requires personnel or expertise it does not currently have available with its existing resources.

All Recommendations in 2020-114

Agency responses received are posted verbatim.