THE AUTHORITY CAN IMPROVE THE QUALITY AND TRANSPARENCY OF ITS MONITORING AND REPORTING FOR KEY GOALS
As one of the most expensive transportation projects in the United States, the construction of the high‑speed rail system has significant implications for the State's environment, its small and disadvantaged businesses, and its economy. However, we identified several ways that the Authority can better measure and report on these impacts. The Authority's sustainability policy has relevant goals to limit the negative environmental impacts of the high‑speed rail system, but it does not sufficiently focus on the environmental impacts of construction activities. Additionally, the Authority has not collected complete and accurate data on the environmental impact of its construction activities, and thus far it has not comprehensively measured construction impact trends and set targets for future construction. The Authority reports consistently on its contracting with small, disabled veteran, and disadvantaged businesses, having set goals for the percentage of its total expenditures that will go to those businesses. However, its reporting could be more complete and transparent, as that reporting has omitted $930 million in contracts. In contrast, although the Authority faces some limitations that may affect the precision with which it estimates the overall economic impact of its activities in the Central Valley and elsewhere, it has appropriately disclosed these limitations as part of its reporting.
The Authority Can Better Account for the Environmental Impact of the System's Construction by Strengthening Its Sustainability Policy, Monitoring, and Measurement
According to its sustainability policy, the Authority intends its approach to the design, construction, and operation of the high‑speed rail system to contribute to a more sustainable California. The policy also states the Authority's commitment to employing leading edge construction methods to make the project a model for future rail infrastructure. Described by the Authority as "all‑encompassing," the sustainability policy is supposed to guide the system's energy and natural resource use, impact on local communities, construction practices, and operations. State and federal commitments mandate certain aspects of the Authority's focus. For example, Proposition 1A—which provided funding for the high‑speed rail system—requires the Authority to plan and construct the system in a manner that minimizes the impact on the natural environment. Further, in a 2010 memorandum of understanding with the federal government, the Authority pledged to be environmentally conscious throughout the design, construction, and operation of the system. Other aspects of the Authority's sustainability policy, such as encouraging transit development in local communities, are compatible with legislative priorities for sustainable transportation planning.
Although the Authority's sustainability policy includes goals consistent with best practices, it also has shortcomings that limit its effectiveness. According to the expert we retained to assist us in assessing the sustainability policy, it includes valuable objectives that generally align with established best practices. However, our expert found the policy does not sufficiently distinguish between construction of the system—which has a significant impact on the State's environment—and its eventual operation. Some of the policy's priorities, such as reducing car and other vehicle travel, clearly focus on the effects of system operations and have no direct construction implications. However, the Authority has not identified construction‑related objectives for all of its priorities even though the priorities themselves have relevance during the construction stage. For example, although the policy lists conservation of nonrenewable energy as a priority, the related objectives pertain only to the system's operations. Our expert noted that best practices concerning this priority, including those from the Global Reporting Initiative—with whose standards the Authority claims compliance—require a project to consider the total amount of energy used during construction.6
Our expert observed that because the policy does not consistently and explicitly address the impacts caused by the construction phase of the system, the Authority's implementation plan—which details how it will assess compliance with the policy—is not always specific about what the Authority should measure during construction in order to determine success. Further, the implementation plan does not include measurable, process‑focused metrics related to construction for many of the objectives. For example, the plan states that the Authority will monitor the degree to which the system's eventual operation improves air quality by tracking the number of emergency room visits for asthma sufferers; however, the plan does not include a metric to measure the degree to which the construction affects current air quality. Our expert concluded that this lack of actionable detail makes ensuring the system's current and future compliance with the policy's goals challenging.
Because construction is significantly underway in the Central Valley, we asked our sustainability expert to review best practices for monitoring sustainable construction and compare those practices to the Authority's plans and actions. After reviewing 13 comparable infrastructure projects as well as guidelines published by the American Public Transportation Association, our expert determined that current industry standards call for organizations to estimate the material impacts resulting from construction before beginning projects. Organizations should then establish specific goals and—once construction has begun—measure actual progress against those goals to determine where they have been successful.
The Authority has set initial estimates for some construction impacts, but it has not comprehensively measured actual progress against those estimates. Before beginning construction on the system, the Authority estimated the level of greenhouse gases and other pollutants that construction activities would emit. Our expert reviewed the Authority's calculations and found them to be reasonable and in line with guidance from the California Air Resources Board. Further, the Authority required each of its construction contractors to submit estimates of impacts, such as greenhouse gas emissions, within sixty days of receiving approval to begin work. Relying on contractors to determine these estimates presents a risk that they may overestimate emissions, but our expert concluded that doing so is a standard industry practice.
Although it properly completed initial estimates, the Authority did not ensure the accuracy of subsequently collected sustainability data. The Authority relies on each of its construction contractors to self‑report information on their sustainability performance—such as their usage of heavy equipment and water—into a central database. The oversight firms for the three current construction projects then review these submissions and pass them on to the Authority's sustainability unit, which RDP consultants lead and almost completely staff. Despite these protocols, when we attempted to validate a selection of nine database entries against supporting documentation, we found that the Authority could provide sufficient supporting documentation for only three entries. We therefore were unable to determine the accuracy of the Authority's data. An inability to ensure accurate data could limit the Authority's ability to reliably compare actual performance against estimates.
Further, our expert found that the Authority has not comprehensively evaluated the sustainability performance of the currently active construction projects. Neglecting to monitor all pertinent aspects of performance continuously throughout construction could result in the individual construction projects falling short of their goals; if it reviews progress only after it completes a specific project, the Authority will have missed any opportunity to intervene in order to improve sustainability outcomes. The Authority provided documentation showing that it is tracking two environmental impacts, greenhouse gas emissions and other air pollutants, against benchmark estimates. However, that documentation did not include equivalent comparisons related to the environmental impact of waste produced from construction, despite the fact that each construction contractor provided a benchmark estimate for waste. In addition, our expert observed that the documentation the Authority provided also does not allow it to effectively project whether contractors will meet or exceed their estimates because it does not account for actual construction progress to date. Finally, we noted that the current construction contracts do not require contractors to estimate the environmental impact of water usage for the project, and as a result, construction contractors did not provide benchmark estimates for water use.
Evaluating the sustainability impact of the system's construction on an ongoing basis is also critical because it will enable the Authority to set standards for future construction. The Authority has not yet developed a systemwide baseline or identified a universal metric against which to anchor future construction projects, which may differ in scope and type from current projects. For example, the Authority's plans for completing the system call for significant tunneling, which it has not yet attempted. Although its sustainability report from 2016 stated that the Authority planned to adopt 2015 as its baseline year, the Authority has not moved forward with this plan. Therefore, it is not yet prepared to hold future construction contractors to a baseline it has established using current construction activities. The Authority's sustainability director told us that her team is in the process of using recently completed systemwide construction plans and current construction data to develop a metric that will allow the Authority to set standards for future projects. She stated that the Authority plans to complete this process before it enters into any additional construction contracts, but that the process is complex and will be challenging.
Although the Authority Reports Regularly on Its Utilization of Small, Disabled Veteran, and Disadvantaged Businesses, It Excludes $930 Million in Contracts From Its Reporting
The Authority's Small Business
Small Business: State law defines small businesses as independently owned and operated businesses that are located in California and whose officers also live in California. Small business must have 100 or fewer employees and average annual gross receipts of $10 million or less. Of these small businesses, a business is a microbusiness if it has 25 or fewer employees and average gross receipts of $2.5 million or less.
Disabled Veteran Business Enterprise (DVBE): State regulations define DVBEs as businesses that are at least 51 percent owned and controlled by one or more disabled veterans who live in California.
Disadvantaged Business Enterprise (DBE): Federal regulations define DBEs as for-profit, small businesses that are at least 51 percent owned and controlled by one or more individuals who are both socially and economically disadvantaged.
Source: California Government Code section 14837(d); California Code of Regulations, title 2, section 1896.81; and Federal Code of Regulations title 49, section 26.5.
State law, regulations, and policy, as well as federal regulations, mandate that the Authority report on its contracting activity with small businesses, disabled veteran owned businesses, and disadvantaged businesses. In particular, executive orders require certain state agencies, including the Authority, to establish processes to meet a small business participation goal of 25 percent and a Disabled Veteran Business Enterprises (DVBE) participation goal of 3 percent. Additionally, as a condition of receiving federal financial assistance from the Federal Railroad Administration, the Authority must report on its actual use of Disadvantaged Business Enterprises (DBEs), with a goal of 10 percent utilization. The Authority has established its own small and disadvantaged business enterprise program (small business program) that, as the text box describes, includes small and microbusinesses, as well as DVBE and DBE firms. The Authority established an overall 30 percent participation goal for its small business program participants, inclusive of the 10 percent DBE goal and 3 percent DVBE goals.
To determine the extent to which the Authority contracts with these types of businesses, we reviewed the small business utilization reports (utilization reports) that the Authority posts on its website and provides to the Federal Railroad Administration. These reports include the Authority's actual utilization rates for 37 of its professional services contracts as well as its three construction contracts, with a total contract value for all 40 contracts of just over $4 billion. When we reviewed 10 of these contracts, we found that the documentation the Authority had collected from contractors sufficiently supported the utilization that the contractors reported. The documentation also supported the utilization percentages that the Authority reported for the six of these contracts that were active at the time of our review. In its most recent report, which it issued in June 2018, the Authority reported a nearly 30 percent actual utilization rate—inclusive of small businesses, DVBE firms, DBE firms, and microbusinesses—for its professional service contracts and a utilization rate of almost 16 percent for its construction contracts.
Although both state and federal programs require reporting, neither of their reporting formats fully capture the extent of the Authority's contracting activities with relevant businesses. For example, in its role administering the State's small business and DVBE programs, the Department of General Services (DGS) instructs state departments to exclude contracts with federal and state entities, as well as contracts with any counties or cities. Furthermore, DGS requires reporting only on the dollar amounts awarded to small businesses and DVBE firms, rather than actual expenditures; thus, its reporting requirements do not provide the Authority's actual impact on these businesses. In contrast, the federal Department of Transportation and the Federal Railroad Administration instruct the Authority to report on commitments, awards, and payments to DBE firms for federally funded contracts. According to the Authority's contract compliance administrator, its quarterly utilization reports list active contracts that are at least partially paid for with federal funding and that are subject to small business utilization goals. However, the utilization report does not include federally funded contracts for highly specialized services, such as expert witness services, for which the Authority has asserted it cannot find small business contractors.
We reviewed the Authority's utilization report and found that it excluded $930 million of the Authority's contracts. As of January 2018, two‑thirds of the contracts that the Authority excluded were with public entities, such as California cities and counties. These contracts represent $627 million in contract value and $326 million in actual expenditures. The utilization report also excludes $303 million of contract value for services from private entities. Given the limitations and exclusions inherent in each reporting format, the Authority does not currently release a report that includes its total small business program utilization out of its total contracting dollars.
Because the Authority does not mention that it exempts contracts with public entities on its reports—including its quarterly utilization reports, business plans, and economic impact reports—we asked the Authority about its reasons for not reporting on these contracts. The Authority's recently appointed chief administrative officer explained that its contracts with public entities and utilities generally do not contain small business provisions primarily because they are not the result of a competitive bidding process. Instead, the Authority enters into contracts with these entities because of specific program needs that only the public entities or utilities can serve. An example is contracting with a city or a county for permitting activities in its jurisdiction. In such instances, the Authority has no bargaining power to compel the public entities to participate in its business utilization program. The chief administrative officer also stated that DGS specifically instructs state entities not to include these types of contracts in their small business and DVBE reporting, as we previously describe.
Nonetheless, we believe that the Authority could better account for its contracts with these public entities in its reporting. For example, Caltrans—with whom the Authority has a $290 million contract—participates in the Authority's small business program. However, the Authority does not include this contract on its utilization report. When we asked the Authority why it does not include the Caltrans contract on its utilization report, a contract compliance administrator stated that it had not received utilization reporting from Caltrans on a consistent enough basis to allow the Authority to confidently include the agency in its reporting. The contract compliance administrator asserted that as it receives more information from Caltrans, the Authority will incorporate Caltrans' small business utilization rates into its reports, as well as monitor Caltrans' small business utilization through to the contract's completion.
The Authority provided a different reason for not reporting on all of its contracts with private entities. Although it is required to include all of its contracts with private entities on its quarterly utilization reports, those reports have not included $303 million in such contracts. The chief administrative officer stated that the Authority exempts certain contracts because the scopes of the work are too specific for it to require the contractors to be small, DVBE, or DBE businesses or to subcontract with such businesses. For example, many of the contracts omitted from the report are for expert witness or legal services. The chief administrative officer also explained that partly due to our inquiries, she realized that the Authority has no written policy explaining these exemptions or the process by which it determines whether to include these provisions in its contracts. She stated that she intends to oversee the development of such a policy.
Although there are few consequences if the Authority or its contractors do not meet their utilization goals, the emphasis on reporting in both state and federal requirements suggests that transparency and accountability related to contracting practices are key principles of the business utilization programs. However, by limiting the contracts it includes in its public reporting, the Authority is not accurately reflecting the proportion of its total expenditures that go to these businesses. We believe that given the magnitude of the Authority's contracting, the public would be best served by a higher degree of transparency in its reporting. An important part of increasing that transparency would be for the Authority to disclose both the total value of its contracts as well as the extent to which it has exempted contracts from small business requirements. The Authority's reasons for making those exemptions—whether its own prospective policies or state guidelines—would be an important part of that disclosure.
The Authority Followed Industry Standards When Estimating Its Economic Impact and Has Adequately Disclosed the Limitations of Those Estimates
The Authority used two widely accepted economic modeling programs to measure the total economic impacts of its spending from its contracting and construction activities. In September 2017, the Authority issued a retrospective report regarding the total economic impact from its spending activities from fiscal years 2006–07 through 2015–16, which it estimated to be between $3.5 billion and $4.1 billion. In its 2018 business plan, it updated this information to include fiscal year 2016–17, estimating that its economic impact increased an additional $1.6 billion to $1.8 billion during this time.
Although we identified some inconsistencies between the data that the Authority used for its economic modeling and the documentation supporting those data, these inconsistencies were relatively minor. To determine the accuracy of the Authority's data, we randomly selected and reviewed 58 data entries, including expenditure amounts and geographic locations at the zip code level, and we verified the information's accuracy by comparing it to the original invoices or other expenditure data sources. We identified certain inconsistencies: for example, the Authority attributed some expenditures to incorrect counties, which affected the Authority's estimations of jobs in a particular county. In other instances, the Authority explained that due to incomplete expenditure information, it used higher‑level financial or past geographical data to make assumptions about the size of expenditures, which could affect the precision of the dollar amounts it reported.
Nonetheless, given the magnitude of the total amounts that the Authority reported and the fact that the amounts are intended to be estimates, we are not concerned that the Authority's economic impact reporting is misleading or substantially under‑ or overstated. Further, the Authority disclosed the assumptions it made when designing its methodology by discussing those assumptions in the public technical memorandum that accompanied its reports.
To help improve the effectiveness of its sustainability policy, the Authority should revise the policy by May 2019 to more clearly differentiate between the construction and operation phases of the high‑speed rail system. Further, it should ensure that each objective in each section of the policy is associated with quantifiable metrics for evaluating implementation.
To allow it to evaluate the sustainability of the high‑speed rail system's construction, the Authority should, by May 2019, perform and document a review of its compliance with its existing quality controls related to ensuring the validity and completeness of contractor‑reported data. The Authority should also establish a formal process to perform such reviews periodically.
To help ensure that it meets its sustainability goals, the Authority should comprehensively compare the three construction projects' performances to their construction contractors' original baseline estimates on a quarterly basis. It should perform the first of these comparisons no later than May 2019.
To help ensure that its contractors' proposed environmental impacts are reasonable and to measure the progress of its sustainable construction efforts over time, the Authority should, by November 2019, identify and track standardized measures—such as project miles—that will allow it to compare construction impacts across the high‑speed rail system's different construction projects.
To increase the transparency of its reporting, the Authority should, by May 2019, expand its quarterly small business, DVBE, and DBE utilization reporting to account for the total value of all its contracts and to identify the reasons it has exempted specific contracts.
We conducted this audit under the authority vested in the California State Auditor by section 8543 et seq. of the California Government Code and according to generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives specified in the Scope and Methodology section of the report. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.
ELAINE M. HOWLE, CPA
California State Auditor
November 15, 2018
6 The Global Reporting Initiative is an international organization that develops sustainability standards that many of the world's largest corporations use. Go back to text