Report 2010-122 Recommendations and Responses in 2013-041

Report 2010-122: California Department of Transportation: Its Capital Outlay Support Program Should Strengthen Budgeting Practices, Refine Its Performance Measures, and Improve Internal Controls

Department Number of Years Reported As Not Fully Implemented Total Recommendations to Department Not Implemented After One Year Not Implemented as of 2012-041 Response Not Implemented as of Most Recent Response
Department of Transportation 2 11 7 5 0

Recommendation To: Transportation, Department of

To improve accountability internally and with the public, Caltrans should create and incorporate an analysis of support cost budget variances in its quarterly report to the agency and in its annual report to the Legislature and the governor. The analysis should report on the number of completed projects with budget variances and on the number of open projects for which the estimates at completion predict budget variances. Further, the analysis should report on the overrun and underrun ratios for those projects, and the portions of the variances due to rates and hours. Also, Caltrans should include in its strategic plan a measurable goal for reducing variances.

Response

The annual report was transmitted to the Legislature on November 15, 2012, and included the performance measure that targets support expenditures that are within a specified range of the support budget. The annual report can be found at http://www.dot.ca.gov/reports-legislature.htm.

As previously reported, Caltrans did not included in its report an analysis of variances due to rates or hours because the largest impact to the project costs are generally due to changes in scope of work such as additional permit requirements and mitigation requirements. As the audit report noted, the primary cause for cost overruns in the projects reviewed was the increase in hourly rates. Caltrans does not anticipate this being the case any time in the near future. However, Caltrans does perform a separate analysis of rates each year for planning purposes.

Recommendation To: Transportation, Department of

To improve performance metrics related to the support program, Caltrans should continue to explore the use of additional metrics, such as a measure based on a productivity index as described in a March 2011 draft study by the University of California, Davis.

Response

Efforts to date on this recommendation included the results of the UC Davis Study and the Production Study modeled after the UC Davis Study. There were no useful applications for measurable metrics from either study. A copy of the Production Study was transmitted with the 2012 annual status.

After an additional year of evaluation, we concluded that these efforts do not have sufficient information to evaluate productivity due to the wide diversity of project factors (Locations, Environment, Type of Project, Design Strategy, Right of Way, etc.) that contribute to production hours needed to complete projects. At this time, Caltrans plans to continue its use of Support-to-Capital as a program-level performance metric based on project size.

Recommendation To: Transportation, Department of

To ensure that it monitors the status of projects, Caltrans should implement earned value management throughout its districts in a manner similar to the implementation in the Los Angeles district. To allow for performance evaluation of project work, Caltrans should ensure that these performance metrics are available at the task level for both active and completed projects. Caltrans should instruct districts to aggregate this information for all projects by task level, to better assess the effectiveness and efficiency of support costs by task level. Caltrans should also make available to project managers graphical displays of project cost and schedule performance.

Response

Caltrans determined that full implementation of earned value management was contingent on the roll-out of the PRSM application. PRSM will provide earned value metrics and report at the task level for all projects in the system. The date for a complete roll-out of PRSM moved from June 2013 to September 2013 due to technical and training issues during the early district roll-out. Roll-out of PRSM was completed on August 29, 2013.

Recommendation To: Transportation, Department of

To better address costs associated with the support program, Caltrans should ensure that the PRSM system contains strong controls that ensure employees only charge time to projects and phases for which they are assigned.

Response

Full implementation of PRSM was moved from June 2013 to September 2013 due to technical and training issues during the early district roll-out. Roll-out of PRSM was completed on August 29, 2013, and put in strong controls to ensure employees only charge time to projects their units are assigned.

Recommendation To: Transportation, Department of

To better address costs associated with the support program, Caltrans should commission an independent study of the costs and benefits of using consultants to address temporary increases in workload and, if the study reveals cost savings, use consultants. To the extent possible, Caltrans should also use temporary staff appointments for temporary increases in workload when consultants are unavailable.

Response

As mentioned previously, two reports were prepared by CTC and Associates LLC (dated July 15, 2011, and October 28, 2011, respectively), which compared the use of in-house staff and consultants. In general, these reports concluded that cost should not be the overriding factor in deciding whether to outsource. Other factors such as expediting project delivery and managing workload should be taken into consideration when determining when and what work to outsource. Caltrans subsequently contracted an additional study to identify options or tools to improve decision-making processes regarding resource mix during workload peaks and valleys. Available resources (State staff, consultants, temporary help, overtime, students, retired annuitants, etc.) were identified along with recommendations on the best utilization of each of these resources. This final report was completed in May 2013, and was emailed to John Lewis.

Current Status of Recommendations

All Recommendations in 2013-041