Responses to the Audit
Use the links below to skip to the specific response you wish to view:
- Regents of the University of California
- University of California Office of the President
August 7, 2017
Ms. Elaine Howle
California State Auditor
621 Capitol Mall, Suite 1200
Sacramento, CA 95814
State Auditor Howle:
Thank you for the opportunity to respond to the recommendation made to the Board of Regents in the draft audit report on UCPath and IT contracting. The Regents welcome this constructive input on oversight of IT projects at the university. Please find our response to the recommendation below.
To ensure that they can exercise necessary oversight for the university's significant IT projects, the regents should develop by December 2017 status reporting standards for the Office of the President and all university locations to follow. Such reporting standards should apply to all university IT projects more than a specified cost and, at a minimum, should establish the following:
- The frequency with which the Office of the President and all university locations must report to the regents. Such updates should occur at least three times per calendar year and coincide with regents meetings to allow for oral discussions.
- The types of disclosures the Office of the President and all university locations must present about each IT project including, but not limited to, changes in scope, projected costs, and schedule.
- The types of significant project risks the Office of the President and all university locations must disclose. The updates should also describe the actions the Office of the President and all university locations are taking to mitigate risks and the risks' potential effects on the project's cost, schedule, and scope.
By December 2017, the Regents will implement reporting standards for significant system wide IT projects based on a dollar threshold mutually agreed upon with UCOP. These standards will require oral or written updates at least three times per calendar year. The standards will also establish the types of required disclosures relating to changes in scope, projected costs and schedule, as well as significant project risks and related mitigation actions.
Chair, UC Board of Regents
Chair, Compliance and Audit Committee
August 7, 2017
Ms. Elaine M. Howle
California State Auditor
621 Capitol Mall, Suite 1200
Sacramento, California 95814
State Auditor Howle:
I write in response to your draft audit report University of California Office of the President — Contracted Employees and Contracting Practices, as it pertains to UCPath and information technology projects. The recommendations are helpful and constructive, and align with our proactive efforts to continually improve UC’s operations, policies and transparency to the Legislature and the public at large.
Our responses to specific recommendations are in the attachment to this letter, but I would like to take this opportunity to provide important information about UCPath, one of the main projects discussed in the audit. The University launched UCPath in 2011 to modernize and standardize its payroll system for more than 190,000 UC employees at 11 locations.
When I arrived at UC in September of 2013, UCPath was already well underway. At that time and as I do now, I view it as a necessary project to standardize UC’s business practices, improve accuracy and service quality, and create efficiencies across our system – and not necessarily as a tool for generating immediate savings.
I would like to emphasize some important context that speaks to the implementation and effectiveness of UC Path:
- UCPath is a large-scale and complex undertaking that improves how UC delivers key administrative services. Once implemented systemwide, the upgrade will standardize over 100 processes for payroll, human resources, and benefits through upgraded technology, a centralized database, and a shared services center to serve employees. The challenges of managing a project of this scale and complexity are reflected in the efforts of comparably-sized institutions and organizations that have attempted similar massive-data upgrades, which have resulted in major delays or failure.
- An upgrade is not only prudent, but unavoidable. UC’s 11 current systems are 35 years old and vary among locations. They’re disconnected, outdated, expensive to maintain, and require manual calculations. It has become increasingly challenging for UC to manage employee records and ensure payroll accuracy. An estimated $100 million is spent annually on manual rework and fixing errors. A systemwide upgrade will ensure that our resource-constrained campuses will be able to avoid building their own respective systems, at an estimated cost of more than double the current projected costs of UCPath.
- UC has become more effective at managing this project over time. Significant changes were made to the management of UCPath in 2013, after UC concluded that the IT consulting firm contracted to manage UCPath and oversee the transformation of business processes had failed to perform as expected and to achieve sufficient progress on implementing the project. UC then conducted an exhaustive assessment of the project, instituted better controls, and revamped the deployment approach. As a result, we revised the original business case, cost estimates, and projected timelines. And as we continue to move forward on our implementation, the accuracy of the projections delivered by the reconstituted project team has and will continue to improve. Many of the recommendations contained in the report will help us further that progress.
- UC has already made significant progress. Since November 2015, UCPath has been providing payroll, human resources, and benefits services for UCOP employees. Significantly, 70 percent of the functionality that will be needed at our campuses and medical centers was achieved with this rollout. This greatly reduces the implementation efforts that will be required on the campuses. Finally, we are well on track to deploy the system later this year at UCLA, UC Riverside, UC Merced, and the Associated Students of UCLA.
- UCPath benefits from governance that is diversified and rigorous. Various stakeholders oversee UCPath, including the UC Board of Regents, UCOP leadership and policy experts, and campus leaders – including vice chancellors, controllers, chief human resource officers, chief information officers, and academic personnel directors. The multiple governance bodies, along with the UC Board of Regents, oversee project progress, scope, schedule, and budget.
- UC continually improves oversight and communication of UCPath. Since the start of the UCPath project, the project leadership has made 79 presentations about and references to UCPath at public Regents’ meetings. This number does not reflect additional discussions, briefings, and consultations with individual board members and, in particular, committee and board leaders, in between the formal meetings. Nor does it include the public-facing website that we have created to provide important milestones and updates on the project. As a result of the recent restructuring of the committees of the UC Board of Regents, we have updated our practices for briefing The Regents on the project, which receives oversight from the Finance and Capital Strategies Committee.
I have complete confidence in UC’s ability to continue successful implementation of UCPath, a necessary project with significant, expansive, and long-term benefits to the University. As with any extremely complex undertaking, we are constantly finding ways to improve and be more efficient. That is why we welcome your recommendations and appreciate the time and diligence your staff spent in identifying ways in which we might further our substantial progress.
Yours very truly,
Response - CSA UCPath and IT Projects Recommendations
Responses to CSA UCPath and IT Projects Recommendations
- To ensure it fully reports the cost of IT projects, the Office of the President should develop cost reporting guidelines by December 2017 for UCPath and other significant IT projects across all university locations. These cost guidelines should identify cost categories at both the university and campus levels to ensure that estimates capture and communicate all development and implementation costs. In addition the Office of the President should produce cost reports to share with stakeholders on at least a quarterly basis.
UCOP will provide project sponsors and managers with guidelines for reporting standards and project risk management practices to formalize what is currently an informal practice. By December 2017, UC will develop cost reporting guidelines for UCPath and large systemwide projects/programs with budgets of $25M or greater. The guidelines will include cost reporting direction.
- To ensure it consistently follows best practices related to project management, the Office of the President should develop and implement guidelines for IT project development by June 2018. The guidelines should apply to all IT projects undertaken by any university location with a cost estimate of at least $5 million or more and should include the following elements:
- A means to assess schedules for reasonableness, and requirements for the creation and maintenance of an integrated schedule and resource plan for each project.
- Requirements for a rigorous change management process that establishes a means of assessing the implications of changes to a project’s scope, cost and schedule.
- Sound requirements for identifying, reviewing, and resolving risks to a project.
- IV&V to oversee the technical aspects of project development.
UCOP agrees that there are opportunities to strengthen and improve the project management approach for systemwide projects over $25M and will develop guidelines for IT project development by June 2018. All projects can benefit from the discipline and rigor contained in best practices; however, best practice in one context and at one moment in the process might not be applicable in another.
We disagree with statements made by CSA that IV&V practices should be run throughout any and every systemwide program. This would create a parallel management structure to the program, which would add cost and complexity while diluting the effectiveness of such a function. It is our current and proposed continued practice to use diligent governance as a way to identify and bring in IV&V third-parties for specific topics and situations as appropriate or necessary.
- The Office of the President should require that all university locations follow best practices by ensuring the university creates a deliverable expectations document for each IT contract similar to the documents the State’s management framework describes. The Office of the President should establish this requirement by December 2017. The deliverable expectations document should, at a minimum, identify the deliverables for each milestone and define the scope, content, entrance criteria, acceptance criteria, and development schedule for each deliverable.
UCOP agrees with CSA’s conclusion that our campuses follow established vendor payment practices. By December 2017, we will build upon the current practice by establishing a requirement for each large IT project of $25M or greater to produce a “deliverable expectations document.” We will review the State’s management framework to inform the development of this required document.
CALIFORNIA STATE AUDITOR’S COMMENTS ON THE RESPONSE FROM THE UNIVERSITY OF CALIFORNIA OFFICE OF THE PRESIDENT
To provide clarity and perspective, we are commenting on the response from the Office of the President. The numbers below correspond to the numbers we have placed in the margin of the Office of the President’s response.
Although the university president asserts in her response that the university spends $100 million annually for manual rework and error corrections for the legacy payroll system, we were unable to substantiate the Office of the President’s claim that UCPath will result in savings. In August 2017 the Office of the President sent us an email asserting that the elimination of manual rework would result in annual savings of $100 million. It based this assertion on a 2010 consultant study that formed the basis of its 2011 business case for the UCPath project and its original estimate that UCPath would result in savings of $753 million. In addition, the Office of the President claims that the retirement of the university’s payroll legacy system will result in savings of $13 million annually based on its 2011 business case. The Office of the President requested that we include these estimated savings in Figure 5. However, because the Office of the President stated that it no longer believed that UCPath would achieve the savings of $753 million and because it was unable to demonstrate that its new savings assertions would result in an actual reduction of university staff, we were unable to conclude that any savings would materialize.
Further, the Office of the President asserted that additional savings would result from its reduction of the projected staffing of the UCPath center from 568 positions to 440 positions, as well as its expectation of lower than projected salary levels for the staff filling the 440 positions. The Office of the President requested that we also add this claimed savings to Figure 5. However, we did not do so because the 2017 status cost of $130 million for “UCPath Center facility, equipment, and operations” in is already based on the projected staffing level of 440 positions at the lower staffing cost, during fiscal year 2018–19, which is when the Office of the President projects that the UCPath Center will be fully staffed. Therefore, we stand by Figure 5 as it is presented.
The Office of the President overstates the UCPath project’s level of communication with the regents and the project’s governance. As noted here, our review of the Office of the President’s 79 communications to the regents determined that only five status updates provided the regents with substantial information about UCPath’s status, cost, or schedule. We display those five status updates, along with an additional two updates and a financing request in Figure 6. The remaining 74 of the 79 communications either mentioned UCPath in relation to another issue or did not provide any substantive information on the project. Further, it was not until July 2017 that the Office of the President recognized the regents’ role in governance and expanded the project’s governance approach to include communicating critical decisions to the regents as we describe here. However, as we also note, the expanded governance approach does not go far enough in engaging the regents in decision making to truly allow them to fulfill their oversight role.
Although the Office of the President correctly acknowledges that our recommendations, including the use of IV&V, are best practices, it incorrectly concludes that IV&V would create a parallel management structure and add complexity. IV&V, when properly employed, helps to ensure that an IT system will perform as intended and meet its users’ needs, as well as provide an early warning of process and technical discrepancies, and problems that may go undetected or be detected too late. Further, our IT project management expert indicates the benefits that IV&V provides generally outweigh its cost. Given UCPath’s size, complexity, and risk, our IT project management expert believes that it would be prudent for the university to employ IV&V to mitigate weaknesses the Office of the President has already displayed in its project management of UCPath. Moreover, according to our expert, the Office of the President’s current and proposed continued practice to employ “IV&V third parties for specific topics and situations” does not represent the best practice definition of IV&V. As noted here, our IT project management expert concluded that the Office of the President’s approach results in oversight that is not equivalent to IV&V, which provides a more robust analysis of technical activities, and is embedded in the project as opposed to occurring periodically.