Report 2014-602 Summary - March 2015
High Risk Update—California Department of Technology:
Lack of Guidance, Potentially Conflicting Roles, and Staffing Issues Continue to Make Oversight of State Information Technology Projects High Risk
Our audit of the California Department of Technology's (CalTech) oversight of information technology (IT) projects, highlighted the following:
- CalTech faces challenges in pursuing effective project oversight.
- It lacks guidance for suspending or terminating an IT project and for recommending corrective actions to managers.
- It does not formally set expectations for its oversight authority with sponsoring agencies—the state agencies that are implementing IT projects.
- There is a potential conflict between the independent project oversight (IPO) analysts' role to oversee IT projects and their role of providing advice to sponsoring agencies.
- CalTech needs to document the oversight actions taken on projects and consistently retain its oversight documents.
- High turnover, an insufficient state job classification, potentially inadequate personnel resources, and inconsistent training impact CalTech's staffing practices and present risks to IT oversight.
- Although CalTech was aware of significant problems with two troubled IT projects, it did not intervene to require the sponsoring agencies to correct such problems. CalTech ultimately terminated one of these projects and suspended the other.
RESULTS IN BRIEF
In September 2013 the California State Auditor published its most recent assessment of the high-risk issues the State and certain agencies face. Our assessment identified oversight of the State's information technology (IT) projects as one area of ongoing concern. Our focus with this audit is the State's oversight of the execution stage of IT projects. In addition to having the authority to approve IT projects, the California Department of Technology (CalTech) has the responsibility to provide oversight during the execution stage of the projects it approves. IT project oversight continues to be a high-risk issue, in part, because of the needed improvements in CalTech's oversight discussed below and because of the negative impact to the State's fiscal health when these IT projects fail. For example, between 1994 and 2013, the State terminated or suspended seven IT projects after spending almost $1 billion. Furthermore, as of February 2015, the State had 45 IT projects under development that were under CalTech's oversight, with a reported cost of more than $4 billion.
Despite clear statutory authority to curtail troubled state IT projects, CalTech faces challenges in pursuing effective project oversight. One challenge is that CalTech lacks guidance in two critical situations: when CalTech management should suspend or terminate a project and when its independent project oversight (IPO) analysts should escalate concerns to CalTech management. In addition, CalTech does not formally set expectations for its oversight authority with sponsoring agencies—the state agencies that are implementing IT projects. This lack of communication may contribute to an environment wherein sponsoring agencies view CalTech as a service provider whose oversight they do not have to rigorously follow.
Moreover, there is a potential conflict between IPO analysts' role to oversee IT projects and their role to provide lessons learned and advice to sponsoring agencies, which heightens the risk that CalTech's oversight will not be sufficiently independent and objective. Finally, CalTech needs to document the oversight actions taken on projects and consistently retain its oversight documents, and it also needs to provide sponsoring agencies clearer guidance to ensure that the project status reports they submit contain accurate and appropriate information.
High turnover, an insufficient state job classification, potentially inadequate personnel resources, and inconsistent training impact CalTech's staffing practices and present risks to the oversight of state IT projects. Specifically, high turnover contributes to the loss of project knowledge and perspective, which disrupts the consistency and reliability of its oversight of state IT projects. Further, the current job classification used for IPO analysts may not ensure that they have adequate experience, as required by CalTech's own policy. However, CalTech is taking steps that may mitigate these risks. For example, to develop a stronger IT oversight workforce, CalTech is pursuing the modification and use of an existing job classification more relevant to IPO work, and it is developing a training plan for both new and current oversight staff. In addition, CalTech is developing a workload analysis tool to determine whether it needs additional resources for oversight or could use its existing resources more effectively. However, CalTech does not plan to contract with IPO consultants if the assessment indicates that it has insufficient staff available or lacks the necessary expertise.
Our IT expert assessed at least 12 monthly reports for each of the four state IT projects we reviewed—the California Department of Motor Vehicles' IT Modernization Project, the California State Controller's Office's MyCalPays Project, the Employment Development Department's Unemployment Insurance Modernization Project, and the Franchise Tax Board's Enterprise Data to Revenue Project—to determine the nature and significance of oversight findings and review any evidence of CalTech's response. He determined that although CalTech was aware of significant problems with the IT Modernization and MyCalPays projects, it did not intervene to require the sponsoring agencies to correct such problems. Our IT expert believes earlier intervention might have improved the outcomes of the projects. However, CalTech ultimately terminated the IT Modernization Project and suspended the MyCalPays Project. For the remaining two IT projects, our IT expert concluded that CalTech's actions were appropriate and that there were no significant issues the sponsoring agencies were not adequately addressing that would require CalTech's intervention. Until the conditions we discuss in this report are effectively addressed, the oversight of state IT projects will remain a high-risk area.
By December 2015 CalTech should develop and adopt criteria to guide the type and degree of intervention it will take to prevent IT projects with significant problems from continuing without correction, including the following:
- When and how IPO analysts should recommend corrective action and escalate issues to CalTech's management.
- What conditions could trigger CalTech to consider suspending or terminating an IT project.
To address the challenges it faces in providing effective oversight, CalTech should do the following:
- Develop, by December 2015, a method to formally document and communicate its expectations with sponsoring agencies.
- Develop a policy and training plan regarding expectations for the independence of its IPO analysts.
- Track oversight actions taken and consistently retain oversight documents.
- Provide sponsoring agencies clear guidance for accurately reporting IT project status.
To address its staffing issues, CalTech should continue its efforts to modify and use an existing, more relevant job classification for the IPO analyst role; conduct a workload assessment, by December 2015, to determine the resources needed for oversight activities; and implement, by June 2015, a consistent and repeatable training program for IPO analysts.
CalTech agrees with our recommendations and indicates it will continue its efforts to improve the oversight of state IT projects.