Report 2020-612 Recommendation 6 Responses

Report 2020-612: California Department of Public Health: It Could Do More to Ensure Federal Funds for Expanding the State's COVID‑19 Testing and Contact Tracing Programs Are Used Effectively (Release Date: April 2021)

Recommendation #6 To: Public Health, Department of

To ensure that the State has accurate COVID-19 data and to help mitigate the risks it caused by not having IV&V during the development phase of the CCRS project, Public Health should direct its IV&V consultant to monitor system performance and Public Health's data validation efforts and to provide regular reports on the system's reliability until the IV&V contract expires in December 2021.

This recommendation is fully implemented. Independent Verification and Validation (IV&V)consultants monitored system performance and Public Health's data validation efforts and provided monthly reports on the California COVID Reporting System's (CCRS) reliability. Monthly reporting continued through December 2021.

Monthly IV&V CCRS Performance Monitoring reports were submitted to CDPH for the August 2021 through December 2021 reporting period. The report covers the following areas:

1. CCRS Data Validation and Quality: Data validation and quality checks are performed when the Electronic Laboratory Record (ELR) is first received by CDPH. The process consists of three main activities: Confirm the file format is correct, perform further checks on individual messages, and confirm the message is not a duplicate.

2. CCRS System Performance: Monitoring focuses on the reporting databases and capabilities used by CDPH Data Scientists to create COVID-19 disease and vaccination status reporting. The performance of the CCRS system is enhanced by a robust Change Request (CR) management process that ensures change requests are reviewed and analyzed for impacts before approval, prioritization and placing in backlog of existing CRs.

3. CCRS System Reliability and Availability: Monitoring is focused on the CCRS system reliability and availability compliance to the contractual Service Level Agreements (SLAs) outlined in the Optum contract and amendments. System reliability and availability metrics are based on CCRS system availability and incident response.

To foster continuous improvement of CCRS, the IV&V Contract was extended for an additional year and will continue to require IV&V consultants provide monthly CCRS Performance Monitoring reports.

California State Auditor's Assessment of Status: Fully Implemented

This recommendation is fully implemented. Independent Verification and Validation (IV&V)consultants monitored system performance and Public Health's data validation efforts and provided monthly reports on the California COVID Reporting System's (CCRS) reliability. Monthly reporting continued through December 2021.

Monthly IV&V CCRS Performance Monitoring reports were submitted to CDPH for the August 2021 through December 2021 reporting period. The report covers the following areas:

1. CCRS Data Validation and Quality: Data validation and quality checks are performed when the Electronic Laboratory Record (ELR) is first received by CDPH. The process consists of three main activities: Confirm the file format is correct, perform further checks on individual messages, and confirm the message is not a duplicate.

2. CCRS System Performance: Monitoring focuses on the reporting databases and capabilities used by CDPH Data Scientists to create COVID-19 disease and vaccination status reporting. The performance of the CCRS system is enhanced by a robust Change Request (CR) management process that ensures change requests are reviewed and analyzed for impacts before approval, prioritization and placing in backlog of existing CRs.

3. CCRS System Reliability and Availability: Monitoring is focused on the CCRS system reliability and availability compliance to the contractual Service Level Agreements (SLAs) outlined in the Optum contract and amendments. System reliability and availability metrics are based on CCRS system availability and incident response.

To foster continuous improvement of CCRS, the IV&V Contract was extended for an additional year and will continue to require IV&V consultants provide monthly CCRS Performance Monitoring reports.

California State Auditor's Assessment of Status: Fully Implemented

Independent Verification and Validation (IV&V)consultants continue to monitor system performance and Public Health's data validation efforts and provide monthly reports on the California COVID Reporting System's (CCRS) reliability. Monthly reporting will continue until the IV&V contract expires in December 2021.

The monthly IV&V reports, addressing Recommendation #6 of the State Auditor's Report, were submitted to Public Health for the August 2021 through November 2021 reporting period. The IV&V CCRS Performance Monitoring report covers the following areas:

1.CCRS Data Validation and Quality: The data validation and quality check is performed when the Electronic Laboratory Record (ELR) is first received by Public Health. The process consists of three main activities: Confirm the file format is correct, perform further checks on individual messages, and confirm the message is not a duplicate.

2.CCRS System Performance: System performance monitoring is focused on the reporting databases and capabilities used by the Public Health Data Scientists to create COVID-19 disease and vaccination status reporting. The performance of the CCRS system is enhanced by a robust Change Request (CR) management process that ensures change requests are properly reviewed and analyzed for impacts before approval, prioritization and placing in backlog of existing CRs.

3.CCRS System Reliability and Availability: System reliability and availability is focused on monitoring the CCRS system reliability and availability compliance to the contractual Service Level Agreements (SLAs) outlined in the Optum contract and amendments. The system reliability and availability metrics are based on CCRS system availability and incident response.

California State Auditor's Assessment of Status: Pending

Independent Verification and Validation (IV&V)consultants continue to monitor system performance and Public Health's data validation efforts and provide monthly reports on the California COVID Reporting System's (CCRS) reliability. Monthly reporting will continue until the IV&V contract expires in December 2021.

The monthly IV&V reports, addressing Recommendation #6 of the State Auditor's Report, were submitted to Public Health for the April 2021 through July 2021reporting period. The IV&V CCRS Performance Monitoring report covers the following areas:

1. CCRS Data Validation and Quality: The data validation and quality check is performed when the Electronic Laboratory Record (ELR) is first received by Public Health. The process consists of three main activities: Confirm the file format is correct, perform further checks on individual messages, and confirm the message is not a duplicate.

2. CCRS System Performance: System performance monitoring is focused on the reporting databases and capabilities used by the Public Health Data Scientists to create COVID-19 disease and vaccination status reporting. The performance of the CCRS system is enhanced by a robust Change Request (CR) management process that ensures change requests are properly reviewed and analyzed for impacts before approval, prioritization and placing in backlog of existing CRs.

3. CCRS System Reliability and Availability: System reliability and availability is focused on monitoring the CCRS system reliability and availability compliance to the contractual Service Level Agreements (SLAs) outlined in the Optum contract and amendments. The system reliability and availability metrics are based on CCRS system availability and incident response.

California State Auditor's Assessment of Status: Pending

Public Health directed Independent Verification and Validation (IV&V) consultants to monitor system performance and Public Health's data validation efforts and provide monthly reports on the California COVID Reporting System (CCRS)s reliability until the IV&V contract expires in December 2021.

The first IV&V report, addressing Recommendation #6 of the State Auditor Report, was submitted to Public Health for the April 2021 reporting period. The IV&V CCRS Performance Monitoring report covers the following areas:

1. CCRS Data Validation and Quality: The data validation and quality check is performed when the Electronic Laboratory Record (ELR) is first received by Public Health. The process consists of four main activities: Confirm the file format is correct, perform further checks on individual messages and confirm the message is not a duplicate.

2. CCRS System Performance: System performance monitoring is focused on the reporting databases and capabilities used by the CDPH Data Scientists to create COVID-19 disease and vaccination status reporting. The performance of the CCRS system is enhanced by a robust Change Request (CR) management process that ensures change requests are properly reviewed and analyzed for impacts before approval, prioritization and placing in backlog of existing CRs.

3. CCRS System Reliability and Availability: System reliability and availability is focused on the monitoring the CCRS system reliability and availability compliance to the contractual Service Level Agreements (SLAs) outlined in the Optum contract and amendments. The system reliability and availability metrics are based on CCRS system availability and incident response.

California State Auditor's Assessment of Status: Pending

All Recommendations in 2020-612

Agency responses received are posted verbatim.