Report 2017-107 All Recommendation Responses

Report 2017-107: Santa Clara County Registrar of Voters: Insufficient Policies and Procedures Have Led to Errors That May Have Reduced Voters' Confidence in the Registrar's Office (Release Date: October 2017)

Recommendation #1 To: Santa Clara County Registrar of Voters

To ensure the accuracy of voting district boundaries and to allow Santa Clara to make changes to existing boundaries as necessary, Santa Clara should establish a procedure requiring Mapping staff to ask each voting district either to certify that its boundaries are accurate and unchanged or to provide an updated map of its boundaries at least 125 days before each general district election.

60-Day Agency Response

Originally responded 12/22/17. We have developed and implemented a comprehensive procedure, and accompanying policy, whereby Mapping Division staff will, at least 125 days before each general district election (i.e., Consolidated Election held in odd-numbered years, and Primary Election held in even-numbered years), mail notification letters to all voting districts requesting confirmation of district boundaries. A paper map representing the most current map on file with the Registrar of Voters (ROV) will accompany the notification letter. The paper map includes a section for the district official to either sign-off on the accuracy of the map on file, or else provide additional information so that the Mapping Division can make changes to the district boundary. If the Mapping Division makes changes, an updated map would be sent to the district for a fresh review and sign-off. Included in this procedure is the maintenance of a detailed Tracking Log. This procedure has already been applied beginning in November 2017 and has resulted in better response and information than received in prior years, with the map package receiving positive feedback from several districts. The procedure and policy have been attached separately as Supporting Documentation.

California State Auditor's Assessment of 60-Day Status: Fully Implemented

Recommendation #2 To: Santa Clara County Registrar of Voters

Santa Clara should immediately coordinate with Information Services to access the most current maps from other county departments, such as the Santa Clara Assessor, to verify the accuracy of the district maps.

6-Month Agency Response

Progress from the last report includes more frequent access to the Assessor's data for use of verification of data in GIS and EIMS, evidence is attached separately as Supporting Documentation. The Enterprise GIS Committee regular monthly meeting schedule has begun; Agendas and Minutes have been uploaded separately as Supporting Documentation. Various subcommittees have branched out of the Enterprise GIS Committee to which several ROV personnel have been assigned, specifically relating to Countywide GIS Licenses and development of a Countywide Contract for GIS Services, evidence is attached separately as Supporting Documentation.

California State Auditor's Assessment of 6-Month Status: Fully Implemented

60-Day Agency Response

Originally responded 12/22/17. Since 2013, the Mapping Division has received a report annually from the Santa Clara County Assessor's Office that has been used for proofing district boundaries by verifying which districts belong to which addresses based on the Assessor's active roll. As stated in the State Audit Report (2017-107), a meeting was held in May 2017 with Information Services Department (ISD) representatives, Assistant Registrar of Voters Virginia Bloom, Mapping Division Manager Bob Nguyen, and the State Audit Team, whereat it was determined that more frequent access to the Assessor's rolls would be granted on a quarterly basis. On August 10, 2017, ISD provided the Santa Clara County Statement of Understanding Pertaining to GIS Data to the ROV Mapping Division Manager for execution, allowing for open access (i.e., at any time). Note that the Assessor's open access data sets are designated as "interim reference" data, except when released as part of the published annual roll. On December 14, 2017, Virginia Bloom and Bob Nguyen attended the kickoff County-wide Enterprise GIS committee meeting, and as committee members plan to attend and request relevant GIS data-sharing agenda items be added to future meetings. The signed Understanding Pertaining to GIS Data and materials related to the Enterprise GIS committee meeting have been attached separately as Supporting Documentation. An update to the status of this recommendation will be made at the next report back to the State Auditors.

California State Auditor's Assessment of 60-Day Status: Pending

Recommendation #3 To: Santa Clara County Registrar of Voters

To reduce errors and potentially its workload, Santa Clara should research by January 2018 its opportunities to integrate mapping software with its election management software, and Santa Clara should implement this integration of mapping software technology by June 2018.

Annual Follow-Up Agency Response From October 2019

Progress from the last report includes the project completion of the data conversion from street address range to single point addresses in the Election Information Management System (EIMS). The project was conducted by our Geographic Information System (GIS) Mapping Division and our vendor, DFM Associates. The single point address conversion was a necessary step to allow integration with mapping software. Now that it is complete the GIS Mapping division may utilize the importing functions of the EIMS program. The project consisted of several phases and took several months to complete. The success of this data conversion will now allow for GIS importing into EIMS. See extract reports attached as samples, both before and after the data conversion. GIS Mapping staff have worked very diligently to test the data for quality assurance purposes. This project also included several other operations of the Registrar of Voters Office, such as the County TSS (Technology Services & Solutions) Department, which oversees the County's websites in order to allow for a much larger data file for address lookup, and the ROV Administrative Services Division, for implementation of web hosting of the data, working with TSS for the scripting of the program, and building of a new ROV Website, as a whole. The GIS Mapping Division staff have continued to access the Assessor's data for verification of data in GIS and EIMS, as well as participate in the Enterprise GIS Committee and subcommittees monthly meetings for county department collaboration purposes. Supporting documentation has been uploaded separately. Status: Completed October 2019

California State Auditor's Assessment of Annual Follow-Up Status: Fully Implemented

1-Year Agency Response

Progress from the last report includes continued access to the Assessor's data for verification of data in GIS and EIMS; Enterprise GIS Committee and subcommittees continue to meet monthly. The scheduling of a WebEx with the EIMS vendor, DFM, and the Mapping Division has been completed. The WebEx is to facilitate to collection of additional address data sources from other county agencies. Due to the proximity of the election and the required blackout of the EIMS during the election process, the WebEx has been scheduled for January 3, 2019. Vendor has informed us that the process is estimated to take approximately 30-60 days for the data conversion into a single point address and for verification of the accuracy of the data. Once this process is complete, then the import functionality into the EIMS system will work. Evidence as supporting documentation has been uploaded separately.

California State Auditor's Assessment of 1-Year Status: Pending

6-Month Agency Response

The Santa Clara County ROV had conducted the recommended research within the timeframe, however discovered that we could not implement it due to the configuration of the data. To meet the demands of the Mapping Division requirements, we have hired additional staffing and have used all of 2018 as training to get its new staff familiar with the EIMS data system. Our vendor, DFM has agreed to assist our Mapping Division with the conversion of its data system so that the import options may be implemented. It was also determined that due to the time that the conversation of the data will take, the work must be done outside of an election cycle therefore is planned for 2019.

California State Auditor's Assessment of 6-Month Status: Pending

60-Day Agency Response

Originally responded 12.22.17. On several occasions in recent months we have reached out to our election management system vendor, DFM Associates, requesting a meeting to discuss options for enhancement of the Precinct/District Modules of the Election Information Management System (EIMS). A request was also sent out to all counties via the California Association of Clerks and Election Officials' (CACEO) ElecNet listserv requesting best practices and procedures, including those relating to import/export options for district boundary verification. The Mapping Division Manager has prepared a summary of the best practices received by other counties. He called Alameda County to request information relating to their import/export program, however was told that the import/export program is proprietary to the vendor (DIMS) of that county's system, therefore Alameda was not at liberty to share it with us. A follow-up to our inquiry with our vendor DFM was made on December 14, 2017, at which time DFM's representatives acknowledged receipt of our request and advised that they have not conducted any research into further options, offering that our staff work with theirs for use of existing reports for proofing purposes. The Summary of Best Practices in other counties, correspondence with other counties and DFM, and a memo regarding recent communications have been attached separately as Supporting Documentation. Further research will be conducted and an update to the status of this recommendation will be made at the next report back to the State Auditors.

California State Auditor's Assessment of 60-Day Status: Pending

Recommendation #4 To: Santa Clara County Registrar of Voters

Santa Clara should promptly seek compensation from its vendors for all costs associated with rectifying vendor errors that occur in the future.

60-Day Agency Response

Originally responded 12.22.17. On December 15, 2017, we implemented "ROV Policy - Decision Matrix_Implemented 121517," which incorporates the action item to track time spent by staff on rectifying vendor errors, and the requirement to seek reimbursement for error costs where vendor agreements allow. This two-part action item in the ROV Decision Matrix (see 12.22.17 Response #9) will be carried out when vendor product(s) and/or service(s) are determined to be root cause of an error in voting materials or information. The action items that will be carried out after each such error include: a) Identify Root Cause, a.i) If cause of error was due to vendor or district error, then add follow-up with vendor/district, conduct timesheet time study for hours spent on error, and seek reimbursement for error costs (where vendor agreements allow), b) Determine Affected Voters or Potential Affected Voters, c) Identify Prevention Steps, d) Update Error Tracking Log, d.i) Document any reasons from deviating from this Decision Matrix in the ROV Error Tracking Log, and e) Include Information in Post-Election Report. While this policy was implemented on December 15, 2017, the ROV has yet to have the opportunity to put the policy into practice. ROV Policy - Decision Matrix_Implemented 121517 has been attached separately as Supporting Documentation.

California State Auditor's Assessment of 60-Day Status: Fully Implemented

To the extent that Santa Clara follows its established policy by seeking compensation from its vendors for all costs associated with rectifying vendor errors that occur in the future, this recommendation is fully implemented.

Recommendation #5 To: Santa Clara County Registrar of Voters

To make certain that its staff learn of election-related errors and identify trends in error types, and to allow Santa Clara to identify necessary modifications to processes that will reduce or eliminate such errors, Santa Clara should immediately formalize a policy requiring the continued use of a spreadsheet similar to the one it created to track election-related errors.

60-Day Agency Response

Originally responded 12.22.17. On December 15, 2017, we implemented "ROV Policy - Tracking Errors_Implemented 121517," which requires the Registrar of Voters administration to maintain an error log in spreadsheet format. The information the spreadsheet "ROV Error Tracking Log" will track include : a) Election Date, b) Election Type, c) Error Description, d) Approximate # of Voters Affected, e) # Ballot Types or CVIGs Affected, f) Notification Methods, g) ROV Error?, h) Other Error Source(s), i) Believed Root Cause, j) Steps Taken/Procedure Changes/Notes, and k) Deviations from ROV Decision Matrix. The ROV Error Tracking Log will be shared with managers and staff annually at a Leadership Team Meeting in January, and again after any election-related error necessitates an update to the log. Any deviations to the post-error actions spelled out in the ROV Decision Matrix (see 12.22.17 Response #9) will be documented in the ROV Error Tracking Log. While this policy was implemented on December 15, 2017, the ROV has yet to have the opportunity to put the policy into practice. The policy has been attached separately as Supporting Documentation.

California State Auditor's Assessment of 60-Day Status: Fully Implemented

Recommendation #6 To: Santa Clara County Registrar of Voters

To ensure accuracy and consistency in the creation, review, and distribution of election-related materials, Santa Clara should review and document in detail all policies and procedures by October 2018, prioritizing its documentation for the divisions that are responsible for the most frequent and egregious election-related errors. Specifically, Santa Clara should review and formalize Mapping policies and procedures by January 2018, to allow time for implementation before the June primary election process. By October 2018, Santa Clara should review and formalize policies and procedures for the remaining divisions--including Ballot Layout, Candidate Services, and Vote by Mail--to provide adequate time for implementation before the November general election process.

Annual Follow-Up Agency Response From October 2020

Progress from the last report includes the expansion of the Ballot Layout Division's (BLD) operations with the creation of official ballots in nine languages for the first Voters' Choice Act election, the March Presidential Primary Election, along with four new language-specific Election Specialists; as well as five new languages requiring the creation of facsimile ballots for the November Presidential General Election. BLD has been able to fine tune several of their procedures, such as the Candidate Statement Proofreading Checklist; Election Preparation and Setup Overview; Measure Document Proofreading Checklist; Proofreading CVIG Instructional Filler Pages-Updated; Proofreading Division Request-Updated; Proofreading English Candidate Statements; Proofreading Official Optical Scan Ballot; Standards and Style Guide-Updated; and How to Proof the Office Data Detail Proof List Report OIMR005.

We prioritized the four divisions of Mapping, Candidate Services, Ballot Layout, and Vote-by-Mail to meet this reporting deadline. Cumulative submissions and several procedures currently in progress represent a significant portion of the review and documentation of procedures required to fulfill the audit recommendation. Due to new legislation, as well as the new voting model and new voting equipment, it is essential that existing procedures be updated on a continuous basis. We believe our progress to date satisfies this recommendation.

California State Auditor's Assessment of Annual Follow-Up Status: Fully Implemented

Annual Follow-Up Agency Response From October 2019

GIS Mapping: Access the Planning and Development Department Server, Creating GIS District Boundary Data, Generating Drayage Route Report Assignments, Generating E-Night Courier Assignments, Introduction to the ROV Online Precinct Map Viewer, Uploading a GIS Precinct Map into the Online Election Night Reporting Module, Uploading a Precinct PDF File Through the County's Content Management Application Website, and Verifying a City Annexation procedures.

For Candidate Services: City Clerk and District Administrator Meetings, Official Candidate List, Random Alpha Drawing, Receiving Measure Resolutions, and Withdrawal of Candidacy procedures.

The Ballot Layout procedures consist of Proofreading Candidate Statements, Proofreading Covers, Proofreading Labels, Proofreading Official Optical Scan Ballots, and JAWS screen reader Program materials of which training PowerPoints have also been created for each of these processes.

The Vote-by-Mail procedures include E+3 Postmark, Write-In Tally, procedures with training materials and PowerPoints. Supporting documentation has been uploaded.

A centralized location for all procedures for each division has been created. We prioritized the four divisions (Mapping, Candidate Services, Ballot Layout, and Vote-by-Mail)in an effort to meet this reporting deadline. Cumulative submissions submitted, as well as a number of procedures currently in progress represent a significant portion of the procedures for recommendation. Due to new legislation, as well as new voting model and new voting equipment, it is understood that many of our existing procedures will need to be updated, therefore preparation of new policies and procedures will be constant; we believe that our progress to date represents that we have met this recommendation.

California State Auditor's Assessment of Annual Follow-Up Status: Partially Implemented

We commend Santa Clara in its efforts to implement this recommendation, and we recognize that it has established detailed policies and procedures for most of its divisions. We will continue to report this recommendation as partially implemented until Santa Clara finalizes its policies and procedures for its Ballot Layout Division.

1-Year Agency Response

Progress from the last report includes the formalized process of creating policies and procedures for all divisions prioritizing the Ballot Layout, Candidate Services, Mapping, and Vote-by-Mail divisions. Numerous draft documents have been created for each of the named divisions and tested during both the June 2018 and set up of the November 2018 elections. As multiple offices are involved with the review and approval of each of these documents, it is anticipated that this process will continue to take further time for full development and implementation; therefore, this recommendation is not yet completed. A few of the completed procedures are attached as evidence of supporting documentation and have been uploaded separately.

California State Auditor's Assessment of 1-Year Status: Pending

6-Month Agency Response

Progress from the last report includes continued efforts to document all policies and procedures in time for implementation before the June 2018 primary election process. For the record, several key positions within the Registrar's Office have changed since the December 2017 report. The vacant Management Analyst position of the Policies & Procedures Division has been filled as of late March. And, also for that division, the recruitment for the vacant Associate Management Analyst is on-going. This division is responsible for the creation and implementation of the Department's Policies and Procedures. The Election Division Coordinators for both the Ballot Layout Division (BLD) and the Candidate Services Division (CSD) have both retired effective end of 2017. The Election Process Supervisor was promoted to Coordinator in BLD and now have several new personnel within BLD. In CSD, the recruitment to fill this vacancy is on-going at this time. The office has also hired three Management Aide positions to help write up the documentation of training, policies, and procedures while working with each of the specific division managers. We have begun to formalize policies and procedures for all of our divisions prioritizing the Ballot Layout, Candidate Services, Mapping and Vote-by-Mail divisions. Numerous draft documents have been created for each of the named divisions and tested during the nomination filing and ballot layout periods of the June 2018 Election. Those documents are currently under review at this time. It is anticipated that approved final policies and procedures will be produced at the next report back in October 2018.

California State Auditor's Assessment of 6-Month Status: Pending

60-Day Agency Response

Originally responded 12.22.17. In accordance with the recommended implementation time frame of January 2018, we have reviewed and formalized a policy and numerous procedures for the Mapping Division. We have begun reviewing and formalizing policies and procedures for the Ballot Layout, Candidate Services, and Vote-by-Mail divisions targeting completion by October 2018, in order to meet the recommended implementation timeframe and provide adequate time for implementation before the November 2018 General Election. To the extent possible, the intention is to centralize all procedures for each division into a unified procedure manual for each division. We have prioritized the Mapping Division Procedure Manual first in order to meet the January 2018 recommended implementation time frame, and implement Recommendation #1's procedure to request each voting district certify that its boundaries are accurate through receiving and verifying the accuracy of the current map on file with the Registrar of Voters. In addition, on December 15, 2017, we implemented "ROV Policy - Requesting Proof from Printing Services or Vendor_Implemented 121517" in order to address a concern in the State Audit report #2017-107 that we had not documented our practice of verifying the accuracy of voter data in print or mailing jobs. This policy, a new policy for the Mapping Division to implement the procedure to request each voting district certify its boundaries, and the Mapping Division Procedure Manual as of December 22, 2017, have been attached separately as Supporting Documentation.

California State Auditor's Assessment of 60-Day Status: Pending

Recommendation #7 To: Santa Clara County Registrar of Voters

To reduce the risk of staff errors, inconsistencies in procedures, and the loss of institutional knowledge in the creation, review, and distribution of election-related materials, Santa Clara should develop and implement training for its staff that includes instructions on its comprehensive policies and procedures. The development of this training should take place concurrently with Santa Clara's detailed documentation of its policies and procedures, and Santa Clara should require relevant staff to attend this training before each major election.

Annual Follow-Up Agency Response From October 2020

Progress from the last report includes the expansion of the Ballot Layout Division's (BLD) operations with the addition of four new language-specific Election Specialists to the team and the creation of official ballots in nine languages for the March Presidential Primary Election. Training was conducted with all staff on all areas of operation, and the sign-in sheets for the training sessions are attached as evidence, including training with the vendor when the training included the ballot creation system. Separate training was conducted for the General Election, as a software enhancement of the new voting system was implemented for use. BLD has been able to fine tune several of their procedures and trainings due to new staff joining the team.

Also, the Candidate Services Division has a new Division Coordinator who, due to COVID, has undertaken the conversion of the paper/in-person process of the nomination filing period into an on-line processing and created new training materials for both customers and staff. The trainings conducted were attended by all customer jurisdictions, as well as all divisional staff. All trainings were also recorded and posted for easy access by both customers and staff. The division personnel worked closely with County Counsel throughout the filing periods.

We prioritized the four divisions of Mapping, Candidate Services, Ballot Layout, and Vote-by-Mail to meet this reporting deadline. Cumulative submissions represent a significant portion of the trainings for this recommendation. Due to new legislation, as well as the new voting model and new voting equipment, it is essential that existing procedures and training materials be updated on a continuous basis. We believe our progress to date satisfies this recommendation.

California State Auditor's Assessment of Annual Follow-Up Status: Fully Implemented

Annual Follow-Up Agency Response From October 2019

Training & Staff Development Div. created a new Learning on Demand Portal. All managers have been presented with the materials. The intention is to have each division upload their training materials into this program so that an employee can have access to training materials and resources. The use of program has begun in October 2019 and expected to accelerate staff trainings.

Mapping attended trainings at CACEO Cal-Peak Course on Redistricting and Drawing new maps, and ESRI-GIS program training. Staff developed trainings on Adding/Removing a District in EIMS, Precinct Consolidation, and Splitting a Precinct.

CSD attended trainings with State Fair Political Practices Commission, and Secretary of State's Office. CSD attended a FPPC Campaigns and Committees filing Officer Duties training on April 8; April 29, CSD & BLD attended WebEx by DFM on EIMS 4.3 Office/Incumbent Module; May 2, CSD attended WebEx by DFM on 5.1 Election Set Up Part 1, Part 2 was on May 24, by WebEx and also attended by BLD. May 9, CSD attended WebEx by DFM on 5.2 Candidate Contests and Processing. On September 11, CSD, along with Voter Registration and Election Logistics and Voting Systems attended a nomination filing training by Secretary of State's office. CSD, along with Policies and Procedures, developed trainings on procedures relating to Random Alpha Drawing, Receiving Measure Resolution, and Withdrawal of Candidacy.

BLD's trainings of Proofreading Candidate Statements, Proofreading Covers, Proofreading Labels, Proofreading Official Optical Scan Ballots, and JAWS Program training materials.

Vote-by-Mail's materials are E+3 Postmark and Write-In Tally trainings.

Process for implementing training is complete, although training materials will be continually updated as procedures are finalized and updated to reflect changes in law and developing best practices. Therefore, cumulative submissions of evidence previously submitted represent the completion for this recommendation.

California State Auditor's Assessment of Annual Follow-Up Status: Partially Implemented

We commend Santa Clara in its efforts to implement this recommendation, and we recognize that it has established training that includes instructions on its policies and procedures for most of its divisions. We will continue to report this recommendation as partially implemented until Santa Clara finalizes its policies and procedures for its Ballot Layout Division.

1-Year Agency Response

Progress from the last report includes evidence as to the four divisions and the trainings that each has conducted relating to both the June and November elections. The Executives remind the divisions to use the policies and procedures for each of their trainings and to provide updates as feedback to the Policies & Procedures Team for further development of training materials, as well training templates. Evidence as supporting documentation has been uploaded separately.

California State Auditor's Assessment of 1-Year Status: Pending

Santa Clara has demonstrated progress in developing training for its staff. However, until Santa Clara completes its comprehensive policies and procedures, and aligns its training with those policies and procedures, we will continue to report this recommendation as pending.

6-Month Agency Response

Progress from the last report includes continued efforts to formalize the training by division to align with the policies and procedures created. The EIMS vendor, DFM, was contacted and is currently conducting trainings of its specific modules for each of the specific divisions; i.e., Candidate Services, Precinct Operations, and VoteCal for our Voter Registration Division. Evidence as supporting documentation has been uploaded separately. Additionally, CACEO is offering Signature Verification Training in different locations throughout the state and the office has made arrangements to send key staff to attend those trainings. Also attached separately as Supporting Documentation are the staff emails identifying those to attend, as well as the CACEO flyers announcing the trainings. Individual Training by Position procedures, as well as sign-in sheet templates, have been created for the Candidate Services Division, Ballot Layout Division, Vote by Mail Division, and the Mapping Division. Each of those documents have been uploaded separately as Supporting Documentation, as well as evidence of those trainings by division. The Mapping Division also collected the training materials from the County ISD GIS division which is also included. Specific training was also created for the Candidate Services and Ballot Layout divisions relating to the Changes and Issues Log as outlined in the Handoff procedures for each of those divisions.

California State Auditor's Assessment of 6-Month Status: Pending

60-Day Agency Response

Originally responded 12.22.17. As the development of more detailed and centralized policies and procedures progresses, our division managers will introduce more formalized training for staff that includes instruction on the comprehensive policies and procedures. Staff will be required to attend this formalized training prior to each major election. The recently updated Mapping Division Procedure Manual (see 12.22.17 Response #6) will facilitate formalized training of the Mapping Division staff prior to the major elections in 2018. The Candidate Services Division has begun the process of formalizing its nomination training by requiring, before each training session, staff to sign an attendance sheet that includes the materials to be covered for that session. In addition, procedures have been created for developing and conducting nomination training of both staff and candidates for elected office. A procedure for preparing and conducting nomination training for staff, a procedure for preparing for and conducting a judicial seminar, and a procedure for preparing for and conducting a seminar for city office and district candidates, have been attached separately as Supporting Documentation. Also attached separately as Supporting Documentation are the staff nomination training attendance sheets and the PowerPoint presentation used for the December 7, 2017, Judicial Seminar.

California State Auditor's Assessment of 60-Day Status: Pending

Recommendation #8 To: Santa Clara County Registrar of Voters

To ensure the accuracy of election-related materials, Santa Clara should immediately implement a procedure for candidates, voting districts, or others who submit documents to have them verify the accuracy of the electronic versions of those documents once Santa Clara has formatted them.

6-Month Agency Response

Progress from the last report includes the implementation and use of the policy, procedure and Municipality and Authors Review Tracking Log. The tracking log includes all the authors' contact information and their specific responses back to our office, as well as the final approval of each of those documents. A specific email address was created for the Municipality and Author Review so that the appropriate personnel within the division are notified when a response from an author is received in response to the office's request for review and approval. Evidence to support these practices, as well as the Tracking Log, have been submitted separately as Supporting Documentation.

California State Auditor's Assessment of 6-Month Status: Fully Implemented

60-Day Agency Response

Originally responded 12.22.17. We have developed a procedure for candidates, voting districts, and measure argument submitters to verify the accuracy of electronic versions of voting material documents once the Ballot Layout Division has formatted them, against the original submitted documents. This procedure, called the "Municipality and Author Review," requires the Candidate Services Division to ensure a legible email address is provided, and after materials have been transferred to Ballot Layout, the Ballot Layout Division to create a special log for tracking the review process. After a document has been formatted in English, it is a) proofread multiple times, b) placed in a "Done" tray, c) uploaded to the translation vendor's portal, d) emailed to the municipal clerk or author, e) logged as "emailed" in the special log, and f) filed in batch order awaiting response. Any responses are reviewed to ensure if they are valid (i.e., an error was introduced by the ROV) or invalid (i.e., the error was in the original). These actions are recorded in the special log, until all documents have been checked off as completed. While this procedure was approved on December 21, 2017, the ROV has yet to have the opportunity to put the procedure into practice. The procedure and sample tracking log have been attached separately as Supporting Documentation.

California State Auditor's Assessment of 60-Day Status: Partially Implemented

Although Santa Clara has developed a procedure for candidates, voting districts, and others who submit documents to have them verify the accuracy of the electronic versions of those documents once it has formatted them, it has not yet implemented this procedure. Until Santa Clara can demonstrate that it has implemented this procedure, we will continue to report this recommendation as partially implemented.

Recommendation #9 To: Santa Clara County Registrar of Voters

To ensure consistency in responding to election-related errors, Santa Clara should immediately implement a contingency plan or decision matrix that includes specific guidelines for the actions it will take based on the number of voters affected and the significance of the error. In instances in which it chooses to deviate from this plan, Santa Clara should document its reasons for deciding to do so.

60-Day Agency Response

Originally responded 12.22.17. On December 15, 2017, we implemented "ROV Policy - Decision Matrix_Implemented 121517," which requires ROV administration, after an error in voting materials has been discovered, to follow the guidelines listed in a written decision tool. This written decision tool we created, called the "ROV Decision Matrix," includes several action items that must be taken after any error that affects voting ability is discovered (see 12.22.17 Response #5), while specifying certain actions be taken depending on the severity or circumstances of the error. Any deviations from the guidelines in the ROV Decision Matrix will be documented in the ROV Error Tracking Log (see 12.22.17 Response #5). While this policy was implemented on December 15, 2017, the ROV has yet to have the opportunity to put the policy into practice. The policy has been attached separately as Supporting Documentation.

California State Auditor's Assessment of 60-Day Status: Fully Implemented

To the extent that Santa Clara follows its established decision matrix when responding to election-related errors, this recommendation is fully implemented.

Recommendation #10 To: Santa Clara County Registrar of Voters

To maintain the public's confidence in it and its functions, Santa Clara should immediately include in its postelection reports descriptions of any election-related errors, accounts of why the errors occurred, and explanations of how it plans to prevent similar errors from occurring in the future.

1-Year Agency Response

Progress from the last report reflects that our office has prepared the Post-Election Report for the June 5, 2018 Primary Election and incorporated an Errata Chapter. As standard practice, this report is emailed out to all members of the Board of Supervisors and members of the Citizens Advisory Commission on Elections (CACE), as well as posted on our website. Evidence as supporting documentation has been uploaded separately. This recommendation is now deemed completed as of October 30, 2018.

California State Auditor's Assessment of 1-Year Status: Fully Implemented

6-Month Agency Response

Since the last report of December 2017, our office has not conducted any elections nor prepared any Post Election Reports; therefore, we have not yet had an opportunity to include any descriptions of any election-related errors, accounts of why the errors occurred, and explanations of how we plan to prevent the errors from occurring in the future.

California State Auditor's Assessment of 6-Month Status: Partially Implemented

60-Day Agency Response

Originally responded 12.22.17. On December 15, 2017, we implemented "ROV Policy - Decision Matrix_Implemented 121517," which incorporates the requirement to include information concerning an error in voting materials in the Post-Election Report published after each election. Information concerning the error will be based on specific details documented in the ROV Error Tracking Log. This information will include a description of the error, an account of the reason(s) for the error, and an explanation of how the department plans to prevent future errors (see 12.22.17 Responses #4, #5, and #9). While this policy was implemented on December 15, 2017, the ROV has yet to have the opportunity to put the policy into practice. ROV Policy - Decision Matrix_Implemented 121517 has been attached separately as Supporting Documentation.

California State Auditor's Assessment of 60-Day Status: Partially Implemented

Although Santa Clara established a policy which requires it to include information about errors in voting materials in its post-election reports, it has not yet demonstrated that it is following this policy. Until Santa Clara can demonstrate that it is following this policy, we will continue to report the status of this recommendation as partially implemented.

Recommendation #11 To: Secretary of State, Office of the

The Secretary of State should adopt regulations establishing clear criteria for mistakes in election-related materials that constitute reportable errors and require counties to report these errors to it after each election.

Annual Follow-Up Agency Response From September 2023

The Secretary of State continues to provide ongoing guidance to county elections officials through weekly meetings, guidance documents, and memorandums to minimize the potential for errors in election-related materials. The Secretary of State also continues to request that counties timely report to our office any significant errors in election materials such as typographical, missing information, incorrect information, translations, mailing related, and technical errors. When errors are brought to our attention, we immediately work with the county elections official to mitigate the issue and minimize any potential impact to voters. The Secretary of State continues to research the authority to develop regulations to establish criteria for mistakes in election-related materials that constitute reportable errors.

California State Auditor's Assessment of Annual Follow-Up Status: Pending

Annual Follow-Up Agency Response From October 2022

The Secretary of State is in the process of researching the development of regulations to establish criteria for mistakes in election-related materials that constitute reportable errors after the completion of the November General Election canvass in early December 2022. The Secretary of State continues to request that counties report to our office any substantive errors in election-related materials provided to voters so that we may assist in mitigating those errors as they occur and track them. If an issue occurs, we share with the reporting county what other counties have done to mitigate similar issues.

California State Auditor's Assessment of Annual Follow-Up Status: Pending

Annual Follow-Up Agency Response From October 2021

Due to the unexpected statewide gubernatorial recall election, we will begin developing regulations to establish criteria for mistakes in election-related materials that constitute reportable errors after the completion of the gubernatorial recall election canvass in late October 2021. For the 2021 recall election, the Secretary of State again requested that counties report to our office any substantive errors in election-related materials provided to voters so that we may assist in mitigating those errors as they occur and track them, as we did in 2018 and 2020.

California State Auditor's Assessment of Annual Follow-Up Status: Pending

Annual Follow-Up Agency Response From October 2020

As previously reported, we will begin developing regulations to establish criteria for mistakes in election-related materials that constitute reportable errors after the completion of the 2020 election cycle. For the 2020 election cycle, the Secretary of State again requested that counties report to our office any substantive errors in election-related materials provided to voters so that we may assist in mitigating those errors as they occur and track them, as we did in 2018.

California State Auditor's Assessment of Annual Follow-Up Status: Pending

Annual Follow-Up Agency Response From October 2019

The Secretary of State is currently in the process of developing and promulgating three sets of statutorily required regulations related to the administration of elections that must be in place before the 2020 election cycle. We will begin developing regulations to establish criteria for mistakes in election-related materials that constitute reportable errors after the completion of the 2020 election cycle. For the 2020 election cycle, the Secretary of State will again request that counties report to our office any substantive errors in election-related materials provided to voters so that we may assist in mitigating those errors as they occur and track them, as we did in 2018.

California State Auditor's Assessment of Annual Follow-Up Status: Pending

1-Year Agency Response

The Secretary of State does not currently have the resources to develop and promulgate regulations during the 2018 election cycle, which is in progress; however, we plan to do so in the next non-statewide election year, 2019. For the 2018 election cycle, the Secretary of State has asked counties to report to our office any substantive errors in election-related materials provided to voters so that we may assist in mitigating those errors as they occur. In addition, we will use any reported errors to develop best practices to share with all county elections officials to assist them in the preparation of their election related materials.

California State Auditor's Assessment of 1-Year Status: Pending

6-Month Agency Response

The Secretary of State does not currently have the resources to develop and promulgate regulations during the 2018 election cycle, which has already begun; however, we plan to do so in the next non-statewide election year, 2019. For the 2018 election cycle, the Secretary of State has asked counties report to our office any substantive errors in election-related materials provided to voters so that we may assist in mitigating those errors as they occur. In addition, we will use any reported errors to develop best practices to share with all county elections officials to assist them in the preparation of their election-related materials.

California State Auditor's Assessment of 6-Month Status: Pending

60-Day Agency Response

The Secretary of State does not currently have the resources to develop and promulgate regulations during the 2018 election cycle, which has already begun; however, we plan to do so in the next non-statewide election year, 2019. For the 2018 election cycle, the Secretary of State will request counties report to our office any substantive errors in election-related materials provided to voters so that we may assist in mitigating those errors as they occur. In addition, we will use any reported errors to develop best practices to share with all county elections officials to assist them in the preparation of their election-related materials.

California State Auditor's Assessment of 60-Day Status: Pending

Recommendation #12 To: Secretary of State, Office of the

Beginning in December 2018, the Secretary of State should implement annual risk-based reviews of a selection of county election officials' offices to ensure their compliance with state election laws and regulations.

Annual Follow-Up Agency Response From September 2023

The Secretary of State provides ongoing guidance to county elections officials through weekly meetings, guidance documents, and memorandums and works in partnership with county elections officials to ensure compliance with state and federal election laws and regulations. The Secretary of State performs election observation in every state election in a selection of counties to observe and document compliance with election laws and rules. In addition, the Secretary of State will work with a selection of county elections officials to form a working group on developing best practices for reducing errors in the preparation of election materials. The topics of these best practices will be based on the reports of errors received from counties. The best practices will then be shared with all counties in order to help reduce errors. In addition, they will be used to develop potential trainings for counties.

California State Auditor's Assessment of Annual Follow-Up Status: Pending

Annual Follow-Up Agency Response From October 2022

The Secretary of State continues to explore the appropriate framework and potential resources and funding options to conduct annual risk-based reviews of a selection of county elections officials' offices to ensure compliance with election laws and rules. The reports of errors we received from counties during the 2018, 2020, 2021, and 2022 election cycles will help inform the development of any proposals for conducting these risk-based reviews.

California State Auditor's Assessment of Annual Follow-Up Status: Pending

Annual Follow-Up Agency Response From October 2021

The Secretary of State will continue to explore potential resources and funding options to conduct annual risk-based reviews of a selection of county elections officials' offices to ensure compliance with election laws and rules based on the reports of errors we received from counties during the 2018, 2020, and 2021 election cycles.

California State Auditor's Assessment of Annual Follow-Up Status: Pending

Annual Follow-Up Agency Response From October 2020

The Secretary of State will continue to explore potential resources and funding options to conduct annual risk-based reviews of a selection of county elections officials' offices to ensure compliance with election laws and rules beginning in December 2020 based on the reports of errors we receive from counties during the 2020 election cycle.

California State Auditor's Assessment of Annual Follow-Up Status: Pending

Annual Follow-Up Agency Response From October 2019

The Secretary of State is currently identifying potential resources and funding options to conduct annual risk-based reviews of a selection of county elections officials' offices to ensure compliance with election laws and rules beginning in December 2020 based on the reports of errors we receive from counties during the 2020 election cycle.

California State Auditor's Assessment of Annual Follow-Up Status: Pending

1-Year Agency Response

The Secretary of State will begin conducting annual reviews of county elections officials' offices to ensure compliance with election laws and rules in December 2018 as resources allow.

California State Auditor's Assessment of 1-Year Status: Pending

6-Month Agency Response

The Secretary of State will begin conducting annual reviews of county elections officials' offices to ensure compliance with election laws and rules in December 2018 as resources allow.

California State Auditor's Assessment of 6-Month Status: Pending

60-Day Agency Response

The Secretary of State will begin conducting annual reviews of county elections officials' offices to ensure compliance with election laws and rules in December 2018 as resources allow.

California State Auditor's Assessment of 60-Day Status: Pending

Recommendation #13 To: Secretary of State, Office of the

To inform and enhance the guidance it provides to county election officials, the Secretary of State should analyze error reports and its risk-based review results to focus its guidance on topics most relevant to improving elections throughout the State.

Annual Follow-Up Agency Response From September 2023

The Secretary of State continues to request counties provide reports of substantive errors in the creation and distribution of election-related materials for each election, providing guidance to them regarding the appropriate corrective action and best practices based on prior reports. The Secretary of State will work with a selection of county elections officials to form a working group on developing best practices for reducing errors in the preparation of election materials. The topics of these best practices will be based on the reports of errors received from counties. The best practices will then be shared with all counties in order to help reduce errors. In addition, they will be used to develop potential trainings for counties.

California State Auditor's Assessment of Annual Follow-Up Status: Pending

Annual Follow-Up Agency Response From October 2022

The Secretary of State continues to request counties provide reports of substantive errors in the creation and distribution of election-related materials for each election, providing guidance to them regarding the appropriate corrective action and best practices based on prior reports. As further guidance and/or regulations are promulgated, error reports will be required from counties. Once the appropriate legal framework and resources are allocated for risk-based reviews and a process can be established, an analysis of the results can be used to provide guidance to county elections officials.

California State Auditor's Assessment of Annual Follow-Up Status: Pending

Annual Follow-Up Agency Response From October 2021

The Secretary of State requested counties provide reports of substantive errors in the creation and distribution of election-related materials during the 2018, 2020, and 2021 election cycles, providing guidance to them regarding the appropriate corrective action. Based on lessons learned from the reports, we will develop best practices for all county elections officials in the preparation of their election-related materials. After the promulgation of regulations, we will require error reports from counties.

California State Auditor's Assessment of Annual Follow-Up Status: Pending

Annual Follow-Up Agency Response From October 2020

The Secretary of State requested counties provide reports of substantive errors in the creation and distribution of election-related materials during the 2018 election cycle and the March 2020 election, providing guidance to them regarding the appropriate corrective action. We requested reports for the November 2020 election and will continue to review these reports from counties in order to provide guidance to them. Based on lessons learned from the reports, we will develop best practices for all county elections officials in the preparation of their election-related materials. After the promulgation of regulations, we will require error reports from counties.

California State Auditor's Assessment of Annual Follow-Up Status: Pending

Annual Follow-Up Agency Response From October 2019

The Secretary of State requested counties provide reports of substantive errors in the creation and distribution of election-related materials during the 2018 election cycle and provided guidance to them regarding the appropriate corrective action. We will continue to request and review these reports from counties in 2020 in order to provide guidance to them and will develop best practices for all county elections officials in the preparation of their election-related materials. After the promulgation of regulations, we will require error reports from counties.

California State Auditor's Assessment of Annual Follow-Up Status: Pending

1-Year Agency Response

The Secretary of State has requested counties provide reports of substantive errors in the creation and distribution of election-related materials in 2018 and will review the reports in order to provide guidance and best practices to all county elections officials in the preparation of their election-related materials. After the promulgation of regulations, we will require error reports from counties.

California State Auditor's Assessment of 1-Year Status: Pending

6-Month Agency Response

The Secretary of State has requested counties provide reports of substantive errors in the creation and distribution of election-related materials in 2018 and will review the reports in order to provide guidance and best practices to all county elections officials in the preparation of their election-related materials. After the promulgation of regulations, we will require error reports from counties.

California State Auditor's Assessment of 6-Month Status: Pending

60-Day Agency Response

The Secretary of State will request counties provide reports of substantive errors in the creation and distribution of election-related materials in 2018 and will analyze the reports in order to provide guidance and best practices to all county elections officials in the preparation of their election-related materials. After the promulgation of regulations, we will require error reports from counties.

California State Auditor's Assessment of 60-Day Status: Pending

All Recommendations in 2017-107

Agency responses received are posted verbatim.