Report 2018-114 Recommendation 13 Responses

Report 2018-114: Disabled Veteran Business Enterprise Program: The Departments of General Services and Veterans Affairs Have Failed to Maximize Participation and to Accurately Measure Program Success (Release Date: February 2019)

Recommendation #13 To: Veterans Affairs, Department of

To ensure that its outreach efforts are effective and result in a greater number of DVBE firms available that can provide the necessary goods and services awarding departments are seeking, CalVet should develop an outreach plan to include outreach activities found to be effective in the past based on its assessment. This plan should also emphasize outreach to increase the number of DVBE firms that provide the types of goods and services that awarding departments struggle to obtain from DVBE firms.

Annual Follow-Up Agency Response From October 2022

CalVet developed several mechanisms to gain a deeper understanding of the needs of DVBEs and prospective DVBEs, as well as the needs of state agencies and departments. To build a comprehensive, data driven outreach plan, CalVet developed several surveys directed toward multiple respondents to capture feedback and inquiries made to CalVet regarding the DVBE program. CalVet conducts a monthly survey of newly certified DVBEs to ensure that firms are aware of the resources available to them since becoming certified, state contracting opportunities, best practices when procuring with the state, and user awareness of Cal eProcure and its vital role in the procurement process connecting their business with state purchasers. The DVBE State Advocate Survey was also designed and implemented to identify areas where CalVet can outreach and assist agencies and departments in procuring from DVBEs. In addition, to enhancing outreach and training for existing and new DVBEs. Survey question content was specific to capturing goods and services awarding departments struggle to procure from DVBEs. CalVet also created the Request for Response survey which allows the DVBE community, small businesses, state advocates, contractors, veterans, and the general public to contact CalVet regarding the State of California's DVBE program. This survey provides key insights on the specific needs of the respondents, and allows CalVet to continually evolve it's outreach efforts to target those specific needs. In addition to surveys, CalVet established monthly meetings with certification officers from DGS-OSDS to effectively track and engage with those who expressed specific interest in becoming a certified DVBE, resulting in a 25% success rate of those veteran businesses becoming certified DVBEs. Over the last 12 months, CalVet has utilized data and information collected through the described mechanisms to develop a data driven outreach plan to further support of the DVBE program.

California State Auditor's Assessment of Annual Follow-Up Status: Fully Implemented

Annual Follow-Up Agency Response From October 2021

In order to support the development of a comprehensive outreach plan, CalVet has implemented several instruments to gain a deeper understanding of the needs of established and prospective DVBEs, as well as state agencies. Assessments conducted by CalVet over the last year found that there were several opportunities to improve outreach efforts. In response, CalVet implemented several surveys to capture feedback and inquiries made to CalVet about the DVBE program. The purpose of the Certified DVBE Survey is to gather information from newly certified Disabled Veteran Business Enterprise firms. The purpose of the Advocate Survey is to identify areas where CalVet can assist departments in procuring from DVBEs. In addition to enhancing outreach and training for existing and new DVBEs. Survey question content was specific to capturing goods and services awarding departments struggle to procure from DVBEs. Another tool developed is the Request for Response survey which allows the DVBE Community, small businesses, state advocates, contractors, veterans, and the general public to contact CalVet regarding the State of California's DVBE program. CalVet also established a collaborative effort with certification officers from DGS-OSDS to effectively track and engage with those who expressed specific interest in becoming a certified DVBE. Over the next 6 months, CalVet will use the data and information collected through these instruments to develop a data driven outreach plan to further support of the DVBE program.

California State Auditor's Assessment of Annual Follow-Up Status: Pending

Annual Follow-Up Agency Response From October 2020

CalVet is currently reviewing the recently released DGS's Consolidated Annual Report (CAR) for FY 18-19. With the completion of hiring of DVBE staff on October 1, 2020, work will continue to prioritize which underachieving departments could benefit the most from CalVet support as well as establishing a method to support all DVBE advocates in their respective departments. The staff funded through the IA are currently working with DGS and department DVBE Advocates through DGS's advocates program and will perform research on the goods and services all awarding departments struggle to obtain from DVBE firms.

The areas of need identified will also allow CalVet to work with the revamped DVBE Advisory Council to establish a meaningful, community-informed outreach strategy that works to grow the number of DVBEs across the state. This strategy will also include recommendations on how to disseminate real-time procurement opportunities to DVBEs, as well as mitigate challenges by acting as a culturally competent bridge between DVBEs and state contract opportunities.

Documented Evidence:

1) Duty Statements (signed) for the two DVBE employees

2) CalVet DVBE Advisory Council Process Draft

3) DVBE AC Process Flow Draft

California State Auditor's Assessment of Annual Follow-Up Status: Pending

1-Year Agency Response

In 2019, the Statewide DVBE Advocate, in coordination with DGS, hosted nine "Discovery Meetings" with underachieving departments as identified within the DGS Consolidated Annual Report FY 2017-18. These meetings informed DVBE advocates about best practices for finding qualified DVBEs and encouraged underachieving departments to participate in multiple community-based DVBE outreach events. Particular focus was concentrated on identifying those goods and services that awarding departments struggle to find through a DVBE contractor or subcontractor. Currently, CalVet anticipates the release of DGS's Consolidated Annual Report (CAR) for FY 18-19, which will help to evaluate the impact of these past outreach efforts. CalVet will continue visiting underachieving departments through 2020, upon implementation of the IA, to prioritize which underachieving departments could benefit the most from CalVet support.

The staff funded through the IA will perform research on the goods and services all awarding departments struggle to obtain from DVBE firms and compare this information to the procurement and certification data that will be made available from DGS and advocates. This will allow CalVet to establish baseline of areas of need. CalVet will then actively work to connect state DVBE advocates with DVBEs that can assist awarding departments in meeting their 3% goal.

The areas of need identified will also allow CalVet to work with the revamped DVBE Advisory Council to coordinate the outreach and certification efforts of DGS and DVBE-focused organizations to establish a meaningful, community-informed outreach strategy that works to grow the number of DVBEs across the state. This strategy will also include the dissemination of real-time procurement opportunities, as well as mitigate challenges by acting as a culturally competent bridge between DVBEs and state contract opportunities.

California State Auditor's Assessment of 1-Year Status: Pending

6-Month Agency Response

CalVet continues to implement DVBE Promotion and Outreach Enhancement Plan. For example, CalVet designed a tailored outreach event for CalTrans District 6 in Fresno, CA on May 22, 2019.

Approximately 50 vendors and 150 DVBEs and small businesses attended. CalVet facilitated one-on-one matchmaking sessions between DVBEs and state and local government buyers. On July 23, 2019, CalVet sent surveys to participating DVBEs. We asked them to evaluate the effectiveness of this event. The responses to the survey were submittted and are now under review within CalVet and DGS.

California State Auditor's Assessment of 6-Month Status: Pending

CalVet provided a draft outreach plan that includes a schedule for outreach events during fiscal year 2019-20. However, it has not yet assessed the effectiveness of its past outreach efforts in increasing the number of DVBE firms that become certified. Without such an assessment, it cannot ensure that its outreach efforts will be effective and will result in a greater number of DVBE firms available that can provide the necessary goods and services awarding departments are seeking.

60-Day Agency Response

CalVet agrees with this recommendation. Since January 2019, CalVet has scheduled five meetings with awarding agencies. CalVet will continue to develop a "DVBE Roadmap and Success Timeline" based on assessments of each agency. Tailored outreach events are being designed to suit the awarding department's needs accordingly. An example is below:

CalVet is currently running a DVBE Collaborative pilot program in Fresno County. The DVBE Collaborative consists of sister agencies from federal, local and state governments, and local non-profits that have a special interest in ensuring that veteran businesses are successful. These agencies include the following: Small Business Administration, Small Business Outreach Centers, Veteran Business Office Centers, Small Business Development Centers, Procurement Technical Assistant Centers, SCORE, and local non-profits and small business incubators. The DVBE Collaborative pilot program was created to bring awareness to the DVBE Program, increase the number of DVBEs, and align government agencies with qualified DVBEs based on their industry.

This first DVBE Collaborative pilot program meeting was held in February 2019. Two additional meetings took place in March and April. The meetings focus on strategic DVBE outreach planning as well as work to facilitate a targeted DVBE outreach and matchmaking event on May 22 at a local Fresno small business incubator. CalVet will review outcomes of that pilot program at the end of the year.

CalVet is developing pre-event and post-event surveys. Each survey will evaluate the effectiveness of contracting opportunities, allow CalVet to gauge which DVBE firms will be in attendance, follow-up with newly certified DVBEs that attended the event, and gauge contracting opportunities that DVBEs were able to accomplish with prime contractors.

California State Auditor's Assessment of 60-Day Status: Pending

All Recommendations in 2018-114

Agency responses received are posted verbatim.