A featured contribution from Leadership Perspectives, a curated forum for enterprise technology leaders, nominated by our subscribers and vetted by the CIOApplications Editorial Board.
Chief Information Officer At City Of San Diego
Getting Started with Generative AI in the Public Sector
Getting started with emerging technologies like generative AI can be challenging, especially when technology staff and operational users have limited exposure and training to draw. Generative AI has the potential for significant efficiency increases, but organizations need to be mindful of the risks to succeed.
Risks
The emergence of ChatGPT and other generative AI solutions has resulted in lessons learned for many organizations. Some did not realize that whatever they input into the prompt became part of the data used by the learning model. Organizations must know the risks of sharing sensitive data, PII, proprietary content, and data privacy. Questions about copyright infringement, inaccurate data, or algorithmic bias can create liabilities if not managed properly. Even if the appropriate development and guardrails are implemented, there are additional risks if a user blindly accepts the model's output without validating its accuracy. Generative AI also creates new cyber security risks from adversaries, using it to streamline attacks and potentially increase vulnerabilities as new solutions are implemented.
Policy
The best starting point is to develop a comprehensive policy to govern a generative AI system's procurement, operation, security, standards, and maintenance. The policy development for AI is in the early stages for most organizations. However, early examples include resources from NIST, various federal agencies, state and local governments, the Gov AI Coalition, academic institutions, and commercial entities to draw from. Some organizations are integrating AI policies into existing IT governance, procurement, security, and privacy policies, while others are opting to create a separate policy for AI.
Getting Started with a Use Case
A small pilot project may be the best starting point to familiarize IT teams and operational users with implementing generative AI and realizing the benefits of a narrow internal use case. There are many opportunities to identify beneficial use cases across functions that require significant amounts of time to research documents, manual compilation or summarization of multiple data sources, call center knowledgebases, language translation, streamlining top document or content searches on an intranet site, or creating a generative AI workflow for various step processes that are currently disconnected and slow due to users having to look up information for each step.
Training Tech Teams
Getting technical teams proficient in developing generative AI solutions and getting familiar with best practices to train the model will take time. Many vendor partners will assist organizations in getting off the ground, and a small pilot can build the necessary skills and experience to scale a solution once it is developed. A small pilot will provide perspective to evaluate a solution's effectiveness, technology requirements, security, costs, ease of use, support needs, maintenance requirements, and scalability. One of the first things many organizations realize when they do a pilot is that their training data needs to be improved. Stale data, poorly written documents, and poor data quality can result in a generative AI solution that delivers biased, inaccurate, or poor responses.
Like any new technology implementation, introducing a new generative AI solution will require ongoing maintenance and support resources to keep the solution current, effective, secure, and supported.
Testing and User Training
Generative AI requires a different approach for user testing and training. Traditional IT test scripts usually have clear success criteria and defect reporting. Traditional user training usually follows workflows across specific screens and functions. Training for the use of generative AI is different. It is best accomplished through best practices for prompt engineering and familiarizing users with good approaches to get the information they seek. Users must also know how to validate the information they receive from the generative AI prompt, usually through hyperlinks to cited sources.
When testing a generative AI model, there will likely be a diverse approach to how users interact with the prompt, even though they request the same information. A feedback mechanism is critical to capture user sentiment when the prompt returns information so that issues can be identified and corrected through iterative testing. Even with consistent user feedback on the effectiveness and performance of a solution, there may be situations where multiple users receive the same response. Still, one will rate it as performing well, and another may rate it as performing poorly.
Scaling Up and Operational Support
Like any new technology implementation, introducing a new generative AI solution will require ongoing maintenance and support resources to keep the solution current, effective, secure, and supported. An effective pilot project may generate additional demand for similar use cases from other business functions. The information from the pilot project on the effectiveness, technology requirements, security, costs, ease-of-use, support needs, and maintenance requirements will be important in scaling up the solution further with the appropriate resources and budget to support it.
Generative AI technology will likely continue to evolve rapidly. Still, one thing that we can be sure of is that it is here to stay and will likely introduce significant benefits and disruption to traditional public sector services in the future.
