AI adoption myth-busting

Reality: AI projects can require significant upfront investment and ongoing costs. ROI may take time to realize and depend on proper implementation and use. ROI discussions can evolve quickly, given the speed that the sector introduces innovations and improvements. Some tasks could be accomplished equally well by non-AI technology. During vendor evaluation, look for solutions for cost estimation, ROI projection, and optimization to help make informed decisions and maximize the value of AI investments. Plan for pilots to demonstrate value before making large investments.

Reality: Your specific use case should inform what technology solution you pursue, and AI might be a good fit. Lower-risk, proven ROI AI solutions do exist and continue to be developed. Risk and ROI can also be managed and understood through conducting small-scale pilots before taking solutions to scale. The advantages and disadvantages of AI are nuanced, and it’s worth articulating these through your organizational AI strategy.

Reality: AI systems require ongoing attention, though the level varies significantly. Managed cloud AI services may reduce operational overhead compared to self-hosted solutions, but all AI implementations need monitoring for performance, accuracy, and bias over time. Models may need updates as data patterns change, and business requirements evolve. During vendor evaluation, consider both the vendor's managed service capabilities and your organization's capacity for ongoing AI system management. Look for solutions that match your technical resources and operational preferences.

Reality: Effective AI implementation typically augments rather than replaces human judgment. Different levels of automation are appropriate for different contexts and risk levels.

Reality: Large language models are foundational, but fine-tuning, prompt engineering, and guardrails are critical to make them useful and safe in specific contexts. Different models may be optimal for different use cases and considerations, such as cost, latency or built-in safeguards and bias mitigation. Look for vendors that have the capability to help adapt foundation models to unique needs, embedding safeguards and improving accuracy for specific use cases.

Reality: While transparency is important, its meaning and implementation can vary. Transparency in AI can encompass different technological and operational aspects, such as:

• Model architecture and training data
• AI system capabilities and limitations
• Clarity for users when they are interacting with AI-enabled systems

When considering vendors, look for clear documentation on model purpose, inputs, known limitations, and responsible use cases tailored to public sector needs, commitment to user transparency, such as clear labeling of AI-generated content or interactions, regular updates on model changes and improvements and willingness to discuss the balance between proprietary technology and transparency needs. Remember: Full technical transparency of large models isn't always feasible or meaningful to non-technical users. Focus on transparency that enhances trust, supports responsible use, and aligns with your organization's needs and values, and which will allow your organization to discharge applicable duties – for example, to provide reasons for decisions.

Reality: Not all AI decisions can be explained like a rulebook – context matters. Seek out vendors with support for guardrails that meet the requirements for explainability /interpretability for your specific use case.

Reality: AI systems can perpetuate or amplify existing biases if not carefully designed and monitored. They require robust testing and ongoing oversight, especially for agentic AI making consequential decisions.

Reality: Integration complexity can vary significantly. While some AI systems, particularly within the same vendor ecosystem, can integrate smoothly, cross-vendor integration often requires additional effort. Different GenAI systems often speak different languages – from varying API structures and data formats to different security protocols and performance standards. Making systems work together requires significant integration effort, standardization, and often custom middleware solutions. During vendor evaluation, look for vendors that support open standards, APIs, and modular architectures that allow AI solutions to integrate into existing IT environments and minimize rework when switching providers, clear data processing and storage policies and transparent terms regarding ownership of customizations and prompts. Integration challenges can also vary by AI type. While GenAI might need API alignment, agentic AI requires additional considerations around decision-making protocols, audit trails, and responsibility frameworks.

Reality: You can move the data, but models may need reconfiguration or even retraining when moved to new environments. Look for vendors that have support for standard model formats and container technologies, structured export options and data export/import tools, standard data format support, multi-cloud and cross-platform capabilities and assistance with planning migration strategies that minimize friction.

Reality: Lock-in is not binary. It's about the cost and effort to switch. Look for vendors that have data available for export, such as prompts, or effort you put in initially to customize/calibrate, as well as open-source model formats, containerization options
(e.g., Docker, Kubernetes), and hybrid/multicloud support to enable transitions when needed.

Reality: The goal is not zero risk – it's understood, mitigated, and monitored risk. During vendor evaluation, look for support for responsible AI frameworks, sandboxing environments, and robust logging to help assess and manage risks realistically.

Reality: Not necessarily, this depends on the terms of your contract. The vendor may want to use your data for training purposes elsewhere. Align with your vendor and be clear in your contract on how your organization's data will be used. Regulations like the General Data Protection Regulation (GDPR) in the European Union, or the California Consumer Privacy Act in the State of California, mainly protect personal data, but AI solutions can also process many other types of information as well.

Reality: Good governance can often build on existing positions and frameworks. You don’t need to duplicate efforts or add complexity to existing frameworks. Before creating new processes or roles, think about your existing organizational capacity and where AI governance structures might be a good fit, and where you already have expertise.

Reality: Open source AI offers benefits like transparency and customization flexibility, but it's not automatically better. While you may avoid licensing fees, open source solutions can require significant internal technical expertise, ongoing maintenance, security management, and support infrastructure. Given these requirements, total cost of ownership can sometimes exceed proprietary solutions. During early market engagements and vendor evaluation, look for features like clear explanations of which components are open source versus proprietary, support for major open-source frameworks and permissive licensing, secure and scalable environments for deployment, defined processes for updates and security patches, exit and transition arrangements including knowledge transfer, and clear support and maintenance agreements.

Reality: Neither open source nor proprietary AI is inherently safer. Security and safety depend on implementation quality, ongoing maintenance, vendor support, and your organization's technical capabilities. Open source allows for greater transparency and customization of security measures, while proprietary solutions may offer enterprise-grade support and managed security features. The right choice depends on your organization's technical capacity, security requirements, and risk tolerance. Consider which components are developed using open source, what has been developed by others and is being reused, your use case, your institutional capabilities and capacities, and the extent to which you need active technical support and frequent updates.

Reality: Governance needs vary significantly between different types of AI solutions and different use cases. While content generation might need review processes, automated decision-making requires stronger oversight, clear accountability chains, and specific intervention protocols.

Reality: It's not just about geography. While it is important to consider geopolitical implications on where the data is held, it's probably even more important to understand who has access, how it's encrypted, and who controls the encryption keys. During vendor evaluation, consider local hosting options and sovereign environments when needed, customer-managed encryption keys, and compliance tooling to meet residency and sovereignty regulations and needs in context.