Hacks for better AI procurement

Or, tips for navigating complex procurements in imperfect situations

We know that not everyone lives in a world where great tech procurement feels possible. AI technology is advancing quickly, which means the best solution today may not be the best solution in a few years, making flexibility and adaptability more important than ever. But maybe you have no institutional AI strategy, feel unable to buy technology using an agile approach, or have decentralized procurement that makes it hard to ensure the right people are in the room.

In challenging environments, don’t let perfection be the enemy of “good enough.” Experimentation and iteration can help you better achieve your goals for a specific procurement. But this can also help you build confidence across your organization in new ways of thinking and working together, helping you shift norms around procurement practices.

Here are some strategies for those seeking to still get the most out of your AI procurements despite constraints, especially if you are buying customized or built solutions:

• Find your allies.
  Look for people across your organization from legal, IT, finance, and beyond who are good at getting things done, and willing to help you out. These allies can support you with your current procurement in the short term, and help with strengthening practices generally in the long term. You might also find inspiration in other cities, countries or agencies who have dealt with similar issues and might have ideas.
• Buy just what you need to get started. This might look like just buying research and development, a pilot, or software development services, rather than buying a final product right away with a large scope, budget, and duration.
• Engage the market early before you buy. There are many ways of letting potential vendors know that you are considering making a purchase. You can describe your plans publicly before you issue any Request for Proposals (RFPs) so they can get ready. If you can, start with a Request for Information (RFI) to gather market intelligence to inform your RFP.
• Make your solicitation agile-ish, even if you can’t be agile. Go beyond technology specifications to carefully describe the problem to be solved and intended outcomes, including as much detail as possible about the current situation (such as unmet user needs, past attempts to solve the same or similar problems, organizational constraints such as legacy IT systems, reduced funding and a lack of digital, data and technology skills, etc.), and your envisioned results. You can also articulate your needs in ways that agile developers are more likely to respond to and succeed. For example, if your government blocks certain words or concepts, you can use alternatives. For example, instead of “sprints” you can ask for weekly milestones, “show and tells” can be progress reporting, “demos” can be milestone reviews, and “user research” can be a stakeholder needs assessment.
• Don’t be afraid to ask basic questions during selection. Vendors should be able to tell you in plain language important information like what data they used to create their model, how they will use your government's data, and what success and failure looks like for their solution. You can also request information about model traceability. This can include the training data, the algorithm used, the hyperparameters, and the training environment. Even if a vendor is unable to share the training dataset itself, they should still be able to provide this general background. This information can help you understand the model outputs and identify potential issues.
• Aim for shorter contracts (with smaller budgets). AI technology is changing quickly, and what the best solution is now might be obsolete in two, five or ten years. Shorter contracts can help get the best technology to meet your government’s current needs, and manage risks.
• Make room for improvements (without change orders). By incorporating guarantee periods or structuring flexibility into the contract (such as through options), buyers can better adapt the scope or request changes if the results are not meeting expectations.
• Piggyback off existing contracts to get services and features, this could be task orders, cooperative contracts or other pre-approved vendors.

What if I’m just buying off-the-shelf AI?

The most common way that public sector organizations are procuring AI solutions is through framework agreements or multiple awards. Although buying AI this way is more straightforward, it’s still important to pay attention to several topics that we cover throughout this guidance, including:

• Use case: What you need should inform what you buy and how you contract.
• Data governance and security: You should understand how your organization’s data will be used.
• Testing and experimentation: Before rolling out organization-wide, strongly consider pilots to help mitigate risks by creating time and space for learning what works and what does not.
• Engaging your users, employees, and the public when possible: Input from your end users, employees, and the public along the procurement journey can be very helpful, especially if the solution will be public-facing, or help inform decision-making for service delivery.

You can also aim to be as agile and innovative as is feasible while working inside your pre-approved structures. For example, you can:

• Use framework agreements or multiple award schedules (MAS) as your foundation, not your final destination: Consider if your project team can build custom prompts, models, or applications using the enterprise-licensed AI as your foundation, such as via an API.
• Negotiate for a pilot within your framework agreement: If this is possible, you could test different AI technologies before starting a full procurement.
• Use procurement to add new functions: Consider if you might…
  - Piggyback off existing contracts to get new features, such as through task orders or call-offs.
  - Partner with smaller, agile vendors that are already on the MAS. They might be able to work with you through existing contracts.
  - Leverage cooperative agreements: Some schedules allow you to use solutions procured by other agencies.