Key questions to ask: Overview

Who	Procurement, project, and technology teams, as well as leadership
What	Questions to ask during the AI procurement processes

This is an overview of key questions featured in this guidance to help your organization make informed decisions and implement AI solutions responsibly and effectively. These questions are drawn from the AI readiness, Building your procurement strategy, Solicitation tips, and Contract management sections.

Successful AI procurement is a team effort, so as a procurement official, you are unlikely to have all the answers. However, understanding generally what questions need to be asked, and when, can help you facilitate internal conversations and ensure that the right experts are involved in decision-making.

The questions have been organized by the team member who may be best positioned to lead the inquiry. As you review these questions, consider your own unique organizational setup and who in your organization might be best equipped to provide insights on each topic. At different moments, you may need to consult IT, legal, data analytics, project management, or business units to get the answers you need, as well as consult outside stakeholders who advocate or represent the end users.

Not all questions will be relevant for your context. These questions are most relevant for custom or build purchases. For example, if you are buying off-the-shelf AI, procurement will mostly focus on aspects like licensing and access management, while if you are customizing or building software, the procurement team will work more closely with the project leaders and IT to tackle more complex issues relating to the vendor relationship and technical solution.

AI readiness

Team	Questions
Procurement	General Can we use agile methods for our IT procurements? How can our procurement processes support strong collaboration between technical teams, procurement, legal, and policy stakeholders? What does the market for this product or service look like? How can I engage them, both to get input into the best specifications and to ensure a good, competitive process? Procurement risks How do we guard against vendor lock-in and ensure that contracts include clear exit strategies and data portability? What protections do we need in place to clarify data rights, intellectual property ownership, and the handling of sensitive information? What is the state of our data, and how much does that need to be considered in the procurement strategy?
Agency or department buyer	General How do our overall organizational mission and goals align with our AI initiatives? What are our internal capability gaps?
IT/Data analytics	General Who inside the organization can take the lead on AI and connect the dots across teams? What governance structures and frameworks do we need for responsible and ethical AI adoption? Do we have the necessary AI literacy and understanding across relevant stakeholders? How will we manage internal change and encourage safe adoption of AI tools? How will we ensure inclusive design and testing of AI tools to reflect diverse public needs?

Building your procurement strategy

Team	Questions
Procurement	General Are we engaging the right internal and external stakeholders in our strategy development process? Is our budget realistic, given ongoing maintenance needs?
Agency or department buyer	Model and system architecture What is the problem we are trying to solve? What are our needs and use cases? How will this project impact end users, including public sector staff, frontline workers and communities or service recipients? What are the expected outcomes of the solution we want to acquire? How might we safely test our assumptions, approach, and AI tool options before launching the formal procurement process? Operational considerations Which role or team will be accountable for managing risk and compliance? Do we have the internal capacity for this oversight?
IT/Data analytics	Model and system architecture Is AI the right tool to meet our needs? What deterministic and rule-based solutions might deliver comparable results to generative AI for our use case? If we pursue AI, what type of model best suits our specific use case (e.g., general-purpose vs. specialized models)? What integration capabilities do we need with existing systems? What is our internal capacity to maintain the tool? Where will the tool be hosted? Will we need to acquire additional processing or storage capacity? What are environmental considerations?

Solicitation tips

Solicitation tips: Key questions

Procurement and organizational objectives

Decide on your procurement procedure
Draft your solicitation

Team	Questions
Procurement	Project needs How might we reflect the project team’s needs in a RFP? Can the vendor demonstrate a roadmap and flexibility that aligns with our evolving needs? What level of vendor support or service guarantees will we need if something breaks or goes wrong? Will the vendor support integration with other AI tools or models we may adopt in the future? Operational considerations What kind of selection criteria should be used to rank solicitation responses? What commercial or pricing models are appropriate? Are the right people in the room for this procurement? Risk and compliance How will we ensure compliance with current and evolving AI-related legislation, procurement rules, and standards? How will we manage and track AI-specific costs (e.g., compute, storage, API usage)? How much risk does vendor lock-in pose in our context, and what mitigations are practical?
Agency or department buyer	AI solution Do we know what “good enough” model response performance looks like for our specific use case, and what metrics will we use to measure AI success and ROI (qualitative and quantitative)? What are our fallback strategies if the model gives low-quality or inaccurate responses? How will users interact with the AI solution? If there is a platform or dashboard, who is responsible for developing it? Can we retrieve or export our data and models if we switch vendors? Data strategy and management Do we and our vendor partners understand the quality, completeness, and relevance of our data? What role will the vendor play in helping us prepare or transform data for effective use? How will we manage data ownership, licensing, and usage rights? What safeguards will the vendor provide to ensure that sensitive, personal, or classified data is handled in line with legal and policy requirements? Will the AI system retain or learn from our data inputs? Under what terms, and with what controls? Is the algorithm specialized for our organization? If so, will we have input into how the algorithm is fine-tuned or retrained using our data? What additional data sources do we need to get the best results? Operational considerations How might we approach AI implementation in stages, such as through a pilot or sandbox? How will we measure and monitor model performance over time? What policies will govern human-in-the-loop oversight for critical decision-making? What mechanisms will we put in place for continuous performance monitoring and system health checks? What accident response mechanisms will we put in place? What processes will we follow to learn from accidents and improve their prevention and the management of associated risks and harms? If we are involved in retraining or fine-tuning the model, what internal processes will we establish for prompt engineering and optimization? Who owns operational responsibility for uptime, error handling, and incident response? What logging and audit mechanisms will be used for AI inputs and outputs? How will we ensure traceability and version control for prompts and model configurations? Risk and compliance What processes will we establish for bias detection, mitigation, and fairness audits? What controls will be in place for data privacy, consent, and security (at rest and in transit)? How will we meet model explainability requirements, especially for high-impact or regulated use cases? How will we manage IP and copyright issues relating to AI-generated content? How will we assess and audit third-party AI providers for security, ethical alignment, and accountability? What safeguards will we implement against hallucination, misuse, or reputational risk in public-facing services? How will we address bias detection and adversarial robustness?
IT/Data analytics	Project needs What are our requirements for model transparency and explainability? How will we measure and monitor model performance over time? What are our requirements for model updates and maintenance? What level of customization do we require? What are the latency, reliability, and availability requirements for our applications? Are there specific regulatory cybersecurity requirements, such as around data residency or geographic deployment? Integration How will the AI solution integrate with our existing systems? Does the vendor support open standards and interoperability (e.g., prompt formats, model interchange)? How do open-source and proprietary models compare in terms of cost, performance, and vendor support and accountability? Are there clear exit strategies or migration paths if we outgrow a particular vendor or need to switch platforms? Data strategy and management How will we jointly manage data governance – including storage, audit trails, and compliance with data retention and deletion policies? Are there opportunities to collaborate with vendors on domain-specific dataset creation or fine-tuning? What processes are in place for anonymization or redaction of data before it enters AI systems? How does the vendor support discoverability and metadata management for both inputs and outputs? How will data flows be documented, monitored, and audited across the lifecycle?

Team

Questions

Procurement

Project needs

How might we reflect the project team’s needs in a RFP?
Can the vendor demonstrate a roadmap and flexibility that aligns with our evolving needs?
What level of vendor support or service guarantees will we need if something breaks or goes wrong?
Will the vendor support integration with other AI tools or models we may adopt in the future?

Operational considerations

What kind of selection criteria should be used to rank solicitation responses?
What commercial or pricing models are appropriate?
Are the right people in the room for this procurement?

Risk and compliance

How will we ensure compliance with current and evolving AI-related legislation, procurement rules, and standards?
How will we manage and track AI-specific costs (e.g., compute, storage, API usage)?
How much risk does vendor lock-in pose in our context, and what mitigations are practical?

Agency or department buyer

AI solution

Do we know what “good enough” model response performance looks like for our specific use case, and what metrics will we use to measure AI success and ROI (qualitative and quantitative)?
What are our fallback strategies if the model gives low-quality or inaccurate responses?
How will users interact with the AI solution? If there is a platform or dashboard, who is responsible for developing it?
Can we retrieve or export our data and models if we switch vendors?

Data strategy and management

Do we and our vendor partners understand the quality, completeness, and relevance of our data?
What role will the vendor play in helping us prepare or transform data for effective use?
How will we manage data ownership, licensing, and usage rights?
What safeguards will the vendor provide to ensure that sensitive, personal, or classified data is handled in line with legal and policy requirements?
Will the AI system retain or learn from our data inputs? Under what terms, and with what controls?
Is the algorithm specialized for our organization? If so, will we have input into how the algorithm is fine-tuned or retrained using our data? What additional data sources do we need to get the best results?

Operational considerations

How might we approach AI implementation in stages, such as through a pilot or sandbox?
How will we measure and monitor model performance over time?
What policies will govern human-in-the-loop oversight for critical decision-making?
What mechanisms will we put in place for continuous performance monitoring and system health checks?
What accident response mechanisms will we put in place? What processes will we follow to learn from accidents and improve their prevention and the management of associated risks and harms?
If we are involved in retraining or fine-tuning the model, what internal processes will we establish for prompt engineering and optimization?
Who owns operational responsibility for uptime, error handling, and incident response?
What logging and audit mechanisms will be used for AI inputs and outputs?
How will we ensure traceability and version control for prompts and model configurations?

Risk and compliance

What processes will we establish for bias detection, mitigation, and fairness audits?
What controls will be in place for data privacy, consent, and security (at rest and in transit)?
How will we meet model explainability requirements, especially for high-impact or regulated use cases?
How will we manage IP and copyright issues relating to AI-generated content?
How will we assess and audit third-party AI providers for security, ethical alignment, and accountability?
What safeguards will we implement against hallucination, misuse, or reputational risk in public-facing services?
How will we address bias detection and adversarial robustness?

IT/Data analytics

Project needs

What are our requirements for model transparency and explainability?
How will we measure and monitor model performance over time?
What are our requirements for model updates and maintenance?
What level of customization do we require?
What are the latency, reliability, and availability requirements for our applications?
Are there specific regulatory cybersecurity requirements, such as around data residency or geographic deployment?

Integration

How will the AI solution integrate with our existing systems?
Does the vendor support open standards and interoperability (e.g., prompt formats, model interchange)?
How do open-source and proprietary models compare in terms of cost, performance, and vendor support and accountability?
Are there clear exit strategies or migration paths if we outgrow a particular vendor or need to switch platforms?

Data strategy and management

How will we jointly manage data governance – including storage, audit trails, and compliance with data retention and deletion policies?
Are there opportunities to collaborate with vendors on domain-specific dataset creation or fine-tuning?
What processes are in place for anonymization or redaction of data before it enters AI systems?
How does the vendor support discoverability and metadata management for both inputs and outputs?
How will data flows be documented, monitored, and audited across the lifecycle?

Contract management

Buying AI

Region

Share

Key questions to ask: Overview

AI readiness: Key questions

Building your procurement strategy: Key questions

Solicitation tips: Key questions

Contract management: Key questions

This is a resource by
Open Contracting Partnership