Procurement, third-party risk and contracts
Buying AI moves work to the vendor but not your accountability. Regulators hold deployers to their own duties.
Due diligence questions
- What is the system's intended purpose, and what uses are out of scope?
- What data trained it? Was it lawfully obtained and licensed?
- What testing was done (performance, bias, robustness, red teaming)? Can we see results?
- Will our data or prompts be used to train the vendor's models?
- Where are data stored and processed? Which sub-processors and underlying foundation models are involved?
- What documentation exists: model cards, instructions for use, EU declaration of conformity, ISO/IEC 42001 certification?
- How are changes, incidents and model updates communicated?
- What explanation and logging support does it offer for our obligations?
A vendor that refuses to share material testing information is itself a risk signal.
Contract terms
| Term | Why it matters |
|---|---|
| Permitted use and restrictions | Keeps use within the assessed scope |
| Data use, no training without consent, retention and deletion | Protects confidential and personal data |
| Audit and information rights | Lets you verify claims and meet your duties |
| Performance and accuracy warranties; SLAs | Gives recourse for under-performance |
| Bias testing commitments | Supports anti-discrimination compliance |
| IP indemnity | Covers infringement claims over outputs or training data |
| Incident and change notification | You learn about problems and model changes in time |
| Regulatory cooperation | Support with regulator requests and conformity duties |
| Termination, exit and transition | Lets you leave without losing data or continuity |
Open-source and foundation models
There may be no vendor to hold accountable. Check licence terms (including acceptable-use restrictions), do your own testing, and track the model's version and provenance.
Deploying your own model
A company that builds and deploys its own model has no vendor to share the risk. It takes on the provider's duties (under the EU AI Act, the full set of high-risk provider obligations where they apply), carries more liability when something goes wrong, and must produce the documentation and testing a vendor would otherwise supply. In return it gets full visibility into the data and the model, and full control over changes.
Ongoing third-party management
Due diligence is not one-off: re-review vendors periodically and when their model, terms or ownership change.
Check your understanding
Which clause BEST addresses the risk that a vendor changes its underlying model without warning?
A team wants to deploy an open-source LLM. What is a distinctive governance concern?