Plan and design: scoping the system
Design is where governance is cheapest and most powerful.
The intake questions
- What problem are we solving, and is AI the right tool? A rule or a human process may be safer and cheaper.
- Who is affected? Users, subjects of decisions, bystanders, vulnerable groups.
- What could go wrong, for whom, and how badly?
- Which laws apply? EU AI Act tier, GDPR, sector rules, anti-discrimination law, state laws.
- What data will we need, and do we have the right to use it?
- How much autonomy will it have, and where will humans stay involved?
Classify risk, then scale the process
The answers place the system in a risk tier. The tier sets which assessments, tests, approvals and monitoring are required. Rate each risk by probability and severity, in the context of this use.
Map the stakeholders
List everyone the system touches, not only the team building it: the business owner, users, the people its outputs are about, people affected indirectly, regulators, and the functions that must sign off (legal, privacy, security, risk, HR). Plot each group by how much it's affected and how much influence it has. The groups most affected with the least say, such as job applicants, are the ones most often missed, so plan how to hear from them: research, pilots and feedback channels.
Mitigate in order
When you find a risk, work down a hierarchy:
- Remove or reduce it by design: change the use case, the data or the model, or don't use AI.
- Add controls: thresholds, human review, monitoring, access limits.
- Inform and train the people who use the system.
- Accept what's left only with a named owner signing off.
The EU AI Act's risk-management article (Art. 9) follows the same order: design first, then mitigation and control measures, then information and training for deployers. When a question asks for the best control, it usually wants the highest step that's feasible.
Choose the assessment
| Assessment | When |
|---|---|
| AI impact assessment (AIA) | Organization's own process for any material AI use |
| DPIA (GDPR Art. 35) | Personal data processing likely to be high risk |
| FRIA (EU AI Act Art. 27) | Listed deployers of high-risk AI |
| Assessments a state or sector law requires | For example the independent bias audit NYC Local Law 144 requires for hiring tools |
Combine them where possible so teams fill in one integrated assessment.
Set requirements before building
- Performance metrics and the minimum acceptable level
- Fairness metrics and which groups to test
- Explainability needs: who needs to understand what
- Human oversight model: in, on or out of the loop
- Documentation that will be produced at each gate
Writing these down before training prevents moving the goalposts after results come in.
Check your understanding
Why define fairness metrics before training rather than after?
During intake, the team realizes a simple rules-based check would meet the need. What should governance encourage?