AI Model Cards: What to Include and Why They Matter
Model cards — standardised documents that describe an AI model's capabilities, limitations, and intended use — are moving from academic best practice to regulatory requirement. Under the EU AI Act, providers of high-risk AI systems must maintain technical documentation that covers much of what a model card addresses.
What is a Model Card?
Originally proposed by researchers at Google in 2018, a model card is a structured document that accompanies a machine learning model. Think of it as a nutrition label for AI — it tells you what the model does, how it was trained, where it works well, and where it does not.
Essential Sections
1. Model Overview
Basic identification: model name, version, provider, architecture type, and deployment method. This section answers "what is this?"
2. Intended Use
Clearly state what the model is designed for and, critically, what it is not designed for. Misuse prevention starts with clear scope documentation.
3. Training Data
Describe the data used to train the model: sources, size, preprocessing steps, and any known biases in the training set. This is mandatory under EU AI Act Article 10 (Data and Data Governance).
4. Evaluation Metrics
Quantitative performance metrics: accuracy, precision, recall, F1 score, and any domain-specific metrics. Include performance across different demographic groups if applicable.
5. Bias and Fairness Assessment
Document known biases, the methods used to detect them, and any mitigation measures applied. This is increasingly expected by regulators and essential for high-risk systems.
6. Limitations and Risks
What the model cannot do, known failure modes, and scenarios where it should not be trusted. Honest limitation documentation builds trust and meets regulatory transparency requirements.
7. Ethical Considerations
Privacy implications, potential for harm, human oversight requirements, and any ethical review the model has undergone.
8. Maintenance and Updates
How often the model is retrained, who is responsible for updates, and how performance is monitored post-deployment.
EU AI Act Requirements
For high-risk AI systems, Article 11 requires technical documentation covering:
- General description of the AI system
- Detailed description of development process elements
- Monitoring, functioning, and control of the AI system
- Risk management measures
- Changes throughout the system lifecycle
Model Cards as Evidence
A model card is one of the strongest pieces of evidence you can hold for AI-governance requirements: it substantiates intended use, training data, evaluation, and limitations in one place. Under the EU AI Act and the NIST AI RMF, that documentation is exactly what a requirement-level assessment looks for.
Netallion assesses the model cards and supporting documentation you hold against each applicable AI-governance requirement, determining whether the evidence is sufficient and flagging explicit blockers where it is not — for example, a model card that describes evaluation metrics but shows no evidence of a measured bias assessment. See the assessment methodology, or how the product reads evidence against requirements.