Do We Need Regulation of High-Risk AI?
An Introduction to the Artificial Intelligence Act of the European Union
Artificial intelligence (AI) is moving so fast these days that some people - including Yoshua Bengio, Elon Musk, Steve Wozniak, and others — have signed a letter to stop its rapid progress [1]. Apparently, the topic of AI regulation is becoming increasingly important. For this reason, this article will focus on the Artificial Intelligence Act (AIA) of the European Union.
What is the Artificial Intelligence Act?
The proposal for a regulation on AI was announced by the European Commission in April 2021. It aims to address the risks of certain uses of AI, categorizing them into 4 different levels: unacceptable risk, high risk, limited risk, and minimal risk. With this risk-based approach, the AI regulation wants to ensure that Europeans can trust the AI they use [2].
Let’s look at each of the four risk levels in more detail.
AI systems with unacceptable risk
AI systems fall into this category when their use is considered unacceptable. For example, by violating fundamental rights, manipulating people, or exploiting vulnerable groups.
In addition, AI-based social scoring and real-time remote biometric identification systems (e.g., facial recognition) in publicly accessible spaces for law enforcement are explicitly prohibited.
All AI systems in this category will be banned by the AIA.
AI systems with high risk
AI systems that pose a high risk to the health and safety or fundamental rights of natural persons fall into this category. What matters is not only the function performed by the AI system, but also its intended use.
The AIA specifies two main categories for high-risk AI systems:
- AI systems intended to be used as safety components of products
- stand-alone AI systems with mainly fundamental rights implications that are explicitly listed in Annex III of the AIA [2]
High-risk AI systems will be allowed on the European market if they meet certain mandatory requirements and undergo a conformity assessment.
The capAI conformity assessment procedure was developed and published by Floridi et al. for AI systems under the AIA [4].
The regulatory requirements cover many topics, such as data and data governance, documentation and record keeping, transparency and provision of information to users, human oversight, robustness, accuracy, and security.
AI systems with limited risk
AI systems in this category must mitigate specific risks of manipulation by providing transparency. This applies to systems that (1) interact with humans, (2) are used to detect emotions, or (3) generate or manipulate content (deep fakes).
If an AI system is used to generate or manipulate content, there will be an obligation to disclose that the content was generated by automated means.
AI systems with minimal or no risk
The AIA allows the free use of minimal-risk AI. The majority of AI systems currently in use in the EU fall into this category.
Conclusion
Based on the AIA, generative AI such as ChatGPT, Midjourney, or DALL-E would be classified as being at least of limited risk. This would require these AI systems to provide more transparency that the content is AI-generated.
Self-driving cars would be classified as high-risk, which would require them to go through a certification process.
Because the AIA looks at the intended use of AI systems, ChatGPT, for example, could be deemed unacceptable if it were used in a way that attempted to manipulate people.
What do you think of the EU’s risk-based approach with the AIA proposal?
References
[1] Y. Bengio et al. (2023): Pause Giant AI Experiments: An Open Letter (accessed: 02.04.2023)
[2] European Commision (2021): Proposal for a Regulation laying down harmonised rules on artificial intelligence
[3] European Commision (2022): Regulatory framework proposal on artificial intelligence
[4] L. Floridi, M. Holweg, M. Taddeo, J. A. Silva, J. Mökander, Y. Wen (2022): capAI — A Procedure for Conducting Conformity Assessment of AI Systems in Line with the EU Artificial Intelligence Act, Available from SSRN
