EU AI Act: What Are the Obligations for AI System Providers?
DPO Consulting Joins Grant Thornton
The EU AI Act distinguishes between several categories of operators and assigns different responsibilities to each of them. Two roles are particularly important: the provider and the deployer of an AI system.
This classification is far from insignificant. A company using an AI solution as part of its business activities is not subject to the same requirements as an organization developing and marketing that solution. Under certain circumstances, it may also change classification and become subject to the obligations applicable to AI system providers.
For organizations, one of the first steps towards EU AI Act compliance is therefore to determine precisely which role they play for each AI system they use or market.
Correctly identifying this role is also a critical first step in establishing a structured AI compliance and governance framework.
Under the EU AI Act, a provider is an operator that develops an AI system or general-purpose AI model, or has one developed, and places it on the market or puts the system into service under its own name or trademark.
A deployer is a natural or legal person, public authority, agency or other body using an AI system under its authority, except where the system is used in the course of a personal, non-professional activity.
Therefore, a company that purchases an AI solution from a third-party provider for use in its own internal processes will generally be considered a deployer. A company that develops and markets the solution under its own name will generally be considered a provider.
However, this classification is not fixed. An organization initially acting as a deployer, distributor, importer or other third party may become subject to the obligations of a provider where it:
The organization’s classification should therefore be reassessed whenever the system, its intended use or the conditions under which it is marketed change.
For high-risk AI systems, the EU AI Act places a significant share of the compliance obligations on the provider.
Providers must not only ensure that the system complies with the relevant requirements before it is placed on the market or put into service, but also organize and document its monitoring throughout its lifecycle.
An inventory and classification of AI systems and use cases can therefore help organizations identify their role, determine the relevant risk level and establish the applicable compliance requirements. Dicover our AI maturity assessment
A high-risk AI system must be accompanied by instructions for use containing the information required to enable deployers to understand its characteristics, capabilities and limitations and to use it appropriately.
These instructions should notably provide information on:
This documentation plays a central role. It enables the provider to define the conditions under which the system should be used while providing deployers with the information required to fulfil their own responsibilities under the EU AI Act.
Before placing a high-risk AI system on the market or putting it into service, the provider must comply with the applicable conformity requirements.
Depending on the system concerned, this notably involves:
However, the provider’s responsibility does not end at this stage. Where a provider considers that a system it has placed on the market or put into service does not comply with the EU AI Act, it must take the necessary corrective measures and, where required, inform the relevant operators and authorities.
The provider must also be able to cooperate with the competent authorities and demonstrate the compliance of its AI system.
Compliance with the requirements applicable to a high-risk AI system should not be limited to a one-off assessment before commercialization.
Providers must notably establish a documented quality management system covering different aspects of the AI system lifecycle, including design and development, testing and validation, risk management, change management, post-market monitoring and incident management.
Providers must also ensure appropriate documentation and traceability, notably through the technical documentation and logging mechanisms required by the EU AI Act.
Finally, a post-market monitoring system should enable the provider to collect and analyse relevant information on the system’s performance throughout its lifetime.
The objective is therefore to integrate EU AI Act compliance from the design stage and throughout the entire lifecycle of the system.
Although providers play a central role in ensuring the compliance of high-risk AI systems, deployers retain their own responsibilities regarding the conditions in which those systems are actually used.
Deployers must notably use the system in accordance with the instructions for use, ensure appropriate human oversight, monitor its operation and retain automatically generated logs under their control in accordance with the conditions laid down by the EU AI Act.
Additional obligations may apply depending on the specific context. For example, an employer using a high-risk AI system in the workplace may have to inform workers’ representatives and affected workers in accordance with the EU AI Act and other applicable rules.
In certain circumstances, deployers may also be required to carry out a fundamental rights impact assessment.
Where personal data processing is likely to result in a high risk to individuals’ rights and freedoms, the GDPR requirements concerning a Data Protection Impact Assessment (DPIA) must also be taken into account.
The interaction between the EU AI Act and GDPR should therefore form part of an organization’s AI governance framework, particularly where AI systems process personal data.
Finally, the transparency obligations established under Article 50 should be assessed according to the AI system and its specific use, as some obligations apply to providers while others apply to deployers.
The key challenge is therefore to properly coordinate the responsibilities of providers and deployers rather than assuming that compliance rests exclusively with one or the other.
The EU AI Act also establishes specific obligations for providers of general-purpose AI (GPAI) models.
They must notably:
Additional requirements apply to general-purpose AI models presenting systemic risks, notably regarding model evaluations, the monitoring and reporting of serious incidents and cybersecurity.
Provider obligations also have a highly operational dimension: providers must enable organizations using their AI systems to access the information they need to comply with their own obligations.
For a high-risk AI system, the deployer should notably have access to relevant information regarding:
The instructions for use are particularly important in this respect. The EU declaration of conformity and CE marking also provide evidence that certain regulatory requirements applicable to the system have been addressed.
However, a distinction should be made between the documents that providers are required to prepare or retain and those they are actually required to make available or disclose to deployers.
This distinction is particularly important during due diligence exercises: customers may legitimately seek assurances regarding compliance with the EU AI Act without necessarily having a right of access to all of the provider’s internal documentation.
For organizations developing or marketing AI solutions, identifying their role should be a central component of their AI system inventory and mapping process.
For each system or model, several questions can be documented:
This assessment should also be reviewed when the AI system or its use evolves. An organization that initially considers itself merely a user may, particularly following rebranding, a substantial modification or a change in intended purpose, become subject to the more demanding regime applicable to providers.
Correctly identifying each operator’s role is therefore one of the first building blocks of effective AI governance: before determining how an organization should comply with the EU AI Act, it must first understand the capacity in which it is acting.
Does your organization develop, market or use AI systems? DPO Consulting’s experts can support you in mapping your AI systems, determining your regulatory role and building an effective EU AI Act compliance roadmap: Book an appointment with an expert
.png)
.svg)
.png)
.png)
