AI Conversation – Part 3 – Regulatory

Regulatory Frameworks

Regulatory frameworks like the EU GDPR and the EU AI Act approach corporate algorithmic scoring and automated profiling through two distinct, highly complementary lenses.

While the GDPR regulates the data powering the decision and grants individuals structural rights to contest it, the EU AI Act regulates the system itself, imposing hard architectural limits and classification barriers based on how much risk the scoring model poses to human rights.

Here is how both frameworks attempt to check corporate profiling and scoring algorithms:

1. The EU AI Act: Structural Bans and High-Risk Gates

The EU AI Act targets algorithmic scoring by dividing AI systems into strict risk tiers, completely outlawing certain types of profiling while placing heavy compliance burdens on others.

• The Absolute Prohibition on Social Scoring (Article 5): The AI Act explicitly bans AI-enabled social scoring systems—whether deployed by governments or private corporations. If a company builds an algorithm that evaluates or classifies individuals or groups over time based on their social behavior, personality traits, or predicted lifestyle, resulting in detrimental or unfavorable treatment in unrelated contexts (e.g., a corporate algorithm tracking consumer habits to arbitrarily deny housing, insurance access, or employment opportunities), it is outlawed outright.
• The High-Risk Classification for Credit and HR: Any corporate scoring algorithm used to evaluate creditworthiness, determine credit scores, manage recruitment, filter job applicants, or monitor worker performance is automatically classified as High-Risk.
• Mandatory Constraints on High-Risk Scoring: To deploy a credit-scoring or worker-evaluation model legally, companies must implement:
• Data Governance: Rigorous training data protocols to ensure the scoring model doesn’t ingest biased variables that produce discriminatory outcomes.
• Traceability and Logging: Automatic logging of events and model outputs so auditors can trace why a specific score or denial was generated.
• Human Oversight: Designing the system so that a human operator can actively review, override, or reverse automated scores before they cause tangible harm to a citizen.

2. The GDPR: The Right to Explanation and Contestability

While the AI Act governs the design and deployment of the system, the GDPR governs the personal data processed by that system. It targets automated profiling via foundational rights enshrined in Article 22 (and mirrored in national data protection laws).

• The Restriction on Solely Automated Decisions: The GDPR traditionally restricts decisions based solely on automated processing (including profiling) that produce legal or similarly significant effects (such as an automated corporate system instantly rejecting a loan or a mortgage).
• The Right to Meaningful Information: If a corporation uses automated scoring to judge a consumer, the GDPR forces transparency. Companies cannot hide behind “black-box proprietary algorithms.” They must provide meaningful information about the logic involved, the significance of the profiling, and the anticipated consequences.
• Human Intervention on Demand: Under data protection guardrails, individuals targeted by automated corporate profiling have the enforceable right to demand that a human being look at their case, express their point of view, and contest the algorithmic score.

The Convergence: Catching the Corporate Black Box

When combined, these two frameworks dismantle the corporate excuse that “the algorithm decided it.”

If a company deploys an opaque algorithmic scoring model to evaluate human beings, they face a pincer movement: GDPR forces them to open the black box to the individual user and justify the score, while the AI Act forces them to prove beforehand that the scoring architecture is non-discriminatory, heavily logged, supervised by humans, and clear of prohibited “social scoring” traps.

---

Penalties for non-compliance

Both the EU AI Act and the GDPR use a “whichever is higher” formula for financial penalties—calculating the maximum fine using a flat baseline cash cap or a percentage of a corporation’s total worldwide annual turnover, whichever yields the greater amount.

For multinational technology corporations, the percentage-based global turnover caps dwarf the flat sums, exposing them to billions in potential losses.

1. The EU AI Act Penalties

The EU AI Act structures its maximum penalties into a tiered pyramid based on the severity of the violation. Breaches involving banned practices or data governance failures carry the harshest teeth ever written into European tech regulation.

• Tier 1: Prohibited AI Practices (e.g., Social Scoring & Behavioral Manipulation)
  • Applies to: Deploying banned systems like citizen social-scoring algorithms, subliminal manipulation, or exploitative biometric categorization.
  • Maximum Penalty: Up to €35 million or 7% of the company’s total worldwide annual turnover (whichever is higher).
  • Context: This 7% ceiling explicitly exceeds the GDPR’s maximum cap, specifically designed to ensure tech giants cannot absorb fines as a mere cost of doing business.
• Tier 2: High-Risk AI Obligations & Transparency Failures
  • Applies to: Violating compliance mandates for high-risk systems (such as corporate credit-scoring algorithms, recruitment filters, or biometric tools) or failing transparency requirements.
  • Maximum Penalty: Up to €15 million or 3% of global annual turnover (whichever is higher).
• Tier 3: Procedural Integrity & Misleading Authorities
  • Applies to: Supplying incorrect, incomplete, or misleading information to national competent authorities or notified bodies during audits.
  • Maximum Penalty: Up to €7.5 million or 1% of global annual turnover (whichever is higher).

2. The GDPR Profiling & Data Protection Penalties

Under GDPR (Article 83), fines are similarly split into two tiers depending on which core rights or principles were violated. Unlawful automated profiling, lack of a lawful basis for tracking consumer behavior, or ignoring data subject rights fall squarely into the higher tier.

• Upper Tier: Serious Infringements (e.g., Unlawful Profiling & Consent Violations)
  • Applies to: Breaching fundamental processing principles, running automated profiling without a valid legal basis, violating user rights (Articles 12–22), or executing illegal international data transfers.
  • Maximum Penalty: Up to €20 million or 4% of total worldwide annual turnover (whichever is higher).
  • Real-World Precedent: This tier has driven massive historical enforcement actions against major tech firms, such as Ireland’s DPC issuing a €310 million fine to LinkedIn for unlawful behavioral profiling and targeted advertising.
• Lower Tier: Administrative & Governance Breaches
  • Applies to: Technical security failures, failure to maintain adequate records of processing activities, or failing to notify a supervisory authority of a data breach within 72 hours.
  • Maximum Penalty: Up to €10 million or 2% of total worldwide annual turnover (whichever is higher).

The Cumulative Regulatory Pressure

When corporations deploy automated scoring and profiling systems that cross legal boundaries, they do not just face single-file fines. Regulators routinely issue concurrent penalties—hitting a company simultaneously under GDPR for unlawful personal data profiling and under the AI Act for non-compliant, high-risk algorithmic architecture. For a global enterprise, a systemic failure in its scoring models can trigger combined turnover-pegged penalties reaching up to 11% of global annual revenue.

---

Auditing the Blackbox

Auditing complex neural networks and proprietary scoring algorithms—the ultimate “black boxes”—presents a massive technical hurdle. If a model consists of hundreds of billions of opaque weights distributed across a high-dimensional vector space, a human regulator cannot simply read the source code to find a violation.

To bypass this, European regulators (operating through the European AI Office, national market surveillance authorities, and independent Notified Bodies) do not just try to reverse-engineer the math line-by-line. Instead, they approach the audit through a combination of structural compliance mandates, systematic behavioural probing, and cryptographic logging.

The auditing toolchain relies on three core operational strategies:

1. White-Box Documentation & Data Governance (Before Deployment)

Under the EU AI Act and GDPR, the easiest way to prove a violation is to inspect what the corporation built before the model ever touched live data. Regulators bypass the black box by forcing companies to open their engineering notebooks.

• Data Provenance and Training Audits: Under Article 10 of the AI Act, providers of high-risk scoring systems must prove the datasets used to train the model were representative, clean, and free of systemic bias. Auditors inspect the version-controlled records of data-cleaning steps, gap analyses, and labeling protocols. If a credit-scoring algorithm discriminates against a protected class, regulators trace it straight back to biased training data.
• The Technical Documentation Dossier (Annex IV): Companies must legally submit comprehensive documentation detailing the system’s architecture, optimization objectives, hardware requirements, and the logic behind its classification thresholds. If the deployed model behaves differently than what was documented, the provider faces immediate fines for deceptive compliance.

2. Behavioral Probing and Counterfactual Testing (Black-Box Audits)

When regulators or independent researchers test an active system (such as a corporate hiring filter or insurance scoring engine), they rely on behavioral and counterfactual probing.

Instead of looking inside the model, they systematically manipulate inputs and observe the statistical outputs to map the hidden decision boundaries:

• The Sock-Puppet Audit: Auditors generate hundreds of synthetic profiles with identical financial or professional credentials—varying only a protected characteristic (such as gender, ethnic indicator, or zip code). If the automated scoring model systematically assigns lower trust scores or loan approvals to one demographic, the statistical disparity serves as empirical proof of algorithmic discrimination.
• Adversarial Stress-Testing: Regulators inject edge-case inputs designed to force the model to fail or reveal hidden biases, testing whether the system maintains robustness against manipulation or generates unauthorized profiling metrics.

3. Automated Event Logging and Traceability (The Flight Data Recorder)

Neural networks may be non-deterministic and opaque, but the infrastructure running them is completely deterministic.

Article 12 of the EU AI Act mandates that high-risk AI systems must feature automatic logging capabilities throughout their entire operational lifecycle.

• System Telemetry: The hosting servers must maintain secure, immutable logs recording every significant event: inputs received, confidence scores generated, system errors, and triggers for human oversight intervention.
• Reconstructing the Decision Path: If a citizen files a grievance claiming an illegal automated profiling decision, regulators do not need to understand every neural weight. They pull the system logs for that specific transaction to reconstruct the exact data state, feature weights, and threshold triggers that produced the negative score.

The Ultimate Regulatory Checkmate

The genius of these frameworks is that they convert an impossible computer science problem (explaining a trillion-parameter neural network) into a strict legal liability framework.

A corporation can no longer hide behind the defense that “the AI is a black box and we don’t know why it made that decision.” Under EU regulation, if a proprietary scoring algorithm produces an unlawful discriminatory outcome or violates profiling restrictions, the opacity of the model is not an excuse—it is a regulatory violation in itself.

---

Open Source

The European Union’s AI Act approaches open-source foundational models (such as Meta’s Llama or Mistral AI’s open-weight releases) with a specific, highly nuanced distinction: it grants partial exemptions to the model provider based on licensing, but it offers zero immunity to the deployer who puts that model to work.

The regulatory treatment of these models maps directly onto your classification framework. The EU realized that penalizing open-weight models with heavy bureaucratic friction would kill the open-source ecosystem, yet it could not allow powerful models to escape accountability entirely.

1. The Four-Pronged Open-Source Exemption Test

Under Article 53(2) of the AI Act, providers of General-Purpose AI (GPAI) models are granted a partial exemption from certain administrative burdens, but only if their models satisfy four strict conditions:

1. True Open-Source License: Released under a free and open-source license (e.g., Apache 2.0, MIT) that permits access, usage, modification, and redistribution. (Licenses with “research-only” or restrictive commercial clauses do not qualify).
2. Publicly Available Weights: The actual model parameters (weights) must be public—”open architecture, closed weights” models do not qualify.
3. Architectural Transparency: Information concerning the model’s architecture must be publicly released.
4. Usage Documentation: Clear documentation on model usage must be accessible.

If a model like Mistral or an open-weight Llama variant meets these criteria, the provider is exempt from two major administrative duties:

• They do not have to generate and maintain exhaustive internal technical documentation specifically formatted for EU regulatory bodies (Annex XI).
• They do not have to supply proprietary downstream documentation packages to every enterprise integrator who builds on top of their model (Annex XII).

2. The Non-Negotiable Baseline (What Open-Source Cannot Escape)

Even if a model is fully open-source and satisfies all four conditions, two obligations can never be waived:

1. Copyright Compliance: The provider must maintain an explicit policy ensuring compliance with EU copyright law, specifically respecting machine-readable rights reservations (like web-crawler blocks and robots.txt protocols) used during training.
2. Training Data Summaries: The provider must publish a sufficiently detailed, standardized summary of the content used to train the model.

3. The “Systemic Risk” Ceiling (The Compute Threshold)

The open-source exemption operates as a sliding scale that instantly snaps shut if a model crosses a critical capability threshold.

The AI Act establishes that any GPAI model trained using a cumulative compute power greater than $10^{25}$ FLOPs (Floating-Point Operations) is automatically classified as a Model with Systemic Risk.

• When frontier open-source models scale up to or past this threshold, the open-source exemption vanishes entirely.
• They are subjected to the full suite of systemic risk obligations: mandatory adversarial red-teaming, rigorous tracking and reporting of serious incidents, cybersecurity evaluations, and structural energy-consumption reporting.

4. The Deployer Trap: Where the Exemption Stops

The most crucial rule of the EU AI Act regarding open-source models is this: The license on the model governs the developer, but the use case governs the deployer.

If a bank, a hospital, or an enterprise downloads an open-weight Llama model from Hugging Face under a completely free Apache 2.0 license and integrates it into a High-Risk AI System (such as credit scoring, biometric categorization, or recruitment filters), the open-source nature of the underlying model provides zero legal protection.

The enterprise deploying the model inherits the full weight of the High-Risk obligations:

• They must establish rigorous risk-management systems.
• They must guarantee data governance and mitigate algorithmic bias.
• They must ensure immutable event logging.
• They must bake in active human oversight.

The Verdict on Open-Source Regulation

The EU AI Act treats open-source foundational models as raw infrastructure—similar to how a traditional legal system treats a public highway or a block of steel. The person who mines the steel (the model creator) gets a break on documentation, but the person who builds a vehicle out of it and drives it on public roads (the deployer) is held strictly accountable for its safety.

---

Model Tuning

If an enterprise fine-tunes an open-source model like Llama for internal use, does that enterprise legally become the ‘provider’ of a new AI system under the EU AI Act?

Under the EU AI Act, the short answer is no, not usually—standard internal fine-tuning does not automatically make an enterprise the “provider” of a General-Purpose AI (GPAI) model. For most routine customisations, the enterprise remains legally classified as a deployer.

However, the law establishes a precise, mathematical boundary where minor customisation ends and “substantial modification” begins.

The regulatory test governing whether an enterprise fine-tuning an open-source model like Llama inherits provider obligations relies on the following criteria:

1. The Standard Rule: Fine-Tuning is Not “Developing”

The European Commission’s guidelines clarify that adapting, prompting, quantising, or performing standard parameter-efficient fine-tuning (like LoRA or standard instruction-tuning) on an existing open-source model does not make you the model provider.

If your internal fine-tuning falls within the scope of what the upstream creator (e.g., Meta) originally anticipated or permitted in their technical documentation, you are treated as a downstream deployer. Your legal duties are limited to using the model responsibly, ensuring human oversight if deployed in a high-risk context, and respecting transparency rules—you do not have to recreate upstream GPAI technical documentation or training data summaries.

2. The Exception: The “One-Third” Compute Rule (Substantial Modification)

The line between a deployer modifying a model and becoming a new provider is measured by computational weight—specifically, floating-point operations (FLOPs):

• If your fine-tuning process consumes cumulative compute resources that exceed one-third (33%) of the original model’s base pre-training compute, the EU AI Act presumes you have substantially altered the model.
• For a massive model like Llama, 33% of its original pre-training compute is an astronomical amount of energy. Routine, targeted enterprise fine-tuning on internal clusters rarely comes close to touching this threshold.

If an enterprise does cross that one-third compute threshold through heavy, foundational retraining, it legally crosses the boundary and becomes a GPAI provider for that newly modified version.

3. The “Internal Use” Catch (The Deployment Context)

Even if your fine-tuning stays well below the compute threshold and you avoid becoming a GPAI model provider, the intended use of that fine-tuned model still dictates your legal reality.

If an enterprise fine-tunes Llama entirely for internal use (e.g., an internal document search or code assistant), it avoids many external-facing burdens. However, if that same fine-tuned internal model is integrated into a High-Risk AI System (such as an automated recruitment filter screening incoming job resumes, or an internal credit-scoring tool for applicants), the enterprise instantly inherits all strict High-Risk deployer obligations under the Act—regardless of whether it fine-tuned the model or downloaded it straight off Hugging Face.

When an enterprise takes a fine-tuned, open-source model (like a custom version of Llama) and deploys it in a high-risk context (such as automated recruitment, credit scoring, biometric identification, or essential public services), it triggers Article 26 of the EU AI Act.

Because the enterprise is acting as a deployer rather than the original foundation model provider, its duties shift from building the architecture to governing its operational safety, oversight, and traceability.

The specific documentation and risk-management duties mandated by the EU framework include:

1. Fundamental Rights Impact Assessment (FRIA)

Before putting the fine-tuned model live in a high-risk scenario, certain deployers (including private entities providing public services or operating in sensitive sectors like credit scoring and insurance pricing) must conduct and document a Fundamental Rights Impact Assessment (FRIA).

• What it requires: A formal evaluation mapping out how the AI model will impact the fundamental rights (e.g., non-discrimination, privacy, worker dignity) of the individuals it interacts with.
• Administrative duty: This assessment must be registered in the EU database before the system is put into service.

2. Mandatory Human Oversight (The “Kill Switch” Mandate)

Deployers cannot let a high-risk fine-tuned model operate autonomously without human-in-the-loop safeguards.

• What it requires: The enterprise must assign natural persons to oversee the system who possess the necessary competence, training, authority, and support.
• Operational duty: Overseers must be positioned to fully understand the model outputs, disregard or override automated decisions when necessary, and have the technical ability to halt or pause the system instantly if an anomaly or systemic bias appears.

3. Rigorous Input Data Governance

Even if the upstream open-source model was trained on general internet data, the enterprise controls the local input data fed into the fine-tuned model during deployment.

• What it requires: To the extent the enterprise exercises control over the input data, it must ensure that data is relevant, representative, and cleansed of historical biases that could trigger discriminatory automated profiling.

4. Automated Event Logging (The Flight Recorder)

Neural networks are non-deterministic, but the infrastructure running them must be fully auditable.

• What it requires: Under Article 26(6), deployers must ensure that the logs automatically generated by the high-risk AI system are retained for a minimum of six months (unless superseded by sector-specific financial or data protection laws like GDPR).
• Auditing duty: These logs must be kept under the enterprise’s control so that regulators or auditors can reconstruct the exact feature weights, inputs, and thresholds that triggered a specific decision.

5. Post-Market Monitoring and Incident Reporting

Deployers cannot simply launch a fine-tuned model and walk away.

• What it requires: The enterprise must continuously monitor the operation of the system based on the provider’s instructions for use.
• The Escalation Trigger: If the enterprise identifies a serious incident (e.g., a catastrophic bias cascade, systematic discrimination, or a failure threatening fundamental rights), it must immediately suspend use of the system and notify both the original provider and the relevant national market surveillance authority.

6. Transparency and Worker Notification

If the fine-tuned model is deployed internally for workforce management or recruitment:

• What it requires: The enterprise must explicitly inform workers’ representatives and affected employees before they are subjected to the high-risk AI system, adhering strictly to labor consultation rules.

AI & GDPR compliance

The intersection between the EU AI Act’s Fundamental Rights Impact Assessment (FRIA) (Article 27) and the GDPR’s Data Protection Impact Assessment (DPIA) (Article 35) represents one of the most critical compliance overlaps for enterprises deploying AI systems.

Because almost every high-risk AI application (like automated recruitment, credit scoring, or worker monitoring) processes personal data, enterprises routinely find themselves triggering both assessments simultaneously.

Rather than treating them as isolated silos, the European Union designed the frameworks to interact through structural bridges, overlapping scopes, and explicit legal linkages.

1. Scope and Focus: Data Privacy vs. Total Human Rights

To understand how they intersect, you first have to look at what each assessment is built to evaluate:

• The GDPR DPIA (Article 35): Narrower, deeper, and strictly focused on information privacy and data protection rights. It asks: How does processing personal data impact an individual’s privacy, data security, and informational self-determination? It evaluates necessity, proportionality, data minimization, and technical safeguards.
• The AI Act FRIA (Article 27): Broader and focused on holistic fundamental rights. It asks: How does the deployment of this automated system impact human dignity, non-discrimination, worker rights, freedom of expression, and access to essential services? Data privacy is just one small slice of a FRIA.

2. The Direct Legal Bridge (Article 27(4))

The EU explicitly anticipated the administrative nightmare of forcing companies to run two entirely separate bureaucratic processes for the same software.

Under Article 27(4) of the AI Act, the law provides a legal reuse mechanism:

• If an enterprise has already conducted a DPIA under Article 35 of the GDPR, it can re-use and integrate those findings directly into its FRIA.
• Because a DPIA already maps out data flows, system logic, and data-privacy risks, it serves as the foundational data-architecture chapter of the broader Fundamental Rights Impact Assessment.

3. Key Differences in Operational Requirements

While they can be merged or cross-referenced, a standard GDPR DPIA cannot simply be copy-pasted to satisfy a FRIA. A FRIA demands metrics that a data protection assessment completely ignores:

Requirement	GDPR DPIA (Article 35)	EU AI Act FRIA (Article 27)
Primary Trigger	High risk to data privacy/freedoms via personal data processing.	Deployment of a High-Risk AI system (regardless of underlying data nuances).
Core Focus	Lawfulness, minimization, security, and storage limits of personal data.	Societal harm, systemic bias, socio-economic exclusion, and discrimination.
Human Oversight	Evaluates if staff are trained on privacy rules.	Maps out operational protocols for the human “kill-switch”—ensuring overseers can actually override or halt the AI.
Stakeholder Mapping	Maps data subjects.	Explicitly maps vulnerable groups or communities likely to be impacted by the system’s decisions.
Time/Frequency Scope	Focuses on data life-cycle and retention periods.	Focuses on the duration, frequency, and operational context of the system’s active deployment.

4. The Unified Compliance Workflow in Practice

In an enterprise environment, compliance officers do not treat these as two separate tasks. Instead, they run an Integrated Impact Assessment structured around the following timeline:

1. Phase 1 (The GDPR Base): The data protection officer (DPO) maps out the data flows, ingestion pipelines, storage limits, and privacy risks, completing the DPIA.
2. Phase 2 (The AI Act Extension): The compliance team takes that data-flow foundation and expands it into a FRIA. They evaluate how the model’s outputs affect human dignity, test for demographic bias (counterfactual probing), outline the exact procedures for human oversight, and identify vulnerable groups who might be harmed by a false classification.
3. Phase 3 (Filing and Registration): For high-risk systems, the completed impact assessment package (incorporating both the DPIA data metrics and the FRIA fundamental rights analysis) is logged into the EU database or made available to national market surveillance authorities upon request.

By merging the two, enterprises avoid redundant paperwork while satisfying the dual demand of protecting both data privacy (GDPR) and systemic human rights (AI Act).