Phone Icon1-919-645-1070
Phone Icon1-919-645-1070
SCHEDULE DEMO

Annex 22 Excludes LLMs from Critical GMP Applications. Where Does That Leave Your Agentic AI?

Home Blogs Annex 22 Excludes LLMs from Critical GMP Applications. Where Does That Leave Your Agentic AI?

Annex 22 Excludes LLMs from Critical GMP Applications. Where Does That Leave Your Agentic AI?

Posted on by Rick Soltero
July 29, 2026

Recently, we have been giving demonstrations of our agentic AI MCP server that queries InstantGMP data exclusively. While the results and reports can be impressive, a lingering question remains. Is this allowed by the FDA or other regulatory bodies?

For most of the last two years I could give a comfortable answer, because the regulations had not caught up and nobody could tell you what the boundaries were. That comfortable period is ending. Europe has drafted the first GMP text dedicated to artificial intelligence, and the draft is more restrictive than most of the AI marketing in our industry would lead you to expect. If you have been assuming that a language model reading your batch records is obviously fine, it is worth reading what the regulators actually wrote.

What Annex 22 Actually Says

In July 2025 the European Commission published a draft Annex 22 on Artificial Intelligence, alongside a revised Annex 11 on computerised systems and an updated Chapter 4 on documentation. Public consultation closed that October. EMA convened a multistakeholder workshop at the end of June 2026 to gather expert input on guardrails and mitigation measures, and a final text is expected before the end of the year. As I write this, Annex 22 is still a consultation draft, which means it is a strong signal of regulatory direction rather than an enforceable requirement. It is not a document you can safely ignore on that basis.

The draft applies where AI models are used in critical applications, meaning applications with direct impact on patient safety, product quality, or data integrity. Within that scope, it permits only static, deterministic models: models whose parameters are locked after training and which return the same output when given the same input.

Everything else is pushed outside. Dynamic models that continue to learn during use are excluded from critical applications. Probabilistic models, where the same input can yield a different answer, are excluded. And generative AI and large language models are named explicitly and excluded as well. What the draft does permit is the use of those excluded model types in non-critical applications, provided a qualified person is responsible for evaluating whether the output is suitable for its intended use.

That is a real line, drawn in a real place, and it is not the line most vendors have been describing. What we describe is that our AI provides real time intelligence about the data that exists in a customer’s instance. For example, AI-Generated charts and summaries or connections across multiple sources in the data.

Why the Line Sits Where It Does

The reasoning is not technophobia. It is change control, and once you see it that way the restriction stops feeling arbitrary.

Everything we do in GMP rests on the validated state. You establish that a system does what it is supposed to do, you document it, you lock it, and you manage every subsequent change through a controlled process. That is the whole architecture. A model that keeps adjusting its own behavior after deployment is, by definition, changing without going through change control. It is uncontrolled change wearing a lab coat.

I lived through this argument once already with electronic signatures. The industry spent years insisting that a signature had to be a signature, and the regulations eventually caught up because the underlying control objective could be met a different way. I expect something similar will happen here. But the burden is on us to demonstrate the control objective is met, and right now most AI deployments in our industry cannot demonstrate much of anything.

There is a legitimate criticism of the draft, and I want to give it a fair hearing rather than pretend the industry is united behind it. By keeping adaptive and generative models out of critical applications entirely, Annex 22 also keeps real-time process optimization and adaptive learning out of reach, and those capabilities are proven in other manufacturing sectors. The caution has a cost. Whether that cost is worth paying is a genuine disagreement, not a settled question, and reasonable people in this industry come down on both sides of it.

The Part Nobody Has Settled

Here is where it gets uncomfortable for anyone selling AI-enabled GMP software, including us.

The scope clause assumes you can sort systems cleanly into critical and non-critical. In practice, the most useful AI architectures are hybrids. A system that retrieves structured records deterministically from a validated database and then uses a language model to summarize them is neither purely deterministic nor purely generative. The retrieval is reproducible. The prose around it is not.

This is precisely the boundary case that industry commenters raised during consultation. The general position in the feedback was that the restriction makes sense for functions that directly impact quality, but that the definitional boundary needs clarification, particularly for systems that combine deterministic and learned components. That clarification has not arrived. It may arrive in the final text. It may not.

So when a vendor tells you their agentic AI is definitively outside the scope of Annex 22, they are telling you more than the current draft supports. I would treat that claim the same way I treat any other confident statement about a rule that has not been published yet.

Where I Think a Query Layer Sits

Let me make the argument for our own architecture and then tell you where it stops.

In the InstantGMP MCP Server, the AI does not create records and it does not make release decisions. The deviation records, the CAPA statuses, the batch approval steps, the inventory receipts, all of it was entered and approved through validated workflows with electronic signatures and audit trails. The AI reaches that data through a validated API layer and presents it in a synthesized view. It is a lens over a controlled system, not a data entry point into one. Human review remains the control, and anything your team decides to do in response goes through the same documented approval chains it always did.

On that description, the system reads as non-critical, and I think that reading is correct today.

But the description is about architecture, and the draft is about consequence. The moment a synthesized view becomes the basis for a quality decision, you have moved closer to the critical side of the line regardless of what the diagram shows. If a reviewer approves a batch because the AI summary said the deviations were unrelated, the AI just participated in a quality decision. The architecture did not change. The use did.

That is the distinction I would ask every quality team to sit with, because it is the one that determines your exposure, and it is not a question your software vendor can answer for you.

What to Do Before the Final Text Lands

Waiting for the published Annex is not a strategy, and neither is deploying as though it will not apply. Five things are worth doing now.

Classify by consequence, not by architecture

Write down, for each AI use in your operation, what happens if the output is wrong. An AI that summarizes training completion for a supervisor sits somewhere very different from one whose output informs batch disposition. This is the same risk-based logic behind FDA’s January 2025 draft guidance and its credibility assessment framework, and behind the ISPE GAMP Guide: Artificial Intelligence. Your validation effort should scale to the consequence.

Write the intended-use statement narrowly

Vague intended-use language invites findings. The tighter your written statement of what the AI is and is not authorized to do, the stronger your position, and the easier it is to argue a non-critical classification honestly.

Ground the model in your own records

A language model constrained to retrieve and cite records from your validated system behaves very differently from one generating text freely. This is not a marketing distinction. It is the distinction FDA’s own Chief AI Officer reached for when defending the agency’s internal AI tool against reports that it had fabricated citations, and it is the difference between an output you can defend and one you cannot.

Keep the trace

If the system tells you a vendor carries quality risk, you need to reach the specific deviation records and inventory receipts that produced that conclusion. Annex 22 puts real weight on explainability, and black box summaries are not audit ready. Traceability is also what lets a reviewer catch the AI when it is wrong, which is the only thing standing between human-in-the-loop as a control and human-in-the-loop as a formality.

Validate it, and write down what triggers revalidation

We built a protocol for our AI features rather than asserting they are validated: reference standard comparison against data pulled directly from the application, acceptance criteria defined before testing, repeated runs to confirm factual consistency, negative test cases that confirm the system reports no data found rather than inventing something plausible, and pre-defined change control triggers for model version and server updates. The full method is on our AI Systems Validation page, built on the GAMP 5 validation foundation the platform already ships with.

The Accountability Nobody Can Outsource

One provision in the draft deserves more attention than it gets. When AI used in your GMP operation fails, accountability stays with the manufacturer. Vendor documentation supports your validation. It does not transfer the responsibility. That is true of InstantGMP as much as anyone else, and I would rather say it plainly than let a customer discover it during an inspection.

FDA made the same point from the enforcement side in April 2026, in its warning letter to Purolea Cosmetics Lab. The firm had used AI agents to create specifications, procedures, and master production records without adequate review, which the agency tied to 21 CFR 211.22(c) and the responsibilities of the quality control unit. The detail that stays with me is not the technology failure. When investigators asked about the compliance gaps, personnel said they had not known about certain requirements because the AI agent had not raised them. The tool answered every question it was asked. Nobody asked the right ones.

The Question Worth Asking

Before you deploy agentic AI against your quality data, ask this: if an inspector asked me to classify this system as critical or non-critical and justify the answer, could I do it in writing, today?

If you can, you are in good shape whichever way the final Annex lands, because you have done the thinking the regulation is going to ask for. If you cannot, the problem is not the technology and it is not the guidance. It is that nobody has decided what the system is actually for.

The technology works. I use it every day and it has changed how our quality operations run. The regulatory framework is taking shape and is not finished. The manufacturers who come out ahead will be the ones who deploy deliberately now and document why, rather than the ones waiting for permission or the ones assuming none is required.

InstantGMP is a global manufacturing platform for regulated industries, with electronic batch records, inventory, quality management, and document management in one validated system. The AI Insights features in InstantGMP PRO query your own validated records rather than a separate model. Schedule a demonstration.

Sources

share