SECTOR INTELLIGENCE BRIEFING, BASED ON DOCUMENTED PUBLIC EVIDENCE, RESEARCH DOCUMENT
01The Documented Picture
◈ This briefing draws exclusively on documented, publicly available evidence from UK and international regulators, parliamentary sources, and authoritative industry publications. The predictive findings in Section 04 are inferences from this evidence base, not assertions of findings BreakPoint has made in this sector.
◈ The pharmacovigilance asymmetry
Pharmacovigilance AI processes adverse event reports and surfaces those it classifies as requiring human attention. The human reviewer then assesses what the AI has surfaced. What the AI does not surface does not reach human review. No named individual holds authority over the AI's detection threshold, the parameter that determines what the AI decides is significant enough to show a human. The MHRA's guidance on AI in pharmacovigilance acknowledges this asymmetry. It does not specify who must hold named authority over threshold calibration, or what evidence of that authority's exercise is required. Until it does, the governance gap sits between the AI's detection boundary and any human reviewer.
2024
MHRA published AI Airlock Framework for medical device and drug AI regulation
MHRA, 2024
ICH E6(R3)
Revised GCP guidelines, now address AI/ML in clinical trials for first time
ICH, effective 2025
EU AI Act
High-risk AI classification applies to medical diagnostic AI, UK divergence now an active policy question
EU AI Act, effective Aug 2024
0
published independent audits of AI authority structures in UK pharma pharmacovigilance
Research gap: documented absence
AI is embedded across UK pharmaceutical and life sciences operations in four principal areas: pharmacovigilance and adverse event detection; clinical trial design, patient selection, and statistical analysis; drug discovery and target identification; and regulatory submission preparation and data analysis. Each area involves AI-shaped outputs that inform consequential human decisions, and in each, the governance framework assigns human authority at process level without specifying what evidenced exercise of that authority looks like at the point of individual AI-shaped determination.
The MHRA's 2024 AI Airlock Framework and the revised ICH E6(R3) GCP guidelines represent the first formal regulatory moves toward AI-specific governance in this sector. Both acknowledge that existing frameworks were not designed with AI in mind. Both fall short of specifying the human authority structure at decision level that would constitute a BreakPoint REAL determination.
02The Governance Gap, What the Evidence Shows
The MHRA's AI Airlock Framework, published 2024, introduces a staged regulatory pathway for AI in medical devices and drug development. The Framework explicitly acknowledges that the existing Medical Devices Regulations and drug licensing regime were not designed with AI in mind, and that AI systems that learn and adapt post-deployment present novel governance challenges. The Framework's approach is cautious and iterative. It does not resolve the central authority question: when AI is processing pharmacovigilance data or shaping clinical trial analysis, who holds named authority over the AI's analytical assumptions and detection parameters, and what evidence of that authority's exercise must be maintained?
The International Council for Harmonisation's revised Good Clinical Practice guideline E6(R3), effective 2025 and adopted by the MHRA, addresses the use of AI and automated systems in clinical trials for the first time. It requires risk-based oversight of computer systems, including AI: and places responsibility on the sponsor to ensure that automated processes are validated and that data integrity is maintained. What it does not specify is the human authority structure at the level of individual AI-shaped decisions: who holds named authority over the AI's patient eligibility determinations, endpoint analysis assumptions, or statistical model parameters. The guideline sets the obligation; it does not specify what its exercise looks like.
The EU AI Act classifies AI systems used in medical diagnosis, clinical decision support, and drug safety monitoring as high-risk, requiring conformity assessments, human oversight mechanisms, and post-market monitoring. UK pharma operating in both markets must simultaneously satisfy EU requirements, including specified human oversight provisions, and operate under UK frameworks that have not yet reached equivalent specificity. This regulatory divergence creates an asymmetry where the same AI system may have a specified human authority structure in its EU deployment context and an unspecified one in its UK context. BreakPoint provides a framework-neutral assessment that applies in both regulatory environments.
The Association of the British Pharmaceutical Industry and multiple published industry papers document that AI is now routinely used in preparing regulatory submissions, data extraction, literature review automation, signal detection, and statistical analysis. The issue documented in industry publications is that use of AI in submission preparation is not consistently disclosed to regulators, meaning that MHRA assessors evaluating the evidence base for a drug approval are, in a proportion of cases, assessing AI-shaped analysis without knowing it. This is not a deliberate concealment; it reflects the absence of a disclosure standard. The MHRA is actively developing one. Until it is published, the undisclosed use of AI in regulatory evidence represents an accountability gap in the submission process itself.
The Nuffield Council on Bioethics' 2023 report on AI in healthcare documents the pattern in drug discovery: AI systems identify candidate drug targets and predict their therapeutic potential; human scientists then make investment decisions about which candidates to pursue. The governance concern the report identifies is that the human decision is made after the AI has already shaped the option set, which targets are visible, which are ranked highly, which appear viable. Independent human assessment of the target landscape before AI input is not a documented feature of any published drug discovery workflow. The human authority operates on AI-curated options, not from first principles.
03The Regulatory Framework
MHRA AI Airlock Framework (2024)
Staged regulatory pathway for AI in medical devices and drug development. Acknowledges existing frameworks were not designed for AI. Human oversight required, mechanism not specified at decision level.
Diagnostic AI in clinical devices. AI in drug safety monitoring. AI used in clinical trial conduct or regulatory submission. Any AI whose outputs inform a regulated medical or safety decision.
ICH E6(R3) GCP (eff. 2025)
Risk-based oversight of computer systems including AI in clinical trials. Sponsor responsible for validation and data integrity. Human authority structure at individual AI decision level not specified.
Patient eligibility AI. Endpoint analysis AI. Statistical model AI. Safety signal detection AI in trials. Any automated system that generates, processes, or analyses clinical trial data.
EU AI Act (Aug 2024)
High-risk classification for medical AI. Conformity assessment required. Human oversight mechanism must be specified and operational. Post-market monitoring mandatory. UK divergence creates asymmetry.
Diagnostic AI. Drug safety monitoring AI. Clinical decision support AI. Applies to UK pharma with EU market operations, creating dual-framework exposure.
MHRA Pharmacovigilance Requirements
Adverse event detection and reporting obligations. AI in pharmacovigilance subject to validation. Detection threshold governance, who holds authority over what AI surfaces, not specified at decision level.
Adverse event signal detection AI. Patient safety database AI. Benefit-risk assessment AI. Literature monitoring AI. Any AI that determines which adverse signals reach human review.
04What BreakPoint Would Predict
Epistemic note:The following are indicative and predictive inferences from the documented evidence base above - not assertions of findings BreakPoint has made in this sector. Each prediction identifies a specific gap type that the evidence strongly suggests is present. BreakPoint's role is to determine, for a specific organisation, whether the gap exists and its severity. Note that BreakPoint is not limited to these factors however, and will often surface unexpected risks or risks that were not fully articulated.
Pharmacovigilance teams review adverse event signals that AI surfaces. The AI's detection threshold determines what reaches human review, and therefore what does not. In every pharmacovigilance AI deployment, the threshold is set at implementation and calibrated over time by the AI system itself, with periodic review by the pharmacovigilance or data science team. What is absent, in every published governance framework, is a named individual who holds authority over the detection threshold as a regulatory obligation, not just as a technical parameter. The MHRA's pharmacovigilance requirements place the obligation on the company. BreakPoint asks who, specifically, is named as accountable for the threshold that determines what a human reviewer sees.
Evidence basis: MHRA pharmacovigilance requirements; MHRA AI Airlock Framework 2024 (human oversight required, mechanism unspecified); EU AI Act (high-risk classification for drug safety AI); pattern documented in Nuffield 2023 (AI shapes what humans see before human review).
Statistical AI tools used in clinical trial design and analysis embed assumptions about endpoint definitions, patient population characteristics, and analytical methodology. These assumptions are documented and reviewed by the trial's statistical team and Qualified Persons. What they are not subject to, because ICH E6(R3) does not require it and existing practice does not deliver it, is independent challenge at the assumption level by a function that is not invested in the trial's outcome. The PRA's model risk standard in SS1/23 provides the analogy: reviewing the outputs of a model without independently challenging its embedded assumptions is not sufficient governance. The same logic applies to clinical trial statistical AI.
Evidence basis: ICH E6(R3) (risk-based oversight, sponsor responsibility, human authority structure unspecified); MHRA AI Airlock Framework (validation required, independence standard not specified); EU AI Act (high-risk AI in clinical settings); SS1/23 principle, independent challenge vs. peer review.
AI target identification tools generate ranked candidate lists that inform which targets receive research investment. Scientists and R&D committees then make investment decisions, but those decisions are made from the AI's ranked list, not from an independent prior assessment. The Nuffield Council documented this pattern explicitly: human authority operates on AI-curated options. The AI's ranking methodology determines what the human decision-maker sees. Independent formation of a view before AI consultation, which would constitute genuine independent assessment, is not documented as standard practice in any published drug discovery workflow.
Evidence basis: Nuffield Council on Bioethics 2023 (AI in drug discovery, human decision follows AI option curation); MHRA AI Airlock Framework (applies to AI in drug development pipeline); documented sector pattern of AI-first discovery workflow architecture.
AI is used in preparing regulatory submissions, literature review automation, data extraction, signal analysis, document drafting. The governance sign-off for these tools covers their approved scope. The ABPI and published industry papers document that actual use has expanded as the tools have become more capable. The condition is not that AI is being used improperly; it is that the scope of AI use in submission preparation exceeds what was disclosed or governed at approval, and that the MHRA assessor reviewing the submission does not have visibility of this. The MHRA is developing a disclosure standard. Until it exists, the scope gap is structurally ungoverned.
Evidence basis: ABPI industry publications on AI in regulatory affairs; MHRA active development of AI disclosure standard for submissions; EU AI Act (transparency requirements for AI in regulated submissions, UK divergence); pattern consistent with scope creep Condition across all sectors.
AI-powered prescribing support tools, widely deployed across NHS and private clinical settings, apply drug interaction flags, contraindication alerts, and dosage recommendations based on thresholds and logic embedded by the vendor. The clinical governance structure in deploying organisations assigns responsibility for prescribing decisions to named clinicians. What it does not typically assign is named authority over the vendor-set thresholds that determine what the AI alerts and what it does not. When a prescribing AI fails to flag an interaction because the vendor's threshold classifies it as low-severity, no named clinician in the deploying organisation is accountable for that threshold decision, because they did not make it.
Evidence basis: MHRA medical devices regulation (applies to prescribing support AI as medical device); MHRA AI Airlock (applies to AI with clinical decision support function); EU AI Act high-risk classification (clinical decision support AI); Nuffield 2023 (clinician authority operates within AI-set parameters).
05Source References
MHRA AI Airlock
MHRA,
AI Airlock: A Sandbox for AI in Healthcare Products, 2024. Staged regulatory pathway for AI in medical devices and drug development.
gov.uk →ICH E6(R3)
International Council for Harmonisation,
Guideline for Good Clinical Practice E6(R3), effective 2025. First GCP revision to address AI/ML in clinical trials. Adopted by MHRA.
ich.org →EU AI Act
European Parliament and Council,
Regulation (EU) 2024/1689, Artificial Intelligence Act, effective August 2024. High-risk AI classification for medical and diagnostic AI. Human oversight requirements specified.
ec.europa.eu →Nuffield 2023
Nuffield Council on Bioethics,
The Ethics of Artificial Intelligence in Health and Social Care, 2023. AI in drug discovery, target selection shapes human investment decisions.
nuffieldbioethics.org →ABPI: AI in Reg
Association of the British Pharmaceutical Industry, publications on AI in regulatory affairs and drug development, 2023 to 24. Documents undisclosed AI use in submission preparation and absence of disclosure standard.
MHRA: PV
MHRA, Guidance on Pharmacovigilance (current). Adverse event detection and reporting obligations. AI in pharmacovigilance subject to validation requirements. Detection threshold governance unspecified at decision level.
Luminary Diagnostics · BreakPoint: AI Control Gap Assessment
Pharmacovigilance AI surfaces the signals it detects.
No one is named as responsible for the signals it doesn't.
BreakPoint identifies where human authority exists at decision level, and where it exists only in the governance document. In pharma and life sciences, the gap between the two sits where patient safety risk is highest.
Book a BreakPoint conversation →