The hype of artificial intelligence (AI) is revolutionizing industries such as pharmaceuticals and medical devices, and the regulatory landscape has moved fast to keep pace.
What was "upcoming" in the last article’s version (2023) is now, in mid-2026, largely in force: the EU's AI Act is law and in phased application, the EMA has finalised its guidance, the FDA has issued dedicated AI guidance and changed its policy direction, and a new generation of international standards is in place.
Regulatory agencies and organizations recognize the potential of AI to improve health outcomes. AI offers many possibilities by helping to improve processes, develop novel approaches, and transform data in ways that were not possible before.
Many organizations provide or use AI-based products or services, but are security, safety, trust, ethics, fairness, and transparency adequately addressed? Not completely, which is why a growing body of law and standards now sets clear expectations.
It is essential to address AI concerns to ensure ethical considerations, human rights, transparency, and accountability in AI development and deployment for health. Organizations must understand their responsibilities and take the necessary steps to ensure AI is used responsibly and safely, while governments create and enforce the rules that make this binding.
The responsible use of AI in pharma is increasingly being governed by hard regulation rather than guidance alone, even as regulators continue to work hard to keep up with rapidly evolving AI technologies. This article reviews the regulation now applying to pharma and medical devices to ensure patient safety and protect data privacy and security, updated to reflect developments through mid-2026.
Serious challenges of using AI
To ensure that manufacturers' AI models comply with existing legislation, AI guidance must address serious challenges and caveats. The rise of generative AI and large language models has sharpened several of these. They include:
Ethics in data collection
Using artificial intelligence to handle sensitive medical data raises privacy and security concerns. AI may also undermine individuals' autonomy and dehumanize patient care.
When automating personal data, manufacturers must comply with the General Data Protection Regulation (GDPR) in Europe and provide meaningful information about the logic involved in any automated decision taken by an AI algorithm.
The power granted to AI in the medical field could translate to life-threatening decisions, such as incorrect diagnosis, treatment, and intervention options, among many others.
The principle of autonomy requires that AI does not undermine human autonomy. Individuals must retain control over their medical decisions and actions in healthcare systems.
Cybersecurity threats
The integration of AI in pharma and medical devices raises concerns about the vulnerability of devices to cyberattacks. Privacy breaches can have serious financial and legal consequences.
Companies and healthcare providers must take steps to protect this data and ensure it is secure, with strong security measures, properly trained staff, and confidential handling of patient information. In the US, this is now backed by statute: under section 524B of the FDC Act, manufacturers of "cyber devices" must meet specific cybersecurity requirements (including a software bill of materials and vulnerability management), reflected in the FDA's final cybersecurity guidance updated in June 2025.
Systems must be regularly monitored and patched so that any vulnerabilities are addressed early.
Risk of bias
The quality, distribution, and integrity of input data determine the output of AI algorithms, and therefore how well a device performs.
It is essential to identify and address any biases present within the input data to ensure safe and effective AI applications
Potential for misuse
Using medical AI tools incorrectly can lead to wrong medical assessments and decision-making, which could harm patients.
Several factors can contribute to this misuse, including limited involvement of clinicians and patients in AI development, a lack of training in medical AI among healthcare professionals, and the proliferation of easily accessible online and mobile AI solutions without sufficient explanation and information.
Regulatory framework and standards
Advances in AI continue to outpace regulation, but the gap has narrowed sharply. To adapt existing regulations to AI innovations, regulatory authorities have developed clear guidelines and are collaborating with industry stakeholders, increasingly across borders, as shown by the joint EMA-FDA principles published in January 2026: Guiding Principles of Good AI Practice in Drug Development.
General initiatives in Artificial Intelligence
ISO/IEC 42001:2023
To ensure organizations stay ahead of the curve, the International Organization for Standardization (ISO) published ISO/IEC 42001:2023 Information technology — Artificial Intelligence — Management system (AIMS) in December 2023.
This standard provides guidelines for establishing, implementing, maintaining, and continually improving a structured Artificial Intelligence Management Systems (AIMS), using the Plan-Do-Check-Act methodology. As the world's first AI management-system standard, it addresses challenges such as ethics, transparency, and continuous learning, and sets out a structured way to manage the risks and opportunities of using, developing, monitoring, and providing AI-enabled products and services, balancing innovation with governance.
Since 2023 this has grown into a family of standards. ISO/IEC 23894:2023 provides AI-specific risk-management guidance, and in 2025 two important additions appeared: ISO/IEC 42005 (AI system impact assessment) and ISO/IEC 42006 (requirements for bodies auditing and certifying AI management systems), the latter paving the way for accredited ISO/IEC 42001 certification.
Accredited certification is now offered by major bodies (BSI, DNV, TÜV SÜD and others), and adoption is growing as companies use ISO 42001 to demonstrate AI governance and help evidence EU AI Act readiness.
WHO guidance on Artificial Intelligence for Health
In October 2023, the World Health Organisation (WHO) published a paper titled Regulatory Considerations on Artificial Intelligence for Health, outlining key regulatory considerations for AI for health, emphasizing the importance of establishing safety and effectiveness, making systems available, and fostering dialogue about using AI as a positive tool.
It set out 18 regulatory considerations across six broad categories:
- Documentation and transparency
- Risk management
- Intended use and validation
- Data quality
- Privacy and data protection
- Engagement and collaboration
These guide governments and regulators in developing new AI guidance, or adapting existing guidance, so that AI is effectively regulated to maximise its potential in healthcare while minimising its risks.
WHO has since gone further: in January 2024 it issued dedicated guidance on large multi-modal models (LMMs), with more than 40 recommendations addressing generative AI across diagnosis and clinical care, patient self-use, administrative tasks, education, and scientific research and drug development. This is now WHO's principal generative-AI guidance for health, building on its 2021 report setting out six ethics principles.
AI regulations in pharma and medical devices
Artificial Intelligence in pharma consists of algorithms, models, and techniques incorporated into computer systems to automate tasks and make decisions and predictions.
Data analytics is one of the key AI applications in regulatory compliance. Pharma and medical device companies generate enormous volumes of data that cannot be managed with traditional methods. With AI algorithms, large datasets can be analysed more efficiently, patterns identified, and meaningful insights drawn, providing a faster and more accurate way of scanning and analysing data than manual methods.
Regulators acknowledge that AI presents both challenges and opportunities, and have moved from consultation to concrete rules and guidance. Let's review the current status of AI regulation in the EU and the US.
Current status in the EU
The EU has implemented new AI rules appropriate to current and upcoming medical technology.
As part of the Big Data Workplan 2022-2025, the Big Data Steering Group of the European Medicines Agency (EMA) published a draft reflection paper on the use of AI in the medicinal product lifecycle in 2023. In September 2024 this reflection paper was finalised and adopted by EMA's human and veterinary committees (CHMP and CVMP). It is no longer a draft, and sets out a risk-based approach across the full lifecycle (drug discovery, non-clinical, clinical trials, manufacturing and pharmacovigilance), emphasising human oversight, data integrity and continuous performance monitoring.
The EMA and the Heads of Medicines Agencies are executing a multi-annual AI workplan 2023-2028 (now carried forward under the Network Data Steering Group), reflecting the EU's commitment to letting companies maximise the benefit of AI while ensuring its safety for the public. Concrete milestones have followed: EMA and HMA guiding principles for staff use of large language models (September 2024); an AI Observatory publishing annual reports (2025 and 2026); and, in March 2025, EMA's first qualification opinion accepting data generated with a (human-supervised) AI tool. The broader EU pharmaceutical legislation reform, finalised in March 2026, also introduces regulatory sandboxes into EU pharma law for the first time, explicitly envisaging AI and digital-health tools.
Overarching all of this is the EU AI Act, Europe's first AI law. Proposed in 2021, it entered into force on 1 August 2024 and is now in phased application. The AI Act sets the standard for AI regulation across the EU.
The Artificial Intelligence Act
The EU AI Act provides a uniform legal framework for the development, market availability, and use of AI in the EU. In line with the Medical Devices Regulation (MDR), it emphasises safety, while seeking to promote AI in healthcare and prevent barriers to its use.
It takes a risk-based approach, requiring companies to implement market surveillance and risk management, maintain data quality, keep records, ensure transparency, and employ cybersecurity, among other measures.
AI practices that pose unacceptable risk to people's safety, rights or autonomy are banned outright. The prohibited practices in Article 5 include:
- Subliminal, manipulative or deceptive techniques that materially distort behaviour and cause harm.
- Exploitation of vulnerabilities (due to age, disability or a specific social or economic situation).
- Social scoring leading to detrimental or unjustified treatment.
- Predictive policing based solely on profiling or personality traits.
- Untargeted scraping of facial images from the internet or CCTV to build facial-recognition databases.
- Emotion recognition in the workplace and in education (except for medical or safety reasons).
- Biometric categorisation inferring sensitive attributes (such as race, political opinions, religion or sexual orientation).
- Real-time remote biometric identification in publicly accessible spaces for law-enforcement purposes, subject to narrow, authorised exceptions (for example, searching for victims or preventing an imminent threat).
Note that the "Digital Omnibus" simplification package agreed in May 2026 adds new Article 5 prohibitions targeting AI "nudifiers" (non-consensual intimate imagery) and child sexual abuse material.
As the AI Act applies, organizations must audit their AI systems to determine their risk category and ensure they meet the requirements. AI systems are classified as high-risk where:
- The AI is used as a safety component of a product, or is itself a product, covered by EU product-safety legislation listed in Annex I (which includes the Medical Devices and IVD Regulations); or
- The AI falls within one of the use cases listed in Annex III, unless it does not pose a significant risk to health, safety or fundamental rights.
For medicines specifically, it is worth noting that AI used purely in drug discovery, clinical analysis or manufacturing is generally not automatically "high-risk" under the AI Act, because medicines are not listed in Annex I; such pharma AI is governed primarily by EMA guidance and EU pharmaceutical law. Most medical-device AI, by contrast, is high-risk via the MDR/IVDR route. High-risk AI systems can still be placed on the market, but providers must fulfil requirements including:
- A risk-management system across the lifecycle.
- Data governance, with training, validation and testing data subject to appropriate quality practices.
- Technical documentation prepared before the product is placed on the market.
- Automatic record-keeping (logging) over the system's lifetime.
- Transparency and provision of information to deployers.
- Human oversight of the system.
- Appropriate accuracy, robustness and cybersecurity.
Notes regarding the AI Act:
- Research, testing or development activity before an AI system is placed on the market or put into service is, in general, outside the rules (real-world testing aside, which has its own regime).
- The regulation does not apply to AI used in the course of a purely personal, non-professional activity.
- EU law on the protection of personal data, privacy and confidentiality of communications continues to apply to AI systems.
Who will implement the AI Act?
Member states are responsible for enforcing the AI Act nationally, designating notifying and market-surveillance authorities that work with notified bodies and conformity-assessment bodies across the EU. At Union level, the European Commission's AI Office (operational since 2024) oversees general-purpose AI and coordinates consistent application, supported by the European Artificial Intelligence Board.
Because member states apply similar requirements, harmonised standards are needed; the European standardisation bodies (CEN-CENELEC) are developing them, and a voluntary GPAI Code of Practice was published in July 2025 to help general-purpose AI providers demonstrate compliance.
The Artificial Intelligence Act's key deadlines to look out for
The phased timeline is:
- 1 August 2024: EU AI Act enters into force.
- 2 February 2025: prohibited practices and AI-literacy obligations apply.
- 2 August 2025: obligations for providers of general-purpose AI models apply; governance and penalty provisions take effect.
- 2 August 2026: transparency obligations apply.
- High-risk obligations: originally 2 August 2026 (Annex III) and 2 August 2027 (Annex I). Under the May 2026 Digital Omnibus political agreement, these are postponed to 2 December 2027 (stand-alone Annex III systems) and 2 August 2028 (AI embedded in regulated products such as medical devices). These deferred dates become legally binding only once formally adopted and published in the Official Journal.
Fines for non-compliance range from EUR 7.5 million or 1% of global annual turnover, up to EUR 35 million or 7% of global annual turnover, depending on the severity of the violation and the size of the company.
Further changes to the AI Act are likely. Organizations should review their AI systems against current rules, keep open channels with regulators, and invest in AI governance training (including AI-literacy programmes, now a legal obligation).
Current status in the US
There is still no comprehensive federal AI law in the US, but the policy direction has shifted markedly and the FDA has issued substantive AI guidance.
In the medical device sector:
- In January 2019, the FDA published Developing a Software Pre-certification Program: A Working Model, aimed at more streamlined oversight of software-based medical devices.
- In April 2019, the FDA published a discussion paper, Proposed Regulatory Framework for Modifications to AI/ML-Based Software as a Medical Device (SaMD), describing its approach to premarket review for AI/ML-driven software modifications using real-world performance monitoring, algorithm change protocols and quality assurance.
- In January 2021, the FDA published the AI/ML-Based SaMD Action Plan, outlining five goals:
- A tailored regulatory framework
- Good Machine Learning Practice (GMLP)
- A patient-centred approach with transparency
- Regulatory science methods addressing algorithm bias and robustness
- Real-world performance (RWP) monitoring.
Since then, the FDA has issued binding-track guidance:
- In December 2024 it finalised guidance on Predetermined Change Control Plans (PCCPs) for AI-enabled device software functions, letting manufacturers pre-specify and obtain authorisation for future model changes without a new submission each time.
- In January 2025 it issued draft guidance on AI-Enabled Device Software Functions: Lifecycle Management and Marketing Submission Recommendations, taking a total-product-lifecycle approach (still draft as of mid-2026).
The FDA's public list of AI-enabled medical devices has now grown past roughly 1,000 authorisations, the majority in radiology.
For artificial intelligence in pharma and biotech:
- In 2023, the FDA released two discussion papers, on AI/ML in the development of drug and biological products, and on AI in drug manufacturing (the latter part of CDER's Framework for Regulatory Advanced Manufacturing Evaluation).
- Building on these, in January 2025 the FDA issued its first dedicated draft guidance, "Considerations for the Use of Artificial Intelligence to Support Regulatory Decision-Making for Drug and Biological Products," proposing a risk-based credibility-assessment framework based on a model's "context of use" and "model risk" (still draft, with finalisation anticipated).
- In January 2026, the FDA and EMA jointly published ten "Guiding Principles of Good AI Practice in Drug Development."
On the administration side, the picture has changed completely:
- The October 2023 AI Executive Order (EO 14110, "Safe, Secure, and Trustworthy AI") was rescinded by the incoming administration on 20 January 2025 and replaced by EO 14179, "Removing Barriers to American Leadership in Artificial Intelligence" (23 January 2025), a deregulatory, pro-innovation order that led to an AI Action Plan in mid-2025. So references to the 2023 Executive Order's eight principles and reporting requirements no longer reflect US federal policy.
On security:
- The FDA's Cybersecurity Modernization Action Plan remains relevant, and is now reinforced by statute: under section 524B of the FD&C Act, "cyber devices" must meet specific cybersecurity requirements, addressed in the FDA's final premarket cybersecurity guidance updated in June 2025. The FDA remains committed to ensuring drugs are safe and effective while facilitating innovations such as AI, including by deploying generative AI internally through its "Elsa" tool launched in 2025.
Upcoming regulations in the US
The FDA's dedicated drug/biologics AI guidance remains in draft as of mid-2026, with finalisation anticipated, and the device lifecycle-management guidance is likewise still draft. The agency continues to engage stakeholders and is expected to finalise its AI framework in the near term, increasingly in alignment with international partners such as the EMA.
Conclusion
The integration of artificial intelligence in the pharmaceutical and medical device sectors presents unprecedented opportunities but demands careful attention to ethical, security, and transparency concerns.
What was an emerging patchwork of proposals in 2023 has become an operational framework:
- the EU's AI Act is in force and in phased application,
- the EMA's guidance is final and being applied,
- international standards (ISO/IEC 42001 and its family) are available, and
- the FDA has issued dedicated AI guidance even as US federal policy has turned more deregulatory.
The landscape continues to evolve rapidly, which calls for proactive collaboration, compliance auditing, and continuous learning.
The transformative potential of AI in healthcare is no longer imminent but already here, requiring a shared commitment to responsible adoption for the benefit of individuals and global well-being.
Discover how Scilife eQMS can be your best ally to surf the wave of AI innovation in Life Sciences.
