It is not often the FDA offers truly transformative guidance to enable industry to constructively advance cutting edge technology in a practical way. But with the draft FDA guidance titled “Marketing Submission Recommendations for a Predetermined Change Control Plan for Artificial Intelligence/Machine Learning (AI/ML)-Enabled Device Software Functions” released on April 3, 2023, the FDA has done just that. There is far too much on this topic to restrict to a single post here, as industry is eagerly awaiting the final version of this guidance which FDA will publish at a point TBD in the future which may include significant changes. This post will focus on some key points related to this guidance that impact what device developers need to consider going forward, as a FDA authorized PCCP allows the opportunity for device manufacturers to enact certain changes to AI / ML device software functions (also called ML-DSFs) described in a FDA authorized PCCP without having to submit another marketing submission (i.e, 510(k), de novo, PMA). This post will also look at how this guidance is starting to functionally filter into 510(k) clearances and de novo classifications, which demonstrate how the FDA is accepting marketing submissions that include PCCPs.
First off, at a high level the Predetermined Change Control Plan (PCCP) refers to calling out in a marketing submission, such as a 510(k), de novo classification or a PMA, three major components to describe the PCCP in (1) a Description of Modifications which includes “a range of FDA-authorized specifications”, (2) a Modification Protocol which includes “associated verification and validation testing and acceptance criteria to assure the device remains safe and effective across relevant patient populations” and (3) an Impact Assessment which includes “documentation of the assessment of the benefits and risks of implementing a proposed PCCP”.
In practical terms, the Description of Modifications allows a medical device manufacturer with software that includes AI / MI to describe detailed description of each planned modification to a ML-DSF within the device’s Indications for Use (IFU) statement and describe these changes to device characteristics and performance resulting from implementation of modifications. A FDA webinar on this topic gave three major (but not limited to) examples of these Description of Modifications as the following:
Modifications related to quantitative measures of ML-DSF performance specifications: Improvements to analytical and clinical performance resulting from re-training the ML model based on new data within the intended use population from the same type and range of input signal
Modifications related to device inputs to the ML-DSF: Expanding algorithm to include new sources of same signal type (such as different makes, models, or versions of a data acquisition system or hardware) or limited modifications related to new types of inputs
Limited modifications related to device’s use and performance: Authorization of a device for specific subset of a population within originally indicated population based on re-training on a larger data set for that subpopulation not previously available
This Description of Modification gives the opportunity, broadly speaking, to sketch out in a PCCP what changes you plan to enact in the future with your AI / ML system, and as long as your properly describe and capture these proposed changes that fall within your approved indications for use, you have the opportunity to document per quality system procedures these changes and implement without another marketing submission. This in itself is a dramatic shift from prior understanding at FDA, where changes to AI / ML required another marketing submission due to re-training of data for improved outcomes, use of different hardware such as different smartphones or smartwatches or re-training of data to include a population that wasn’t available on initial submission. It behooves any device manufacturer with AI / ML functionality to very seriously consider these future changes as part of any marketing submission, and potentially include a proposed PCCP in a pre-submission with FDA if one is already planned for other purposes.
The Modification Protocol component of a PCCP creates a testing plan to support the Description of Modifications with four major top level components described in FDA webinar on this topic as (1) data management practices, (2) re-training practices, (3) performance evaluation protocols, and (4) update procedures, including communication and transparency to users and real-world monitoring plans. Fundamentally, the Modification Protocol piece of a PCCP shares many aspects of the quality system driven software documentation efforts that would be needed as part of any marketing submission, but the big difference being the PCCP allows to call out potentially future plans for change.
The Impact Assessment piece of a PCCP in a nutshell looks at the risks / benefits that changes proposed in PCCP. This was described in a FDA webinar with five major high level components: (1) Compare version of device with each modification implemented to version of device without any modifications implemented, (2) Discuss benefits and risks of each individual modification, (3) Discuss how activities proposed within Modification Protocol continue to reasonably ensure safety and effectiveness of device, (4) Discuss how implementation of one modification impacts implementation of another and (5) Discuss collective impact of implementing all modifications. Fundamentally the Impact Assessment part of PCCP shares elements of risk management driven documentation efforts that would be needed as part of any marketing submission, but is looking forward to the future.
The term “future” has been used frequently here, so it is worth noting nothing in a PCCP commits an applicant to make any or all of those changes spelled out in a PCCP. If for whatever reason changes proposed in a PCCP are deemed unnecessary or irrelevant, the PCCP remains as a plan only but one that caries no obligation for execution. It really is a fascinating opportunity presented by FDA to essentially allow for changes that you may do, but don’t necessarily have to do, without requiring additional marketing submission(s) as long as an FDA authorized (i.e., part of a cleared, classified or approved submission) PCCP guidelines are met.
A fair question to ask is, if this is such a great opportunity, who is taking advantage of this? The draft guidance on PCCP was only published in April 2023, but a de novo classification was classified in late February 2023 that spoke about PCCP in multiple locations, which suggests this was a test run of sorts at FDA for working out the concept ahead of draft guidance publishing. In late July 2023, Apple received a 510(k) clearance for an app used on the Apple Watch that spoke extensively about PCCP, which is clearly a sign that industry has taken notice of the opportunity.
As a further positive outcome to industry, the FDA webinar discussion on this guidance made clear that while this guidance “focused on PCCPs for AI/ML-enabled devices, the omnibus provision applies to more than just AI/ML. And in fact, the bill provides authority for PCCPs for all devices.” This is a significant statement and makes clear the FDA is looking ahead at enabling the entire device industry, if a properly considered PCCP is included in their marketing submission, to expand their device utility and functionality without additional marketing submissions if the PCCP is correctly executed. Properly constructing a PCCP is the smart thing to do for those in the digital health device industry, and the medical device industry as a whole, but one has to include as part of as part of a marketing submission. A real boon of the PCCP could be for the FDA itself, since if industry sits up, takes notice and takes advantage of PCCP opportunities, it is practical in reducing the workload of marketing submission review at FDA on what is already a heavily utilized and arguably strained agency.
