The US FDA, Health Canada and the UK MHRA have jointly identified 10 guiding principles that can inform the development of Good Machine Learning Practices (GMLPs)

The UK MHRA, the US Food and Drug Administration (FDA) and Health Canada have come together to work on the ten basic guiding principles for developing good machine learning practices (GMLPs). These principles help us promote high quality, safe and efficient devices using artificial intelligence and machine learning. AI/ML has the potential to capture useful insights from the huge amount of data available on the day-to-day healthcare that is being delivered. Software algorithms are used which are helpful in learning from the real world, which in turn helps us to improve the performance of the product. These software algorithms are also data-driven and iterative in nature, thus taking into account the unique considerations that arise due to their complexity.

The field of AI/ML medical devices is continually evolving, and so are GMLP practices. These ten guiding principles are the foundation for developing good machine learning practices to address the nature of these products and, at the same time, encourage future development in this growing field. These guiding principles can be used to implement positive habits that have proven successful in other fields, to implement practices from other industries that may prove useful in the medical technology and health sector , and to develop innovative procedures suitable for the medical and healthcare industry. . Now let’s look at some of the guiding principles of these practices.

Guiding principles:

• Multiple uses and needs must be considered during product development and throughout its life cycle. To ensure that ML-enabled medical devices are safe and effective and meet clinically significant needs throughout the lifecycle of the device, it can be helpful to have a thorough understanding of the intended integration of a model in the clinical workflow, as well as desired benefits. and the associated risks to the patient.
• Model design should be implemented with careful attention to fundamentals such as good software engineering practices, data management, and data quality. These practices also incorporate systematic risk management and design processes that can effectively explain and document design, implementation, and risk management decisions and rationales. They also guarantee the integrity and authenticity of the data.
• The data collected must include relevant characteristics of the target patient population, and the measurement inputs must be sufficiently adequate for the training and test data set so that the output can be reasonably generalized. Additionally, it is important to manage biases to promote generalized performance for the patient population and to identify circumstances where the model may underperform.
• The training and test data sets should be selected to be independent of each other. To ensure independence, all potential sources of trust, including patient, data acquisition, and site characteristics, are considered and addressed.
• The most effective techniques for creating a benchmark data set are used to ensure that clinically relevant and well-characterized data are collected and that the limitations of the benchmark are recognized. If available, reliable benchmark data sets that support and illustrate the robustness and generalizability of the model across the intended patient population are used in the creation and testing of the model.
• The clinical benefits and risks of the product are well recognized, used to develop clinically meaningful performance targets for testing, and support the idea that the product can be used safely and effectively for the purpose for which it was designed. Both global and local performance are considered to estimate the uncertainty and variability of device inputs and outputs.
• Human factors and human interpretability considerations must be taken into account. At the same time, model outputs are discussed, focusing more on Human-AI combo performance than just model performance.
• Robust test plans are drawn up, developed and executed. The target patient population, significant subgroups, clinical setting and its use by the team, measurement inputs, and any confounding variables are all factors to keep in mind.
• Users get quick access to understandable, contextually relevant, and audience-specific information, including model performance for a particular group, acceptable inputs, recognized drawbacks, UI interpretation, and integration of the model in clinical workflows. Additionally, users receive information about device upgrades and changes from real-world performance monitoring, basis for decisions if any, and a channel to raise issues regarding the product with the developer.
• Deployed models can be observed in real applications to maintain or improve performance. Additionally, appropriate controls are in place to mitigate the risk of overfitting, unintended bias, or model degradation that may affect model safety and performance when used by the Human-AI team when models are trained regularly or continuously after deployment.

Check out the e-Paper and Reference link. All credit for this research goes to the researchers on this project. Also don’t forget to register. our Reddit page and discord channelwhere we share the latest AI research news, cool AI projects, and more.