In a recent paper, Tinglong Dai, Risa Wolf, and Haiyang Yang wrote about unlearning in Medical AI.
With more and more CPU and storage thrown at Large Language Models (LLMs) and Generative Artificial Intelligence (GenAI) in general, the capacity to “memorise” information grows larger and larger with each generation of LLM. This is further accelerated by the capacity to add specific details to generalist LLMs using Retrieval Augmented Generation (RAG) and agents (e.g., with the ability to query real-world systems at the interface with the physical world).
LLMs are learning more, but what about unlearning? Dai and colleagues didn’t evoke the analogy with human memory: our capacity to learn more relies, in part, on our capacity to forget, to reorganise, to summarise, and to prioritise the learnt information. Sleep and stress play a role in this reorganisation of information; this was the overarching topic of my Ph.D. thesis [link]. I will de-prioritise the visual cues along the path leading to a bakery if I no longer go to this bakery (“unlearning”). However, practising navigation to the bakery improved this skill, and this improvement will serve me later when I need to go to another place (something I could call “secondary learning”). It may seem we diverge from AI, but Dai and colleagues actually start their paper with the EU GDPR possibility for a patient to remove their data from a database, wondering how this is technically possible with LLMs (where data is not structured like in a traditional relational database and where the way data is retrieved is often unknown).
The “unlearning” process in LLMs can be considered from three encapsulated levels: algorithm, legal, and ethical levels.
At the algorithm level, Dai and colleagues focus almost exclusively on processes to add data to a foundational LLM. In this case, where any removal of data requires rebuilding (retraining) the whole model, the unlearning is understandably very complex (and very costly). The authors mention “approximate unlearning” and several methods of unlearning under investigation. However, the authors don’t mention methods that would incorporate the learning and unlearning processes into a second layer of the foundational LLM, rather than as an intrinsic part of it.
At the legal level, Dai and colleagues oppose the EU GDPR from 2016 to an FDA framework (which is more a discussion paper at the moment than something rigid). The comparison can’t really be made at the same level … The GDPR regulates all aspects of privacy in Europe, and it doesn’t tackle AI specifically, nor any medical use of data or any regulatory aspects of “software as a medical device”. The FDA discussion paper is somewhat the opposite, and for the USA: its focus is on AI and its use in software as medical devices, but it doesn’t have the broad scope of data privacy for the US. The other aspect is that authors might go beyond facts when they spread some FUD about potential future regulation in Europe, which I earlier referred to as “secondary learning” (they hypothesise that the EU may add regulations on learning from individual data, which, to my knowledge, is not currently in the books). However, they are correct in the fact that even if one finds a way to remove (or forget/unlearn) individual data from an LLM knowledge base, it may have already built additional knowledge from it. For instance, if several individuals have a specific gene and a specific medical condition, the LLM could have made the association between the gene and the condition. Removing some or even all individual data still leaves the association in the system. Is that a bad thing? Not necessarily: this is how scientists work. The scientific literature is full of clinical trial results where individual data are not displayed (for obvious reasons), but association tables are presented, and conclusions are drawn and remembered (not the individual data).
Finally, at the ethical level, Dai and colleagues raise the challenge of informed consent. If a patient gives their consent for data collection and analysis in a specific study, authors think that future technological advances could re-identify information that was previously de-identified. I personally believe that this issue belongs to the legal level: even if the technology is available, most (if not all) informed consents expressly forbid reuse of data for other goals than the study intention (which poses a problem for “data lakes”, “digital twins”, etc., but this is another discussion). In that view, combining external data and re-analysing new results should be impossible (legally).
In my opinion, the ethical issue of unlimited data retention is what the European GDPR tries to solve, striking a balance between the ever-growing need for data and the right of individuals to opt out of these systems. And the authors correctly identify another ethical issue of unlearning that already exists in the learning process. What if an AI system only learns data from a specific category of people? And therefore, what if the AI system only deletes (unlearns) data from a particular category of people? The issue is similar when we build an AI system only from melanoma images taken from white patients or when a diversified AI system unlearns all melanoma images from black patients: biases appear, and if they seem evident from the example mentioned here, other biases may be less obvious to notice.
The last question for me is also: what are we trying to achieve with unlearning? There are simple answers: to maintain a reasonable data lake for analysis, to reduce storage costs, to comply with GDPR, to make space for new data, and so on. However, if one attempts to mimic biological unlearning or build a human-like companion, we must carefully consider what we implement (as an unlearning algorithm) and the guardrails we put around it.
But my title was about how AI could unlearn in HEOR Modelling!
All the considerations above apply in HEOR, of course! When I mentioned that this paper’s authors didn’t mention RAG, I think this missed a significant opportunity to solve some of the unlearning challenges, especially in HEOR Modelling (and maybe in the application of AI in the medical field).
HEOR Modelling is a technical, numerical field where software plays an important role (as a tool). But we don’t have the time and budget resources to retrain a foundational model, and maybe not even host an LLM with billions of parameters (even a non-retrained one). Using a foundational model for what it is good at (building sentences, finding associations, writing model code, adapting models, …) leaves the possibility of using a second layer (typically a RAG or agents) where HEOR-specifics are learnt and possibly unlearned. Unlearning by simply regenerating the RAG database or changing the agents’ endpoints is much more manageable than retraining the whole foundational LLM. And unlearning using some of the emerging techniques the authors mention is also possible.
A side benefit to using learning contained to a RAG is that some of us need to compartmentalise our modelling development. In Consulting, as well as in some cases in Pharmaceutical companies, data and learnings from one client or one asset cannot be reused with other clients or other assets. Some learnings are beneficial (new developments in survival modelling in oncology could be of some use in models for life-prolonging drugs against rare diseases). But the latest data from company A’s clinical trial in Alzheimer’s Disease cannot interfere with the utility data from company B’s pivotal study. Using different RAGs per client or per project enables us to utilise a standard base (the foundational LLM) with learning and unlearning applied on a case-by-case basis.
These were some of my thoughts based on the Dai et al. paper. I didn’t mention security as I made the hypothesis that modellers are working in a secure environment provided by their company or academic group. However, security is a crucial topic, and adversarial training (and therefore adversarial unlearning) should be taken into consideration. Finally, as for other applications of AI in HEOR Modelling, transparency, explainability, verifiability, and the presence of health economic modellers remain essential.
Note: This post utilised AI to generate the two figures and correct the grammar.
Jean-Etienne's blog 