agAInst bias
Sharing some of our thoughts about the thoughtful design of Health Equity AI for Veterans.
Designing against discrimination - We believe the best way to achieve a fair and equitable system is to co-design with Veterans for Veterans. Because nothing is truly for a Veteran unless it is built by a Veteran. To achieve this we will recruit a panel of Veterans across the religious, geographic, racial and cultural spectrum to inform our technology, policies and procedures. This panel will be led by our advisor, a US marine, and our Veteran serving community based organization partner Volunteers of America (VOA). Should a Veteran opt-out of having AI as part of the clinical summary process, we will provide summarization as a service performed by clinicians. And audited by our Veteran’s panel.
Designing against bias - Our clinical care report utilizes a large language model that is being designed against bias. We are training the model to incorporate inclusive language guides from VOA, Psych/Armor + Google, American Psychological Association, and Northwestern Family Institute. Too often, our clinicians have seen how the language used in clinical documentation can create bias. When a patient is labeled as “non-compliant” this negates the complex social determinants of health that often contribute to access to medications or ability to follow through with a clinical plan. It also has the risk of carrying forward into future clinical notes and biasing the care of a Veteran. At scale, and with AI, if we do not design against this, we will compound this critical issue. We see an opportunity for our AI to be part of a sea change in how we communicate about all patients, particularly Veterans. And, as so many of our future nurses and doctors are VA trained this can also create a generational shift in how clinicians document.
Monitoring for bias - Anti-bias LLM. To actively monitor against bias, we will train an Anti-bias LLM to review our database of clinical care records, flag and extract language that does not meet our standards. This language will be reviewed, edited, and new clinically accurate, non-bias language will be incorporated into our main clinical care summarization model.
De-biasing input: There is an opportunity to report back to CCNPs findings regarding use of bias language or data. With guidance from the VA and our Veteran’s panel we would communicate to CCNPs via “tips” and “nudges” about how to de-bias clinical care documentation. This will de-bias the input to our system.
Sensitive Data Categories - When designing for equity it is important to design not only for the majority, but as importantly minority populations which include service members that are women, of color, and LGBTQ. This approach ensures that we center the privacy, rights, culture, language, ability, and disability of a diverse Veteran community.
In doing so, it is important that we capture and track health equity, diversity and inclusion data in healthcare. However, this same data can introduce bias. Commercial Large language models have been trained on biased data sets, and this bias seeps into the output. Recognizing and planning for this is essential and maintaining vigilance for categories of data that can produce outsized harm should bias seep in is critical. This data includes, but is not limited to, race, ethnicity, language, gender preference, pronouns, political identity and more.
Dataset design and accountability - we will track and report the demographic and equity data of our data sets so that we can be held accountable for the real or synthetic data that our current, and future models will train on. We will maintain a data dashboard that will provide VA officials transparency regarding the data that is used to train our models. This dashboard will be used to identify outliers, trends and omissions to our data set so we can ensure that we do not neglect or over-index certain populations. With this information we can alter our synthetic datasets to address issues. And, to match current, and projected future demographics of the Veteran population by tracking Veterans, active-duty members. This will enable our model to become finely attuned to the demographic changes in the Veteran population.