Hospitals Are Using AI to Detect Intimate Partner Violence. That’s a Problem.

A patient walks into the emergency department (ED) with a nagging migraine that won’t let up. She knows that in the ED she’ll be able to get a medication strong enough to provide relief. She answers the staff’s questions as she waits for imaging to rule out a life-threatening cause.

What she does not know is that the hospital will run her clinical notes, imaging reports, and other available health data through an artificial intelligence (AI) model trained to calculate a risk score about the likelihood of whether she is experiencing intimate partner violence (IPV). Now, something that she has never disclosed to a healthcare provider, something that she held back out of fear of judgment, of not being believed or supported, and worry about what would happen to her children or what her partner might do if he found out, is being reflected in her electronic medical record (EMR).

This spring, the high-profile killings of women, like Cerina Fairfax, DDS, and Coral Springs, Florida, Vice Mayor Nancy Metayer, were a sobering reminder that IPV remains persistently common and widespread. Authorities say Fairfax was killed by her husband before he committed suicide; Metayer’s husband has been formally charged with premeditated murder. While IPV affects people of all gender identities, it disproportionately impacts women, with nearly one in three U.S. women experiencing contact sexual violence, physical violence, or stalking by an intimate partner during their lifetimes. Women with a history of IPV have higher healthcare utilization and costs, which continue long after the violence ends. As a result, healthcare settings are thought of as an optimal setting for screening women for IPV to connect them with needed survivor support services.

Enter AI.

A recently published systematic review by Yang Li, PhD, RN, and colleagues, identified 41 published studies in a range of sectors examining the role of AI for IPV prevention. Among those focused on healthcare settings, deep learning and natural language processing were used to detect or predict the risk of IPV in clinical notes, imaging reports, and other types of hospital data. Another study identified survivors about 3 years before they entered a violence prevention program. Meanwhile, an AI system using a similar approach has already been rolled out into clinical practice. The Automated Intimate Partner Violence Risk Support System (AIRS) uses data, such as clinical notes and imaging reports, to screen patients. It has been implemented in the Brigham and Women’s Hospital emergency department and some of its primary care sites.

The rationale is incredibly well-intentioned and understandable. Only about 7% of survivors disclose IPV with traditional screening methods in clinical settings. This type of screening typically asks the patient directly if they have experienced harm from their intimate partner, such as hitting, kicking, or punching — an approach that can re-traumatize survivors. However, using AI to detect IPV overlooks two major findings from decades of IPV advocacy and research.

First, many survivors want the autonomy to be able to disclose on their own terms, at a time and place that is right for them. Survivors often make daily or even moment-to-moment calculations about their safety; they know their partner and their patterns intimately. An AI system that detects IPV risk removes a survivor’s control over whether, when, and how to disclose to a provider.

For AIRS, based on information from its website, which includes clinician- and patient-facing videos, it appears that patients are not told that they are being screened, and there is no option described that allows patients to opt out. Only patients who are approached by the healthcare team will know there has been a score generated and could then ask for it to be removed from their medical record. However, patients whose scores and other data do not prompt the healthcare team to approach them, or whose clinician does not disclose the tool, will have no idea this screening occurred.

This raises fundamental questions about consent and transparency. It reaffirms a privacy concern that many survivors already have about how their data will be used. While the score is described as only being available to the care team, it is possible that third parties could access these records through data breaches, insurance requests, or court orders around child custody.

Second, it has become increasingly clear that universal education about IPV helps survivors. It’s an approach that has shifted the goal away from disclosure and toward healthcare providers sharing information with all patients about how relationships can affect one’s health and how to access survivor support services. Trials of universal education have been associated with women being more likely to leave relationships that feel unsafe, greater disclosure among youth experiencing IPV, lower self-reported social isolation, and increased awareness and connection to survivor support services.

While the AIRS protocol does include what it calls “universal education,” it only provides it after a “positive alert” from the system, which means that patients with lower risk scores may not receive education about relationships and health. Moreover, we don’t yet know whether identifying a survivor via AI improves their health and safety outcomes.

Survivors want to be able to disclose when they are ready, and to be treated without judgment and with respect for their choices. They want to be connected to the services that promote health and safety, like housing, legal advocacy, economic support, childcare, and mental healthcare. Survivors have said what they want and need, and they should also be able to decide what role, if any, AI should play in their care.