State lawmakers eye...

Psychiatrists, a sponsor of the conservatorship bill, SB 43. “We have to talk to their families who know that they need that care, and we have to say we don’t have any legal basis to bring them into the hospital right now.” disorder is not a trigger for conservatorship. range of performance. One preliminary research paper examining ChatGPT and Google products using open-ended board examination questions from neurosurgery found a hallucination rate of 2%. A study by Stanford researchers, examining the quality of AI responses to 64 clinical scenarios, found fabricated or hallucinated citations 6% of the time, co-author Nigam Shah told KFF Health News. Another preliminary paper found, in complex cardiology cases, ChatGPT agreed with expert opinion half the time.

To Wood, treating these people, even when they’re unable to consent, is the compassionate, moral thing to do.

Privacy is another concern. It’s unclear whether the information fed into this type of AI-based system will stay inside. Enterprising users of ChatGPT, for example, have managed to get the technology to tell them the recipe for napalm, which can be used to make chemical bombs.

In theory, the system has guardrails preventing private information from escaping. For example, when KFF Health News asked ChatGPT its email address, the system refused to divulge that private information. But when told to role-play as a character, and asked about the email address of the author of this article, it happily gave up the information. (It was indeed the author’s correct email address in 2021, when ChatGPT’s archive ends.)

“I would not put patient data in,” said Shah, chief data scientist at Stanford Health Care. “We don’t understand what happens with these data once they hit OpenAI servers.”

Tina Sui, a spokesperson for OpenAI, told KFF Health News that one “should never use our models to provide diagnostic or treatment services for serious medical conditions.” They are “not fine-tuned to provide medical information,” she said.

With the explosion of new research, Topol said, “I don’t think the medical community has a really good clue about what’s about to happen.”

KFF Health News is a national newsroom that produces in-depth journalism about health issues and is one of the core operating programs at KFF—an independent source of health policy research, polling, and journalism. n

Under current California law, a person can be held in the hospital involuntarily if they are a danger to themselves or others or if they are unable to seek food, clothing, or shelter as a result of mental illness or alcoholism. Doctors want to add other substance use disorders to the criteria, as well as an inability to look out for one’s own safety and medical care. (The state law defines what is known as “mental health conservatorship,” which is separate from the probate conservatorship that Britney Spears was under.)

Wood, who practices in Los Angeles, gave two examples of people she and her colleagues have tried, but struggled, to care for under the current rules. One is a man who doesn’t take his diabetes medication because he’s not taking his schizophrenia medication and doesn’t understand the consequences of not managing either condition.

Wood explained that even if he repeatedly ends up in the emergency room with dangerously high blood sugar, no one can compel him to take either medication under current law, because poorly managing one’s health is not a trigger for conservatorship.

Another man Wood described has a developmental disability that went untreated in childhood. He developed an addiction to methamphetamine in his 20s. Wood said the man is now regularly found sleeping in a park and acting inappropriately in public. His family members have begged doctors to treat him, but they can’t, because substance use

“It’s essential that we respect all the rights of our patients, including the right to receive care from us,” she said.

But other advocates, including some of those working for Californians with mental illnesses, see the issue very differently.

Lawyers from the nonprofit Disability Rights California said the proposed expansion of conservatorship and the ongoing rollout of CARE Courts are misguided efforts, focused on depriving people of their liberty and privacy.

Instead, they said, the state should invest in better voluntary mental health services, which help maintain people’s dignity and civil rights. The group filed a petition in January to try to block the implementation of CARE Courts.

These advocates are particularly concerned that people of color, specifically Black residents, who are overrepresented in the homeless population and overdiagnosed with schizophrenia, will now be disproportionately targeted by more forceful measures.

“When people are told that they have to go to court to get what they should be getting voluntarily in the community, and then they get a care plan that subjugates them to services that still do not meet their cultural needs, that is not compassion,” said Keris Myrick, an advocate who has schizophrenia and has experienced homelessness.

More Housing: Another Badly Needed Prescription

Under current state law in Oregon, a person can be held for involuntary treatment if they are a danger to