The Limits of Big Data in Psychiatry

While browsing The Atlantic earlier this week, I came across this:

Yes, I was tempted to click on the article involving electric shocks, but it was the ad "Rising Mental Health Issues Facing Our Children, in Five Charts" that caught my attention. The colorful charts show some alarming-looking numbers that most readers of this blog are probably used to seeing by now: that there is a large increase in children receiving mental health diagnoses, that ADHD is diagnosed at very high rates (especially in the South), that children on Medicaid are more likely to receive a mental health diagnosis than children with commercial insurance, etc. The data for the charts were gathered from pediatrician visits across the country by athenahealth (apparently they're too cool for capital letters), the electronic health record (EHR) company who paid for the ad.

They helpfully included a video at the bottom of the ad so we can get "perspectives on these trends from top health care leaders." Take a look:



Watching this video, I was struck by the words of Kurt Newman, M.D., President and CEO of Children's National Medical Center:

"These graphs are just probably the tip of the iceberg. The directional trend is very disturbing, but also the magnitude is disturbing, and these pediatricians are swamped.
[…]
That's why we need to do more research, we need to have a better system in terms of more providers, we need to be able to pay the providers a reasonable amount for the care they're giving. But I think if we do all that, we're going to have a huge impact for these kids and families."

Classic. There's an epidemic on, doctors are swamped—we need more funding so we can provide more treatment! No wonder he's the CEO. And like many other CEOs, he oversells when talking about the future:

"We're on the cusp of something really huge there. It's kind of like big data and big analytics that are gonna really revolutionize how we can identify these trends or get specific about certain diseases […] Autism might be a hundred different rare diseases that are all rolled up into one. We won't figure that out unless we have the analytics, all of the the really sophisticated capability of probing into: is that patient like that patient, is that child like that child, what made them more similar?"

Perhaps I'm too dumb to comprehend big data/analytics, but I fail to see how information mined from an EHR is going to shed light on the etiology of autism. Also featured in the video is Angela Diaz, M.D., Director of the Mt. Sinai Adolescent Health Center, who seems to have a more common sense take on the data:

"We need to figure out what is leading to these kids…30% of U.S. students to feel sad and hopeless for the last 12 months, and of those, 40% of the girls? What is going on? So we need to get to the root causes of these things, and try to identify and then figure out, how to prevent?"

I certainly agree with Dr. Diaz on the importance of trying to determine the root causes of the rising rates of these conditions. However, having the raw data and figuring out causality are two very different things. I would argue that in psychiatry we already have access to tons of data, but unfortunately much of it is interpreted through a very narrow, biologically-oriented lens. Having faster access to bigger pools of data is not going to help. Example in point: the January 2014 JAACAP article that described rising rates of ADHD in the US, which I had previously blogged about. That article was accompanied by an editorial by Drs. Walkup, Stossel, and Rendleman that essentially heralded the findings as good news and a sign that ADHD is being increasingly recognized and treated, which is desirable from a "public health" point of view.

In the June 2014 issue of JAACAP, Dr. Jonathan Posner wrote a very reasonable letter to the editor (subscription required), pointing out that other, more rigorous studies (relying on both parent and teacher report instead of parent report only) have found rates of ADHD closer to 5-7% instead of the 11% reported in the JAACAP article. He concludes that the reason for the rapidly increasing rates of ADHD diagnosis in the community may be "that a substandard approach to diagnosing ADHD has become the norm." Drs. Walkup and Rendleman wrote a reply (subscription required); here's the first paragraph of their response:

Thank you very much for your comments. Your position is one that we believe is shared by many, which is why we wrote the piece. Although we respect your and others’ opinions, we find it difficult to support the statement that rising rates are due largely to substandard assessment of ADHD—it is just too simplistic an explanation. The solution that you allude to is likely not tenable for a high-prevalence condition such as ADHD, because there just aren’t enough child psychiatrist providers to do it all. We are not advocating poor-quality diagnosis or inappropriate treatment; rather, the goal of the editorial was to understand the role of advocacy and education in rising rates, the importance of a public health approach to high-prevalence conditions, and to help child and adolescent psychiatrists come to terms with the fact that our traditional model of care, which is time intensive and highly personalized, is not likely to be able to address the public health burden of ADHD. We certainly do not want to inhibit the pediatric prescriber from taking on the challenge. They need our support to do it well.

So the assumption they make is that cases of ADHD reflect a biological disorder and that increasing awareness of the condition amongst the population, diagnosing it, and treating it with medications is good and proper.

Imagine, for moment, something like this happening with the obesity epidemic. The maps of child obesity in the U.S. look suspiciously like those of the ADHD epidemic, with the highest rates in the deep South. Sure, there are drugs to treat obesity, but would anyone talk with a straight face about a "public health" approach to obesity consisting of identifying the cases and then treating them with medications? Wouldn't a better (and true) public health approach be to ensure that children can get adequate exercise, good nutrition, and that people aren't incentivized to buy the cheap calories and processed "foods" that are making them obese?

Thus, as long as the prevailing view of psychiatric conditions is a narrow one, the data will be used for narrow purposes, such as academic leaders/CEOs arguing for more resources, or to justify the high rates of psychiatric medication prescribing. Here, I'm not even going to get into some other questions I had about the athenahealth ad, including who its intended audience is and what it is trying to achieve. Most doctors recognize that EHRs do not help them care for patients. These systems mostly appeal to large clinics and hospital organizations, for reasons that I will let Dr. George Dawson's recent blog post explain.