Medical system biases

A respectably sized randomized trial finds transcendental meditation has enormous effects on heart attack mortality, decreasing by half. Among the most vulnerable or the most involved in meditation, mortality decreased by 2/3rds. That would be huge even for a drug. As one of the study authors said, "The effect is as large or larger than major categories of drug treatment for cardiovascular disease."

Nonetheless, in spite of the much larger effect from meditation than any drug therapy, the article paraphrases the researchers as saying that the meditation should complement rather than replace drug treatment. If we believe that randomized trials yield correct information that can be used for treatment, why limit the results in this way?

Yes, the results need to be replicated a few times in different populations, etc., but my suspicion is that even after they are replicated (possibly with smaller effect sizes), the "don't stop taking drugs" message will remain.

Given recent results about increased risk of type 2 diabetes from statins and indications of memory problems from statins, those who advocate drugs need to defend their choice more. It seems that there's an implicit bias that treatment within the medical system must be healthier. Similar to the implicit bias against fat that caused Ancel Keys's views to prevail; now new studies that low-fat diets contribute to unhealthy weight gain are rarely publicized.

UPDATE: Now the article has been held back from publication due to last minute data, and the Telegraph took down the article. Wonder why.

Why the placebo effect is an effect

There was an interesting article in Wired recently that spoke about the placebo effect getting stronger: that the pre-post difference from a placebo drug is greater than it was a decade or two ago and that it differs between countries. That is, if you are looking at antidepressants and your outcome measure is a score on the Beck Depression Inventory that measures how depressed someone is, the score before the drug minus the score after the drug is different now than it was 10 years ago.

One criticism of the article is that the placebo effect cannot be considered an effect unless it is compared with another experimental condition. Since drug trials don't include both patients who receive a placebo and patients who receive nothing, there is no such thing as a placebo effect unless we know what the pre-post difference would have been in the absence of the placebo. Without a nothing arm to compare with, the writer contends that the pre-post difference in the placebo arm of a trial is just by definition the background noise in the trial.

I think that he's making a semantic point because a true placebo effect is impossible to measure.

To break the problem down further:

We do not know what the pre-post difference in a nothing arm of a trial would be. In some trials and for some diseases, there would be spontaneous improvement in the patient's condition: in that case, the pre-post difference in the placebo might just be that spontaneous improvement that would have happened if nothing were done.

In some trials and for some diseases, there would not be much change in the patient's condition, so the nothing arm would have no difference: in that case, the pre-post difference in the placebo arm would represent an "effect" and we could say that we have a placebo effect.

The question is which diseases have spontaneous improvement and which don't. There are three ways I can think of to figure this out.

1. A randomized clinical trial with patients that actually have some disease in which half the patients get a sugar pill and half the patients get nothing. No human subjects board would authorize this trial. Second, the study would not measure what we want it to. Ethically patients have to be told that the two possibilities are sugar pill and nothing. The Wired article contends that the placebo "effect" is based on a patient's prior beliefs about a drug's effectiveness, so it's specific to the drug, rather than being just the effect of a plain sugar pill.

2. The placebo effect could in theory be measured with matching, were there any subjects to match them to. The placebo pre-post difference can be defined in two ways: the pre-post difference of the sugar pill plus the pre-post difference of enrolling in the trial, or just the pre-post difference of the sugar pill alone. I would say it's the former. In that case, where we want to measure the effect of enrolling in a trial and taking a sugar pill, we could match normal patients with placebo patients based on their records and compare their pre-post differences. Except for the fact that medical records of normal patients with a disease are there because the patients are getting some treatment from their doctors. So there's no group to compare the placebo patients to.

3. The one remaining possibility is for each drug trial to divide their control group into two unequal groups: one receiving a sugar pill would be the larger group and one being put on a waiting list for the drug would be a smaller group. The problem is that placebos serve two purposes: one is for the statistical purpose and one is to keep the participants in the study and encourage them against taking other treatments. Depending on the condition, a control participant put on a waiting list might leave the trial or take another treatment in addition to the waiting list. So you might lose a good portion of the nothing arm of the trial.

Given the impossibility of rigorous measurement of what would happen under no treatment, the best we can do is guess which are the diseases where symptoms spontaneously resolve and which are the diseases where they don't. And that's what we already do when we talk about a placebo effect. We compare the pre-post difference in the placebo arm of a trial with our beliefs about what the pre-post difference would be with no treatment. In that sense, the placebo effect is really an effect. It's just imprecise.

Further, it's reasonable to assume that whatever the pre-post difference under nothing is, it's not going to change with time in any systematic way. If we could put all the placebo arms of, say, antidepressant trials together and find a trend with time, that's not sampling error. And that's exactly what the Wired article is talking about.

Overly conservative statistics and yogurt

Are overly conservative statistics preventing the adoption of low-risk potentially beneficial health care?

It seems like probiotics are being talked about everywhere. We know that "good bacteria" are vital in many cases: babies delivered vaginally versus via c-section, for instance, have better immune function due in part to the bacterial colonization they get on their way out. (Of course, if the mother has chlamydia or other bad bacteria, the babies can get colonized by those too and develop eye infections.) Now that flu is in the air, people are citing studies that certain probiotics can help prevent and shorten flu infection. Probiotics are inexpensive and reasonably harmless: the worst side-effects I've seen attributed to them are the same as placebos such as mild GI distress. Probiotics seem like the canonical case of "can't hurt, could help." Kefir and yogurt are tasty, too.

Recently I ran across an immunologist's summary of the report of a 2005 Yale medical school conference about probiotics, mentioning among other things that probiotics might be able to help a disease a friend has. The hypothesized mechanism makes sense that it would help, so I looked at the Cochrane reviews, a formalized method for summarizing medical literature, and they say there's no evidence. The only studies were so hopelessly small, though, that there's no way to know at this point. So I looked up "probiotics" in Cochrane and got these results showing that there are about 82 abstracts relevant to probiotics. Of the 10 or so that I read, the only ones where Cochrane said there was conclusive evidence was for acute infectious diarrhea.

An interesting case: pediatric antibiotic-induced diarrhea. They noted the effects of missing data: if all the study drop-outs were treatment failure, which seems unlikely, the treatment doesn't work. Immediately after that, they acknowledge that there is almost no downside to the treatment: "Probiotics were generally well tolerated and side effects occurred infrequently." and yet they conclude, "Although current data are promising, there is insufficient evidence to routinely recommend the use of probiotics for the prevention of pediatric AAD."

In other words, there's no downside to using probiotics, but because the overly conservative statistical analysis that counts all treatment drop-outs as failures finds that they don't work, they can't recommend them. There are many reasons why subjects might have dropped out of this study, primarily boiling down to the studies being almost certainly poorly funded and unable to adequately compensate busy parents of sick children needing to catch up on their lives after their children recovered. That caution in counting drop-outs as failures is reasonable in some cases: for instance, if the proposed treatment is invasive or risky. Or in the case of the female condom hearings the commercial sex workers who dropped out of the study could have been the ones for whom the condoms didn't work as well. In this case of probiotics, they're virtually risk free and there's a good reason why parents may have dropped out of the study.

In medical statistics (biostatistics), the methods most commonly used are straight out of a textbook, rules of thumb that apply in general. Obviously context counts and we should be more conservative when there's a risk and less conservative when there's little risk. Biostatistics is not my primary area, but I have helped doctors out with the occasional clinical trial, using the textbook methods because that's what they wanted. There are many better methods that could be used to analyze this data, such as decision theory that accounts for risks, or missing data methods that model the potential outcomes of the study drop-outs. Biostatisticians have no malpractice risks, so there's no reason they couldn't be less conservative in their choice of data analysis methods to account for risk. Somehow the conservatism that US doctors practice under has spread to biostatisticians, though. Until statistics becomes less conservative in their analysis methods, patients may end up missing out on low-risk treatments still being studied.