Overly conservative statistics and yogurt

Are overly conservative statistics preventing the adoption of low-risk potentially beneficial health care?

It seems like probiotics are being talked about everywhere. We know that "good bacteria" are vital in many cases: babies delivered vaginally versus via c-section, for instance, have better immune function due in part to the bacterial colonization they get on their way out. (Of course, if the mother has chlamydia or other bad bacteria, the babies can get colonized by those too and develop eye infections.) Now that flu is in the air, people are citing studies that certain probiotics can help prevent and shorten flu infection. Probiotics are inexpensive and reasonably harmless: the worst side-effects I've seen attributed to them are the same as placebos such as mild GI distress. Probiotics seem like the canonical case of "can't hurt, could help." Kefir and yogurt are tasty, too.

Recently I ran across an immunologist's summary of the report of a 2005 Yale medical school conference about probiotics, mentioning among other things that probiotics might be able to help a disease a friend has. The hypothesized mechanism makes sense that it would help, so I looked at the Cochrane reviews, a formalized method for summarizing medical literature, and they say there's no evidence. The only studies were so hopelessly small, though, that there's no way to know at this point. So I looked up "probiotics" in Cochrane and got these results showing that there are about 82 abstracts relevant to probiotics. Of the 10 or so that I read, the only ones where Cochrane said there was conclusive evidence was for acute infectious diarrhea.

An interesting case: pediatric antibiotic-induced diarrhea. They noted the effects of missing data: if all the study drop-outs were treatment failure, which seems unlikely, the treatment doesn't work. Immediately after that, they acknowledge that there is almost no downside to the treatment: "Probiotics were generally well tolerated and side effects occurred infrequently." and yet they conclude, "Although current data are promising, there is insufficient evidence to routinely recommend the use of probiotics for the prevention of pediatric AAD."

In other words, there's no downside to using probiotics, but because the overly conservative statistical analysis that counts all treatment drop-outs as failures finds that they don't work, they can't recommend them. There are many reasons why subjects might have dropped out of this study, primarily boiling down to the studies being almost certainly poorly funded and unable to adequately compensate busy parents of sick children needing to catch up on their lives after their children recovered. That caution in counting drop-outs as failures is reasonable in some cases: for instance, if the proposed treatment is invasive or risky. Or in the case of the female condom hearings the commercial sex workers who dropped out of the study could have been the ones for whom the condoms didn't work as well. In this case of probiotics, they're virtually risk free and there's a good reason why parents may have dropped out of the study.

In medical statistics (biostatistics), the methods most commonly used are straight out of a textbook, rules of thumb that apply in general. Obviously context counts and we should be more conservative when there's a risk and less conservative when there's little risk. Biostatistics is not my primary area, but I have helped doctors out with the occasional clinical trial, using the textbook methods because that's what they wanted. There are many better methods that could be used to analyze this data, such as decision theory that accounts for risks, or missing data methods that model the potential outcomes of the study drop-outs. Biostatisticians have no malpractice risks, so there's no reason they couldn't be less conservative in their choice of data analysis methods to account for risk. Somehow the conservatism that US doctors practice under has spread to biostatisticians, though. Until statistics becomes less conservative in their analysis methods, patients may end up missing out on low-risk treatments still being studied.