Saturday, February 4, 2017

Another bogus 5% success rate claim found in the wild

I have found another bogus 5% success rate claim in the wild, in the comments for this post:

http://www.patheos.com/blogs/secularoutpost/2016/11/04/pseudo-random-thoughts-on-aa-and-free-will/

The claim comes from a misreading of this passage in "Alcoholics Anonymous Effectiveness: Faith Meets Science":
As stated at the outset, the experimental evidence for AA effectiveness(addressing specificity) is the weakest among the six criteria considered crucial for establishing causation. Only two studies provided strong proof of a specific AA or TSF effect: the outpatient arm of Project MATCH (with effects at 1 and 3 years), and the intensive referral condition in Timko’s trial (with effects for abstinence at 6 months and 1 year).
The effect sizes were similar, with the TSF/Intensive referral
conditions having a 5-10% advantage in abstinence rates.
(Emphasis mine)

This passage uses a lot of specialized terminology and I can see why someone would misread it to mean that AA only has a 5-10% success rate. I would render the above paragraphs like this:
As we mentioned before, the evidence for the AA program helping alcoholics (and not other factors, such as the fact that people who take AA's program seriously simply being the people who want to get sober the most) is weak. Only two studies (Note: This was written before Humphreys 2014) provided strong proof that the AA program itself, or doctors trying to get alcoholics engaged in the AA program via TSF, is helping alcoholics.
The amount of effect we saw the program helping people was similar; when doctors tried really hard to get their patients to work the AA program (TSF), the patients were 5-10% more likely to stay sober.
In other words, whether or not an alcoholic works the AA program is a decision only the alcoholic can make. If a doctor tries really hard to get someone to work the program, the alcoholic is 5-10% more likely to get sober. This figure is not an overall success rate for AA. We know that, if the alcoholic himself decides to work the program, they have, depending on the study, between a 75% (Vaillant 1995; Fiorentine 1999) and 67% (Moos and Moos 2006) chance of staying sober.