Response to “Does Alcoholics Anonymous Work” published by The Conversation

I came across an article which criticizes AA using the in my opinion now-debunked 2014 book by Dodes. I have already dealt with why the Dodes’ figures are suspect in previous blog entries, but to summarize: The figures have been, in my opinion, downright discredited in Cochrane 2020’s report on AA efficacy.

One thing that article does is point to a study which “proves” that alcoholics can control and moderate their drinking. One thing to keep in mind is that while it’s pretty easy to find a study which shows that out-of-control alcoholics claim they are controlling their drinking, those studies are generally short term and based on self-reported data. The last time we had a real long-term study was the 10-year follow-up in Pendery 1982, a famous (if old) study showing that alcoholics do not successfully control and moderate their drinking over a long time period.

But, interested, I looked at this meta-study supposedly showing controlled drinking among alcoholics. This meta-study included studies going back to the 1970s. Most of the studies had only short-term follow-ups; there were only two studies with follow-ups longer than two years.

Diving deeper, I looked at the two studies with a final follow-up longer than two years used to support the idea that an alcoholic can control and enjoy his drinking:

• One study with a 3.5 year follow up is Graber et. al. 1988 [1]. N=24 (which is a much lower N than the studies used to support Cochrane 2020) Looking at the study, I observed that while the supposed “controlled drinkers” claimed to be drinking less than the group told to abstain from alcohol, the collaterals (usually, the drinker’s loved ones) reported that the “controlled drinkers” were actually drinking a lot more than the abstinent group: Self-reported data among the abstinent group was 21 drinks a week (collaterals reported less); collaterals reported nearly 41 drinks a week for the controlled drinking group (self-reporting was a good deal lower).

• The other study they used was from back in the 1970s: Caddy et. al. 1978 [2]. I started reading this study before metaphorically throwing it across the room. It looks at alcoholics in the infamous Sobell 1973 study. The very same alcoholics they looked at in Pendery 1982! (To be pedantic, Pendery 1982 only looked at the 20 alcoholics in the controlled drinking experimental group, while Caddy 1978 looked at all 70 subjects across four groups, but still) At this point, I stopped bothering to read the meta-study.

We know exactly what happened to that very same CD-E (experimental controlled drinking) group of 20 alcoholics 10 years later: They weren’t controlling and enjoying their drinking! (4 dead, 8 out-of-control drinking, 6 abstinent, 1 moderately drinking, 1 not located) Despite this, the meta-study, without even mentioning the existence of Pendery 1982, claimed this study showed the controlled drinking group did better than the abstinent group. I can not take seriously a meta-study which uses data based on those 20 people in Pendery 1982 to argue an alcoholic can control and enjoy his drinking.

If we throw out the data based on the subjects in Pendery 1982, we are left either with studies whose final follow-up is at most two years after treatment, or with the all of 24 subjects in Graber et. al. 1988 where the “moderate drinking” (“controlled drinking” as they called it back then) group was apparently drinking a good deal more than the abstinent group, and were, based on their collaterals, being more dishonest about their drinking.

In terms of the N used for long term follow-ups, as a point of comparison, the Match study used by the Cochrane group in their 2020 AA report had, at the three year follow-up, 952 subjects.

To address the rest of that article, to use Dodes’ questionable figures to conclude AA has a low success rate, and then compare that to a figure that, as it turns out, considers 12-step fellowships a part of mainstream treatment to imply AA isn’t “mainstream treatment” is straight up bad science.  [3] The article tries really hard to downplay the results of Cochrane’s 2020 report on Alcoholics Anonymous: After claiming TSF (which, yes, gets people in the rooms of AA where the real recovery starts) isn’t more effective than other treatments, the article finally contradicts itself in the next sentence by conceding that 12-step oriented therapies increase abstinence, but hedges this by stating “the goal of 12-step facilitation therapy is always abstinence, while other therapies may support a goal of controlled drinking or harm reduction for some heavy and dependent drinkers”; never mind that the article wasn’t able to provide a compelling case that controlled drinking is a workable long-term solution for alcoholics (it isn’t, as I already detailed earlier in this post). Did I mention that this article doesn’t even try to address the multiple observational studies showing that regular AA meeting attenders are twice as likely to get and stay sober?

(I originally posted this on Reddit)

[1] “Abstinence or controlled drinking goals for problem drinkers: A randomized clinical trial” 1988

[2] “Individualized behavior therapy for alcoholics: a third year independent double-blind follow-up” 1978

 [3] The science in that article gets worse. The article links to AA’s 2014 membership survey pamphlet stating that, since 27% of people in the survey are in their first year, this shows a 73% relapse rate. Actually, what the pamphlet shows is that 27% of members are in their first year, and 73% have one year or more sober—quite the opposite of a 73% relapse rate. Like I said, bad science.

Some other recent criticism of AA in the press

I have found other criticism of AA in the mainstream press over the last year which I should address here in this blog.

Naltrexone redux

A recent New York Times opinion piece (archive link) brought out the same old tired “AA (or abstinence) doesn’t work but Naltrexone allows alcoholics to drink like gentlemen” chestnut.

While cherry picked studies show remarkably high success rates for Naltrexone, this is offset by other studies which do not replicate those results. Naltrexone, like AA, appears to be effective for a subset of alcoholics, but, as per Cochrane, it only helps about 11% of alcoholics. 

The most disturbing thing about studies showing Naltrexone showing controlled drinking among alcoholics is that they consistently do not have long term follow-up periods. For example, that New York Times opinion piece links to two different studies which supposedly show alcoholics successfully drinking moderately again. In both cases, the final follow-up is one year after treatment.

As someone who remembers Pendery 1982 all too well, positive results for moderate drinking without long term follow-ups are not very encouraging. Sure, anything someone throws a dead cat, they will hit a study of alcoholism showing alcoholics successfully engaging in “moderate drinking” over the short term, but the few long-term follow-ups I have seen show that moderate drinking is not a long-term solution to chronic alcoholism.

As a point of comparison, the 1990s Project MATCH randomized study showed increased abstinence for people who engaged in AA-centered therapies three years after treatment (as per its data as presented in Cochrane 2020), and Moos and Moos 2006 is a longitudinal study showing that Alcoholics Anonymous keeps alcoholics sober 16 years later.

Another thing: That New York Times opinion piece claims that only 25% of alcoholics will achieve abstinence, but the study it links to supporting that outdated 25% figure doesn’t actually conclude this; it claims a 25% success rate for abstinence, and links to an old article from 2005 using results that are nearly two decades old. More recent figures from Cochrane show a 42% success rate.

Abstinence violation effect

There have been two (open access) different letters (paywall) recently published which make the same criticism of Cochrane 2020: They claim that there is an “abstinence violation effect” where the subjects undergoing AA-based treatment who are not abstinent somehow do worse when they drink again than subjects not undergoing AA-based treatment. In other words, the accusation is that AA may increase abstinence, but overall AA doesn’t improve things for alcoholics, because the alcoholics not abstinent supposedly drink more heavily.

This “abstinence violation effect” is actually a myth, as one of the authors of the 2020 Cochrane review on AA explains. Since this letter is paywalled, I will quote the most important part of that letter here:

while more individuals in AA/TSF achieved continuous abstinence, those who were not completely abstinent did not drink more heavily, drink more frequently or experience more alcohol-related consequences

Point being, AA overall improves things for alcoholics. 42% are straight up abstinent from alcohol a year later, and those who do not achieve abstinence do not, repeat not have worse consequences from their drinking.

More on Pendery 1982

Pendery 1982 is a study which saw that, over the long term, alcoholics who try moderate drinking only have a 5% success rate. People who want to moderately drink again, of course, want to invalidate the results of this study. One attempt at doing this is to complain that Pendery 1982 only looked at the 20 alcoholics who tried moderate drinking in the 1970s Sobell study, and did not look at the alcoholics in the abstinence only control group. For example, “Pendery reinterviewed the controlled drinking group to dispute a previous study but didn't make contact with the abstinence group at all [...] If you are inspecting purported results of a study why not investigate all study participants. Why is it OK to investigate only one side” (note that I normally no longer link directly to self-published rants made by an anti-AA pro-drinking proponent, but since they have subsequently deleted their account, I can do so without making the discussion personal).

This is a red herring. If there was a real methodological problem with Pendery 1982, why did Science (one of the most prestigious journals out there) publish it? The abstinent “control” group was not studied because they were not relevant; if a follow-up study can invalidate the experimental condition (as Pendery 1982 did), there is no need to see what happened with the control condition.

Philip Abelson, the editor of Science when Pendery 1982 was studied, had this to say about the process of making that report:

The report that we published in our 9 July issue [Pendery et aI., 1982] was very carefully edited. It was extensively reviewed, including evaluation by an expert statistician. [...] We required that assertions made about patients' histories be documented by court records, police records, hospital records, or affidavits. The final draft was checked repeatedly, sentence by sentence, to ensure that supporting evidence was available. In crucial instances, two or more independent documents corroborated statements made.

Now, let me describe Philip Abelson in more detail: He helped in the research which won the 1951 Nobel Prize in Chemistry. He won the National Medal of Science in 1987. He was part of the Manhattan Project. He was a key researcher in the design of the nuclear submarine. He did a lot of important paleobiology research. He had over 40 years of experience in the scientific technique when he allowed Pendery 1982 to be published. If there was a real problem with the science behind Pendery 1982, he would had found it.

Some history behind Pendery 1982

Let me finish off 2018 by taking about how the seminal work refuting the notion that alcoholics can moderate their drinking again, Pendery 1982, came to be published.

Pendery 1982 (i.e. M.L. Pendery, I.M. Maltzman, and L.J. West. "Controlled Drinking by Alcoholics? New Findings and a Reevaluation of a Major Affirmative Study" Science 217) is a very key paper in the history of alcoholism studies; the release of the paper was the last nail in the coffin that alcoholics can engage in sustained controlled drinking again. It humiliated the controlled drinking for alcoholics proponents; the study undeniably shows that alcoholics who try moderate drinking end up either dead, drinking like an alcoholic, or abstaining from alcohol.

The paper has a very interesting history which can be seen in two articles from 1982, when this very important paper was published:

• Alcoholism study under new attack. This article shows how the 1970s Sobell studies resulted in alcoholism experts from that era believing that alcoholics could learn to drink in a responsible manner, and how Pendery 1982 refuted that notion.
• Showdown nears in feud over alcohol studies. This article shows how hard Pendery and Maltzman had to work to make their paper a reality: “At every step of the way the Sobells have tried to block the investigation by Dr. Pendery and Dr. Maltzman. [...] The Sobells refused to hand over their list of the participants [...] Undaunted by the Sobells' resistance, the Pendery group tracked down a list of the patients' names at a county alcoholism center [...] The Sobells retaliated with a suit to block the use of the names, but a Federal District court dismissed their action in April 1977 [...] It took Dr. Pendery, using thousands of dollars of her own money and whatever time she could squeeze in among her other obligations, several more years to track down patients”

However, despite all this resistance, the paper was published and the notion that alcoholics could control their drinking again was no longer mainstream addiction research. 60 Minutes broadcast a segment describing the Pendery paper; Al Gore, long before he became vice president, wanted to investigate the Sobells for fraud.

Ultimately, the addiction experts who supported Sobell’s point of view would appear to have never let go of Pendery 1982. As recently as 2015, in a poorly researched anti-AA polemic, Glaser inaccurately describes the Sobell study. The Sobell study was not accurate; while an initial 1982 panel felt the Sobells were reasonably accurate in their research, a later 1984 Federal investigation (you know, with subpoena powers) pointed out that the Sobells were “careless in preparing their manuscripts for publication”. Indeed, some of the “controlled drinking” subjects in the study sued the Sobells.

Since people who think alcoholics can drink moderately again bring this up: How the alcoholics given abstinence treatment fared is a red herring. The Sobell study was done in the early 70s, some four decades before we starting finding twelve step facilitation treatments which effectively got more people going to 12-step meetings and abstaining from alcohol.

The fact of the matter is this: With one exception, Pendery 1982 shows us that the supposedly moderately drinking alcoholics in the Sobell study were either dead, engaging in out of control drinking, or were abstaining from alcohol 10 years later.

Moderation for alcoholics: Maybe not

Let’s take a look at this article (original link); this article is an opinion from 2016 on what the science says about Alcoholics Anonymous. First of all, it accurately shows that the 2006 Cochrare review on AA effectiveness is out of date:

Ferri, Amato, and Davoli’s conclusion in a 2006 meta-analysis published in the Cochrane Review [13] has been widely quoted (see e.g., [14]): “No experimental studies unequivocally demonstrated the effectiveness of AA or [Twelve-Step Facilitation] TSF approaches for reducing alcohol dependence or problems” [13].
[...]
Several studies do support some efficacy of TS programs of recovery [15-19]. AA participation is associated with fewer drinks and more abstinent days [15-17], and recent studies show that AA attendance improves sobriety even while controlling for self-selection bias [18]. While these studies do not show unequivocal evidence of success—and are not evidence of sufficient effectiveness to recommend AA/TS programs for everyone—they do support inclusion of TS in the set of appropriate interventions.
[...]
13. Ferri M, Amato L, Davoli M. Alcoholics Anonymous and other 12-step programmes for alcohol dependence. Cochrane Database Syst Rev. 2006;(3):CD005032.
14. Frakt A. What is known about the effectiveness of AA? Incidental Economist. December 20
15. Walitzer KS, Dermen KH, Barrick C. Facilitating involvement in Alcoholics Anonymous during out-patient treatment: a randomized clinical trial. Addiction. 2009;104(3):391-401.
16. Witbrodt J, Ye Y, Bond J, Chi F, Weisner C, Mertens J. Alcohol and drug treatment involvement, 12-step attendance and abstinence: 9-year cross-lagged analysis of adults in an integrated health plan. J Subst Abuse Treat. 2014;46(4):412-419.
17. Moos RH, Moos BS. Participation in treatment and Alcoholics Anonymous: a 16-year follow-up of initially untreated individuals. J Clin Psychol. 2006;62(6):735-750.
18. Humphreys K, Blodgett JC, Wagner TH. Estimating the efficacy of Alcoholics Anonymous without self-selection bias: an instrumental variables re-analysis of randomized clinical trials. Alcohol Clin Exp Res. 2014;38(11):2688-2694.
19. Kaskutas LA. Alcoholics Anonymous effectiveness: faith meets science. J Addict Dis. 2009;28(2):145-157.

So far so good; this shows that AA effectiveness for a subset of alcoholics is pretty much undeniable at this point, and it even links to an experimentatal study (Walitzer 2009) showing this. At this point, the article jumps the shark by giving us this inaccurate whopper without footnotes:

Additionally, TS [Twelve Step] programs promote the goal of abstinence, but moderation is a better goal for some people.

This is a really bold claim, so we would expect about seven footnotes showing research supporting this claim, like we had with the claim that AA helps some alcoholics. But, no, this claim is made without a single footnote.

I will link to two articles which make a strong case this claim is downright false:

• Return to Moderate Drinking is Still a Lie (original article)
• Here’s What Happened When I Tried Moderation Management (original article)

After the disaster that was Sobell 1973, where the moderate drinkers who supposedly did well were either drunk, dead, or abstinent 10 years later, I will not believe any study showing alcoholics moderately drinking again without a 10-year followup. The reason why we need a 10-year followup is because alcoholics are really good at being dishonest about their alcoholic drinking being “moderate”, and it takes up to a decade for the drinking to get so bad the alcoholic can no longer deny their drinking is out of control.

Moderate drinking redux; also more on the Sinclair Method

Let me just post some of my recent disqus postings:

The fact is that the Sobell 1973 study [a study which claimed alcoholics could moderately drink again] was so bad, a fraud investigation was opened. As a 1984 New York Times article put it (look for “Panel finds no fraud by alcohol researchers”):

>>The panel also said it had found ''several errors'' and ''use of ambiguous terminology'' in scientific publications by the Sobells, which indicated that they were ''careless in preparing their manuscripts for publication.'' But the panel said it had seen no evidence that these errors and ambiguities ''represent scientific misconduct - defined as 'fraudulent or highly irregular research practices.' ''<<

Anyway, facts are facts. Those “moderate” drinkers from the Sobell 1973 study were either drinking alcoholically again, or were dead from drinking, or were abstaining from alcohol when a 10-year-followup was performed.

--

Sobell 1973 and the moderate drinkers: Four of those “moderate drinkers” died. Also: Six were abstinent, nine of the “moderate” drinkers were engaging in out of control drinking. There was only one person who, ten years later, could be considered a “moderate” drinker. Please read Pendery 1982 or any of the numerous books which correctly cite it.

Sobell 1973 in comparison to the non-moderate drinkers: This is cherry picking data. The Sobells only compared the fatalities of the “moderate” drinkers with the non-moderate drinkers. I have not seen the abstinent or out-of-control drinking figures for this other group; and it is telling that the Sobells never shared (or, it would seem, even bother to get) those figures.

Since there were only 20 subjects in each study, whether six or four died is an error of approximation, nothing more.

Davies 1962: These “moderate” drinkers also reverted to serious drinking again. Reference: G. Edwards. "D.L. Davies and 'Normal drinking in recovered alcohol addicts': the genesis of a paper" Drug and Alcohol Dependence 35.3 (1994)

Rand report: The 1980 four-year followup shows many of the 1976 Rand report moderate drinkers relapsing.

Moderation management: No peer-reviewed science was ever done with this group, but it’s very telling that its founder drank heavily, to the point police took her to detox in handcuffs while she was still the leader of Moderation Management. She soon afterwards killed two people while driving drunk and eventually killed herself [1].

Point being, pretty much any study showing successful “moderate” drinking, when followed up, shows those same “moderate” drinkers drinking heavily again.

[1] There are claims that the Moderation Management founder was a member of AA at the time of her crash, but as per her autobiography, she did not work the 12-step program in any meaningful way. Instead, she was coming up with excuses for not going to meetings.

--

In response to a claim that the “moderate” drinkers in Sobell 1973 had better success rates than people trying abstinence:

In terms of more recent studies, people who choose to work the AA program have between a 75% (Vaillant 1995) and 67% (Moos and Moos 2006) success rate. The Sobell moderation group had, by comparison, a 30% abstinence success rate and a 5% moderation success rate.

--

In response to a claim that the Sinclair method works better when trying complete abstinence, and works better the longer it is applied:

OK, let me go back to PMC3970823. They use a ”magic number” called “g” which is summarized near the end: “the aggregated Hedges’ g for naltrexone and acamprosate compared to placebo was 0.209 (CI: 0.157 – 0.262) – indicating a small but significant effect. In comparison to other medications prescribed for mental health (e.g., for depression), the effects of naltrexone and acamprosate are somewhat smaller”

So, this information in mind, let’s look at table 2 in PMC3970823. Naltrexone has an abstinence aggregate of .116—which is not very different than a placebo. However, the numbers for “heavy drinking” (.189), craving (.144), and both heavy drinking and craving (.180) are not much better—they are “small but significant.”

It also looks like, when looking at multiple studies, the effect diminishes when doing follow-ups: “naltrexone studies tended to have effect sizes for heavy drinking outcomes at the last follow-up point (g = .135, k = 6) that were slightly smaller compared to end-of-treatment (g = .189, k = 39). Although only two studies provided data, the naltrexone-placebo effect sizes for craving at the last follow-up point were close to zero (g = .053).” In other words, this meta-study didn’t see naltrexone helping people not drink heavily over longer periods.

In light of Pendery 1982, and in light of what happened with the founder of Moderation Management (while she claimed to drink moderately without problem for years, she killed two people while driving drunk, and eventually killed herself), when people suggest moderation as an outcome for alcoholics, I will remain very very skeptical until when and if we have 10-year-followups with very high “g” values and very low “p” values.

The Sinclair method revisited

An advocate for the Sinclair method pointed out, in an online debate, the study “Targeted Naltrexone for Problem Drinkers.”  Here is my response:

That 2011 study (“Targeted Naltrexone for Problem Drinkers”) is a 12-week trial, and the study considers “moderate drinking” a successful outcome. This is a very short trial period, and short-term studies since the 1960s showing alcoholics “successfully” drinking moderately have, if followed up, shown the same drinkers having serious drinking problems five or 10 years later (the most famous example of this is Sobell 1973, showing “moderate” drinking among alcoholics, but Pendery 1982 showed that the same “moderate” drinkers were either dead, abstinent, or drinking heavily a decade later).

The meta-study (a study looking at other studies) “Meta-analysis of naltrexone and acamprosate for treating alcohol use disorders: When are these medications most helpful?” (PMC3970823) points out that “Several reviews and meta-analyses have reported small or mixed effects for each medication (both naltrexone and another medication called acamprosate), and treatment providers cite concerns over efficacy as one barrier to greater medication use.”

This isn’t the first study which showed good results for Naltrexone, but considering the number of other studies which show only a modest effect, and considering the need for a long term follow up to see if an alcoholic is in fact successfully moderately drinking, I will need to see studies with five-year or 10-year follow ups showing continued “moderate” drinking before I will believe this drug is a functional way to let alcoholics control their drinking.

A tale of two papers

I look at two papers in this blog entry, one of which will be questionable, the other of which is well written.

The Alcohol industry is funding a “Moderate Drinking” study

The New York Times has published an exposé last month showing how the NIH solicited contributions from the alcohol insustry for a “Moderate drinking” study.  It would seem that, in order to fund an expensive study on drinking called the “Moderate Alcohol and Cardiovascular Health Trial (M.A.C.H.)” trial, the alcohol industry has spent millions of dollars to make this study a reality.

Whatever finding this study has about moderate drinking are at best suspect, especially for anyone who is a potential alcoholic; the NY Times piece reports that “many people whose health might be compromised by light drinking — anyone with a history of addiction, psychiatric, liver or kidney problems, certain cancers or a family history of breast cancer — will not be allowed to participate. People who have never drunk alcohol also are excluded.”

Knowing how studies are quoted out of context by the kinds of people with an agenda against Alcoholics Anonymous, and knowing how often those kinds of people try to justify moderate drinking for alcoholics, even though that is, as I pointed out before and will point out again, is a fool’s errand, I would not be surprised if those kinds of people, years down the road, use this study to “prove” that Alcoholics Anonymous’ model of abstinence is “harmful.”

A realistic look at Alcoholics Anonymous

Marc Galanter, who helped write a paper showing how prayer helps AA members stay sober, has another excellent paper out there.

Entitled “Combining medically assisted treatment and Twelve-Step programming: a perspective and review”, this paper goes over, among many other things, why Cochrane 2006 had methodological issues in concluding that AA does not work any better than other methods.

A very good paper well worth reading. Nicely enough, this paper is not paywalled.

There is no evidence Audrey Kishline was an active member of AA in 2000

Audrey Kishline was the founder of the group "Moderation Management", which pretended that chronic alcoholics could drink moderately again. Audrey Kishline was never able to drink moderately; in 2000, she got behind the wheel of her truck, and killed two innocent people while driving drunk. 

Soon afterwards, Kishline admitted that "moderation management is nothing but alcoholics covering up their problem."

Audrey Kishline ended up killing herself in late 2014. This was the end result and ultimate consequence of denying she had a drinking problem and deceiving herself and others that she could drink moderately again: Death. Not only her own death, but the death of two innocent bystanders.

One refrain anti-steppers attempt to make when people point out the consequences of Audrey Kishline lying to herself about her moderate drinking, is that she was in AA when she had her fatal drunken driving episode.

There is no evidence that Audrey ever actively worked an AA program before this tragic incident.

The evidence anti steppers bring up is this: http://archive.is/yDrVt where Audrey finally admits she can not drink moderately to the Moderation Management mailing list; she wrote that "I am now following a different path, and to strengthen my sobriety I am attending Alcoholics Anonymous, but will also attend Women for Sobriety and SMART Recovery. I am sure I can learn much from all of these fine programs."

Did Audrey get a sponsor? How often did Audrey go to meetings? Did she work the program as described in the first 164 pages of the Big Book? Was she serious about the program, or was she just trying to look good to others?

The fact that she was considering other programs when she wrote this sentence indicates that she was not serious about AA. I suspect she was just trying to look good to others.

In fact, let's look at an obituary posted after she killed herself to get more details of her life when she drove drunk like that: http://archive.is/SyhFU

When she wrote the email saying that moderation was not working any more, her drinking was so bad, she was forced by law enforcement to go to a detox program. Not a peep in her email about the fact her drinking was so bad, it was giving her legal problems, The email she sent to the listserv was dishonest, so its claim that Audrey was actively going to AA meetings is, at best, suspect.

The only evidence that Audrey went to even a single AA meeting between the time she wrote the email claiming she was going to AA and the time she killed two people driving drunk is an email which we now know was dishonest. 

(Update: The Dateline piece on Audrey's drunk driving has a tiny bit more detail about Audrey's supposed membership in AA between the time she left Moderation Management and that her fateful accident: "The creator of MM was admitting defeat. She checked herself into a detox facility followed up by AA meetings, but she couldn’t play by those rules either.  It did not go unnoticed by her 10-year-old daughter. [Kishline said that] 'I would keep falling off the abstinence wagon.' " Again, no details about the number of AA meetings Audrey went to or whether she actually got serious about working the program)

Update #2: Here is how Audrey Kishline described her year-2000 Alcoholics Anonymous supposed attendance in her autobiography Face to Face:

In the two months that followed [after she wrote the email to the Moderation Management list saying she could not moderate her drinking and before her tragic drunk driving accident] I was supposed to be attending AA meetings, abstaining from drinking, and "working the program." I had huge remorse about having become so drunk and calling the police [This refers to the event where she was forcibly put in detox]. Remorse was my motivation this time to get my act together and quit drinking. But the same old cycle I lived for years began to repeat. I'd abstain for a few days and immediately fall off. Each fall was getting worse and worse. It became increasingly harder for me to remain anonymous too. I was easily recognizable when I attended AA, which only added to my discomfort. I never identified myself as anyone other than "Audrey," but lots of people knew exactly who I was. 

(emphasis in bold like this mine; notes in italics and brackets [like this] added by me to give context)

This is a description of someone who almost never went to AA because they found excuses and justifications not to go, not the story of someone who took AA seriously and worked the 12 steps to the best of their ability. Supposed-to-be-going-to-AA but finding excuses not to go (such as thinking people will recognize and judge them) is not good enough for someone to get sober using the AA program.

A FAQ

A brief FAQ

• 1 Effectiveness of Alcoholics Anonymous
• 2 Other data that is inaccurate
  • 2.1 Randomized controlled trials
  • 2.2 Brandsma 1980
  • 2.3 AA is actually not rated #37 (sometimes #38) compared to other treatments
  • 2.4 5% success rate myth #1: The Sober Truth
  • 2.5 5% success rate myth #2: AA triennial survey
  • 2.6 5% success rate myth #3: Reading Vaillant poorly
• 3 Alcoholics can not moderate their drinking
  • 3.1 How the moderation myth tainted studies about AA
• 4 References

Effectiveness of Alcoholics Anonymous

For the 20%^[1][2][3] or so of alcoholics who become highly engaged with the program, Alcoholics Anonymous has about a 75%^[4][5] effectiveness rate.^[6]
For people who do not take AA as seriously, the number of alcoholics who achieve long-term sobriety is about 20%^[7], with the amount of people achieving sobriety directly correlated to AA participation^[8].
While some of the positive effects of the AA program keeping people sober can be attributed to self-selection bias, one 2014 study makes the case that AA's effectiveness shown in these studies does not come from self-selection^[9]; a 2016 study demonstrates that some of the specific effect is the prayers AA members say.^[10]

Other data that is inaccurate

Randomized controlled trials

Early randomized controlled trials were ineffective because the control group was not really a control group (people who wanted to stay sober went to AA anyway, even if they were not assigned to AA, and people assigned to AA did not necessarily take the program seriously), so their numbers can not be trusted.^[11]

Brandsma 1980

Anti-AA advocates claim that Brandsma 1980^[12] shows increased binge drinking among AA members. However 1) The study looked at about three dozen different findings at multiple follow-up periods, but this result was only seen in this one finding at one follow-up period 2) This result has not been replicated in recent studies^[13] even though 3) The study is 36 years old.^[14]

AA is actually not rated #37 (sometimes #38) compared to other treatments

Another common talking point popular with the anti-AA crowd is a table from Handbook of Alcoholism Treatment Approaches which ranks AA fairly lowly compared to other treatments. That table is, shall we say, weird: It rates acupuncture as one of the top 20 treatments of alcoholism, and gives motivational enhancement (#2 in the table), cognitive therapy (#13), and AA (#37) very different ratings, when the 2006 Cochrane study rate those as being about equal^[15]; other studies rate AA and/or TSF (Twelve Step Facilitation: teaching patients how to go to meetings and be a part of the AA culture) as being superior.^[16] No other reliably published meta-analysis comparing AA to other treatments has rated AA as being inferior.^[17]

5% success rate myth #1: The Sober Truth

Lance Dodes' The Sober Truth is a recent attempt to cook numbers to claim AA has a 5% success rate (for some reason, anti-AAs have a real obsession with coming up with a 5% success rate figure, facts be ignored). While this book was widely reported in the press in 2014, its claimed 5-8% success rate for AA does not come from any peer-reviewed studies, but is instead derived by multiplying numbers from different studies together. Beresford looks at the math Dodes performed to come up with these bogus figures, and explains why the math is invalid at https://www.ncadd.org/blogs/research-update/alcoholics-anonymous-call-for-better-science.

5% success rate myth #2: AA triennial survey

There is a chart in the 1990 AA triennial survey showing that 5% of the members in their first year happened to be in their 12th month, instead of the 8.3% we would expect if AA had a 100% success rate. This has frequently been misread to indicate that only 5% of members of AA stay sober one year. An actual analysis of the graph shows a 26% retention rate.^[3]

5% success rate myth #3: Reading Vaillant poorly

Some claim that Vaillant's book The Natural History of Alcoholism showed a 5% success rate, but that is a myth. The old green-papers site refuted it quite nicely: http://web.archive.org/web/20150329052802/http://green-papers.org/ "95% of patients had relapsed at some time during the study, even though many of these eventually attained sobriety. [...] It's well known that most severe alcoholics only get sober after many relapses, to the extent that relapses can be considered part of the recovery process. So don't pretend that's a failure. [...] And anyway, this was a study of a health network, not AA."

 

Alcoholics can not moderate their drinking

While not directly about AA's effectiveness, there is no reliable scientific evidence that alcoholics can drink moderately again. The studies that claim alcoholics can do so are based on self-reported information from alcoholics over period of a couple years or less. What is happening is that alcoholics have a pattern of being dishonest about their drinking, to the point that surveys using self-reported data from alcoholics are inaccurate. This has been seen time and time again: The Sobell Study from the early 1970s was shown to be a sham in 1982; the Rand report from 1976 was shown to be wrong in the 1980 follow-up; the NESARC study from 2001-2002 was a two-year study which never had a follow-up; and while Moderation Management's effectiveness has never been scientifically analyzed, its founder realized she could not control her drinking, but it was too late: Her runaway drinking resulted in her killing two people while driving drunk; she eventually killed herself.^[18]

How the moderation myth tainted studies about AA

A couple of surveys from the 1970s and early 1980s erroneously reported that AA makes people who drink again have worse relapses. The reason why these surveys were inaccurate is probably because they were based on self-reported data. Since people in AA are less likely to be dishonest about their drinking, they will self-report any drinking they engage in as being more chronic, while people not in AA are more likely to dishonestly report their drinking as being "moderate" or "under control" (e.g. lying about the number of drinks they have when filling out the survey).

References

1. The Natural History of Alcoholism Revisited by George E. Vaillant has a table showing that 19 out of 100 alcoholics attended 300 or more meetings over a 10-year period. The table is on page 197, and can be downloaded from http://web.archive.org/web/20150329052802/http://www.green-papers.org/Vaillant-Nat_hist_of_alcoholism_revisited-excerpts.pdf
2. This article shows a roughly 80% drop out rate in Brazil: Terra, Mauro Barbosa; Barros, Helena Maria Tannhauser; Stein, Airton Tetelbom; Figueira, Ivan; Palermo, Luiz Henrique; Athayde, Luciana Dias; Gonçalves, Marcelo de Souza; Da Silveira, Dartiu Xavier (2008). "Do Alcoholics Anonymous Groups Really Work? Factors of Adherence in a Brazilian Sample of Hospitalized Alcohol Dependents". American Journal on Addictions 17 (1): 48–53. doi:10.1080/10550490701756393. PMID 18214722.  "AA adherence was below 20%"
3. http://hindsfoot.org/recout01.pdf, while independently published and not peer-reviewed, performs statistical analysis of the unpublished 1990 triennial survey which anti-AA advocates frequently quote (since unsophisticated people reading it have incorrectly concluded it shows a 5% retention rate among AA members), showing that 26% of members surveyed make it from their first month to their twelfth month. If you insist on these figures in a more mainstream source, Dodes parrots that 26% figure in The Sober Truth.
4. That table in The Natural History of Alcoholism Revisited has, of those 19 alcoholics who took AA seriously, 14 who achieved long-term sobriety. While the actual 14 people figure is not on this table, 29 people * 48% is 14.
5. Beresford, Thomas (2016), Alcoholics Anonymous and The Atlantic: A Call For Better Science, National Council on Alcoholism and Drug Dependence, retrieved 2016-04-15  This page refers to Fiorentine 1999, pointing out that "the data Fiorentine presents is as follows: 77.7% of individuals who attended AA 12-step meetings at least weekly reported being free of drug use for 6 months prior to a 24-month follow-up, a finding corroborated by urinalysis at the time of the interview, and 74.8 % reported being free of alcohol use during the same time period"
6. Another study, which measures something slightly different, has similar high success figures for people who were highly engaged with Alcoholics Anonymous: Moos, Rudolf H.; Moos, BS (June 2006). "Participation in Treatment and Alcoholics Anonymous: A 16-Year Follow-Up of Initially Untreated Individuals". Journal of Clinical Psychology 62 (6): 735–750. doi:10.1002/jclp.20259. PMC 2220012. PMID 16538654. 
7. Of the 81 alcoholic subjects in The Natural History of Alcoholism Revisited who did not attend 300 AA meetings over a 10-year period, 15 achieved long-term sobriety
8. See the graphs in Kaskutas, Lee Ann (2009). "Alcoholics Anonymous Effectiveness: Faith Meets Science". Journal of Addictive Diseases 28 (2): 145–157. doi:10.1080/10550880902772464. PMC 2746426. PMID 19340677. 
9. Humphreys, Blodgett, Wagner (2014). "Estimating the efficacy of Alcoholics Anonymous without self-selection bias: an instrumental variables re-analysis of randomized clinical trials". Alcoholism: Clinical and Experimental Research 38 (11): 2688–94. doi:10.1111/acer.12557. PMC 4285560. PMID 25421504.  CS1 maint: Multiple names: authors list (link)
10. Marc Galanter, Zoran Josipovic, Helen Dermatis, Jochen Weber, and Mary Alice Millard (2016-03-25). "An initial fMRI study on neural correlates of prayer in members of Alcoholics Anonymous". The American Journal of Drug and Alcohol Abuse. doi:10.3109/00952990.2016.1141912. Lay summary. 
11. There is some discussion of this in Kaskutas, Lee Ann (2009). "Alcoholics Anonymous Effectiveness: Faith Meets Science". Journal of Addictive Diseases 28 (2): 145–157. doi:10.1080/10550880902772464. PMC 2746426. PMID 19340677. 
12. Here's the proper citation for Brandsma 1980 Brandsma, Jeffery M; Maultsby, Maxie C; Welsh, Richard J (1980). Outpatient treatment of alcoholism: a review and comparative study. Baltimore, MD: University Park Press. ISBN 0-8391-1393-5. OCLC 5219646. 
13. While I haven't seen this cited by an anti-stepper in the wild, I should note this: Ogborne AC, Bornet A (1982). "Abstinence and abusive drinking among affiliates of Alcoholics Anonymous: are these the only alternatives?". Addict Behav. PMID 7102452.  This study, like Brandsma 1980, is a museum piece; the study uses data from the "Rand report", which is available for download and a table showing AA effectiveness there is on top of page 98 (page 114 in the PDF), showing fewer AA members who engage in "normal drinking." The notion that the Rand report showed alcoholics drinking normally again has since been discredited, see the blog entry at https://elplatt.com/return-moderate-drinking-still-lie for discussion; what that 1982 paper probably saw was that AA members are less likely to report their drinking as "normal drinking" and more likely to be honest about the negative consequences of their drinking
14. While not a reliable source, the best discussion I have found about Brandsma 1980 is at http://slatestarcodex.com/2014/10/26/alcoholics-anonymous-much-more-than-you-wanted-to-know/, in the beginning of section III. Note that this blog dismisses longitudinal studies too easily, but, then again, it was published before Humphreys, Blodgett, Wagner 2014 demonstrated a specific effect for AA, and before Galanter 2016 showed that prayer is part of that specific effect.
15. Ferri, Marcia; Amato, Laura; Davoli, Marina (19 July 2006). "Alcoholics Anonymous and other 12-step programmes for alcohol dependence". Cochrane Database of Systematic Reviews (3): CD005032. doi:10.1002/14651858.CD005032.pub2. PMID 16856072. 
16. Humphreys, Keith; Moos, Rudolf (May 2001). "Can encouraging substance abuse patients to participate in self-help groups reduce demand for health care? A quasi-experimental study". Alcoholism: Clinical and Experimental Research 1 (5): 711–716. doi:10.1111/j.1530-0277.2001.tb02271.x. PMID 11371720. 12-step patients had higher rates of abstinence at follow-up (45.7% versus 36.2% for patients from CB [cognitive-behavioral] programs, p < 0.001 
17. While not published by a mainstream press nor peer-reviewed, the blog entry at http://slatestarcodex.com/2014/10/26/alcoholics-anonymous-much-more-than-you-wanted-to-know/ devotes an entire section to this table and why it is inaccurate. Scroll down to section VI.
18. While, yes, a blog, a good discussion with references of how alcoholics can not moderate their drinking and why any study claiming otherwise is wrong is at the blog entry https://elplatt.com/return-moderate-drinking-still-lie

Glaser got the Sobell story wrong

I am not about to read, much less point by point refute, all of Gabrielle Glaser's over 8000 word anti-AA screed published in The Atlantic this month.  However, the 300 or so words she writes about the Sobell study and its aftermath are inaccurate.  For me to properly comment on it, I need to give readers here the entire content of what Glaser wrote on that Sobell study:

To many, though, the idea of non-abstinent recovery is anathema.

No one knows that better than Mark and Linda Sobell, who are both psychologists. In the 1970s, the couple conducted a study with a group of 20 patients in Southern California who had been diagnosed with alcohol dependence. Over the course of 17 sessions, they taught the patients how to identify their triggers, how to refuse drinks, and other strategies to help them drink safely. In a follow-up study two years later, the patients had fewer days of heavy drinking, and more days of no drinking, than did a group of 20 alcohol-dependent patients who were told to abstain from drinking entirely. (Both groups were given a standard hospital treatment, which included group therapy, AA meetings, and medications.) The Sobells published their findings in peer-reviewed journals.

In 1980, the University of Toronto recruited the couple to conduct research at its prestigious Addiction Research Foundation. “We didn’t set out to challenge tradition,” Mark Sobell told me. “We just set out to do good research.” Not everyone saw it that way. In 1982, abstinence-only proponents attacked the Sobells in the journal Science; one of the writers, a UCLA psychologist named Irving Maltzman, later accused them of faking their results. The Science article received widespread attention, including a story in The New York Times and a segment on 60 Minutes.

Over the next several years, four panels of investigators in the United States and Canada cleared the couple of the accusations. Their studies were accurate. But the exonerations had scant impact, Mark Sobell said: “Maybe a paragraph on page 14” of the newspaper.

(Emphasis mine) 

I watched that 60 Minutes segment when it came out. It helped save my life. It is very telling that Glaser does not describe what that segment said. Glaser gives the false impression that the critics merely had issues with the Sobells’s honesty. What Glaser is not telling her readers is what happened to those 20 people who the Sobells tried controlled drinking with: Four died drunk. Eight – make that nine – were engaging in out of control drinking. Six were abstaining from alcohol altogether. Only one of those 20 people in the Sobell study was able to achieve long-term controlled drinking.

The panel who investigated the Sobells merely found that the Sobells did not commit fraud; the contemporary New York Times story describing the report points out that “The Federal panel found that the Sobells overstated their success in one passage of a 1978 book” and that
“The panel also found that the Sobells made incorrect statements about the number of times they contacted patients.”

That is a far cry from Glaser’s assertion that “[The Sobells’s] studies were accurate.” The panel made so such conclusion, and Glaser should not be implying that they did. There’s a world of difference between not deliberately lying and making an accurate survey.

This is not the first time I’ve seen Glaser give the wrong impression. In a previous hit piece attacking the 12-step programs, she implied that Joanne Fry was the sponsor for Karla Brada, who, like many alcoholics, died drunk. Joanne Fry has since flat out denied ever being Karla’s sponsor, and Glaser never directly stated that Joanne was her sponsor, but tried to imply it.