More on Pendery 1982

Pendery 1982 is a study which saw that, over the long term, alcoholics who try moderate drinking only have a 5% success rate. People who want to moderately drink again, of course, want to invalidate the results of this study. One attempt at doing this is to complain that Pendery 1982 only looked at the 20 alcoholics who tried moderate drinking in the 1970s Sobell study, and did not look at the alcoholics in the abstinence only control group. For example, “Pendery reinterviewed the controlled drinking group to dispute a previous study but didn't make contact with the abstinence group at all [...] If you are inspecting purported results of a study why not investigate all study participants. Why is it OK to investigate only one side” (note that I normally no longer link directly to self-published rants made by an anti-AA pro-drinking proponent, but since they have subsequently deleted their account, I can do so without making the discussion personal).

This is a red herring. If there was a real methodological problem with Pendery 1982, why did Science (one of the most prestigious journals out there) publish it? The abstinent “control” group was not studied because they were not relevant; if a follow-up study can invalidate the experimental condition (as Pendery 1982 did), there is no need to see what happened with the control condition.

Philip Abelson, the editor of Science when Pendery 1982 was studied, had this to say about the process of making that report:

The report that we published in our 9 July issue [Pendery et aI., 1982] was very carefully edited. It was extensively reviewed, including evaluation by an expert statistician. [...] We required that assertions made about patients' histories be documented by court records, police records, hospital records, or affidavits. The final draft was checked repeatedly, sentence by sentence, to ensure that supporting evidence was available. In crucial instances, two or more independent documents corroborated statements made.

Now, let me describe Philip Abelson in more detail: He helped in the research which won the 1951 Nobel Prize in Chemistry. He won the National Medal of Science in 1987. He was part of the Manhattan Project. He was a key researcher in the design of the nuclear submarine. He did a lot of important paleobiology research. He had over 40 years of experience in the scientific technique when he allowed Pendery 1982 to be published. If there was a real problem with the science behind Pendery 1982, he would had found it.

Herbert Fingarette; Patheos again

Herbert Fingarette, who died last year, was someone who was always opposed to Alcoholics Anonymous and the disease theory of Alcoholism, and believed the lie that alcoholics can moderately drink again. Indeed, Lance Dodes did some creative math with Fingarette’s numbers when concluding Alcoholics Anonymous has a 5% success rate:

University of California professor Herbert Fingarette cited two [...] statistics: at eighteen months, 25 percent of people still attended AA, and of those who did attend, 22 percent consistently maintained sobriety. [Reference: H. Fingarette, Heavy Drinking: The Myth of Alcoholism as a Disease (Berkeley: University of California Press, 1988)] Taken together, these numbers show that about 5.5 percent of all those who started with AA became sober members.

The New York Times points out that Fingarette never did any alcoholism science himself:

Professor Fingarette acknowledged that he had not conducted any experimental or clinical studies into alcoholism; he reached his conclusions by analyzing scientific literature

Point being, not only did not Dodes multiply unrelated numbers to synthesize an artificially low number for AA’s success, but he used figures from an author who himself never published an alcoholism study.

Patheos

Patheos is one of the few places on the Internet where the same old debunked myths about AA having a “5% success rate” keep coming up again and again (the other two big offenders are Reddit, which has little regard for the actual truth but prefers a good fictional story, and TheFix). I posted this refutation to those old discredited numbers in a recent article there:

AA has, among those who are heavily involved with it, a 75% success rate over the short term, and a 67% rate over the long term (16 years later). These numbers have been consistent among multiple longitudinal studies, and are the same numbers AA members observed in the mid-1950s. To name just one study showing a high success rate among regular AA members, please see PMC2220012 (Moos and Moos 2006)

Indeed, the Surgeon General, in 2016 (during the Obama administration), stated that “Well-supported scientific evidence demonstrates the effectiveness of twelve-step mutual aid groups focused on alcohol and twelve-step facilitation interventions.”

Lance Dodes’s numbers concluding that AA has a 5% success rate are very dubious. Quite frankly, Dodes pulled the numbers out of his rear end: He multiplied unrelated numbers from multiple studies together to synthesize an artificially low number for AA’s success. His figures were so bad, multiple highly regarded treatment experts have criticized him, with one calling his reasoning a “pseudostatistical polemic”

People are welcome to their opinions, but not to their facts. The fact is that AA works among a significant subset of alcoholics (and, yes, atheists, there are agnostic AA meetings, or just use SMART/LifeRing/whatever, which appear to be about as effective as long as abstinence is the goal). Denying facts is what we accuse the superstitious of doing; it would be best if we did not do the same thing.

The old anti-AA talking points are being posted less

There have been very few articles posted by a major media outlet parroting false figures for AA efficacy since 2015, with Gabrielle Glaser’s highly biased and inaccurate article The Irrationality of Alcoholics Anonymous.  That article, which has been refuted in the mainstream press (see “Why Alcoholics Anonymous Works” by Jesse Signal), is the last significant article criticizing AA’s efficacy. The only other articles more recent than that parroting the same inaccurate numbers are a 2018 Wired UK article by Stephen Armstrong (“The most popular treatment for alcoholism might be ineffective”) and a late 2018 article by a grade-B free newspaper called “The Stranger” entitled “This Drug Could Change Alcohol Addiction Treatment Forever” (there was also an opinion piece, not article, published in USA Today entitled “It took a village to kill my brother: How families, hospitals and government fail alcoholics” critical of AA’s success).

On the other hand, there have been a number of articles praising AA’s efficacy: Here’s a 2014 article; there is also a 2016 Forbes Article, a 2019 article published in a magazine for medical students praising AA’s success, and a 2018 Vox piece with a balanced view of AA’s real success.

The point being, mainstream media is more positive than negative towards AA as the 2010s end.

Cochrane 2018: Not relevant

There is a 2018 Cochrane report which claims that “We found that the talking therapies led to no differences, or only small differences, for the outcomes assessed.” This report, which is an update to a readily available 2013 report, only mentions 12-step programs in a single old study from 1998.

This Cochrane report ignores the recent studies showing AA efficacy, since they are recent and look only at how AA reduces alcohol drinking. Point being, there hasn’t been a Cochrane review which looks at how well AA reduces drinking since 2006, but there have been multiple experimental studies showing that AA (or TSF which gets people in the AA rooms) reducing alcohol abuse published since then.

I question the neutrality of the Cochrane committee with regards to effective spiritual treatments like the program; they are willing to update for 2018 a report which does not show efficacy for 12-step programs, but they are not updating their outdated 2006 report on Alcoholics Anonymous now that there are multiple experimental studies and a meta-study which scientifically demonstrate the efficacy of the steps.

Good to see the word is getting out

Reddit posted yet another repost of that old 2015 article from The Atlantic about how AA supposedly has a 5% success rate. However, this time someone actually researched the number and posted the following:

From Dodes' book " research indicates that only 5 to 8 percent of the people who go to one or more AA meetings achieve sobriety for longer than one year."
https://en.wikipedia.org/wiki/Effectiveness_of_Alcoholics_Anonymous#The_Sober_Truth
But that number is controversial (and the above wikipedia article gives a good overview of the conflicting views).

Point being, all of the effort I have gone to over the years to find accurate information about AA’s success and making the Wikipedia article on the topic reflect what science actually knows is paying off; people can no longer claim “AA has only a 5% success rate” without the claim being questioned.

The mid-2010s situation where a single doctor, without quoting any actual study, makes up a 5% success rate figure for AA by multiplying numbers from different studies together and not being questioned are behind us. Now, anyone interested in facts can and does find out that Dodes’s numbers were garbage.

AA has a 67-75% success rate among people who regularly go to meetings.  This is the number that science shows us; this number has been replicated across multiple studies and has been consistent since the mid-1950s.

Review: “US of AA” by Joe Miller

I will review “US of AA: How the Twelve Steps Hijacked the Science of Alcoholism” by Joe Miller. Since this blog focuses on the efficacy of AA (67-75%), I will look at how this book looks at AA’s success. To quote the book:

Perhaps the most widely known statistic showing AA's ineffectiveness relative to other treatment methods is the 5 percent success rate -- that served as a centerpiece statistic in The Sober Truth, a 2014 expose by former Harvard psychiatry professor Lance Dodes of the bad science behind AA. In fact, he gathered the statistic from AA’s own surveys. Every three years since 1968 AA has randomly queried several thousands of its members for basic info -- age, career, gender, how they came to AA, and length of sobriety. One of the questions AA asks is the month and year when they first came into AA. In 1990, the AA member who analyzed the results used the data to "show the probability that a member will remain in the Fellowship a given number of months." He calculated that out of every hundred people who came into AA, eighty leave within a month. At the three-month mark, only ten remain. At one year, that number has dwindled to five.

Seems straightforward: AA fails ninety-five percent of the people who come in the door.

In other words, the author has completely disregarded any real research on AA’s success done in the last decade. Indeed, Miller did not correctly read The Sober Truth, since Dodes does concede that that old 1990 AA survey shows a 26% retention rate. Dodes’s book uses another trick to make up the 5% success rate figure, which is also inaccurate (he multiplies numbers from unrelated studies to cook up that artificial 5% figure).

If Miller can not get basic facts about AA’s success rate correct, I see no need to waste my time reading any more of this poorly researched book.

Fortunately, this book did not get the level of press that Dodes’s poorly argued 2014 polemic got.

Another article with a reasonable look at AA

Here is an article, written by a medical student, which looks at Alcoholics Anonymous (AA) from the viewpoint of a doctor who needs to understand the program better. Not only does it have a favorable viewpoint about AA, it links to and summarizes scientific studies showing the AA works for most people who make a regular habit of going to meetings:

http://in-training.org/sitting-in-the-circle-what-a-third-year-medical-student-learned-from-alcoholics-anonymous-16951

Well worth a read.

As one Reddit user put it

Here is a Reddit post which nicely summarizes the “AA does not work” nonsense out there:

Partial truth & twisting of facts is however representative of those with a bias and agenda against 12 Step recovery modalities. They expect the gullible to just accept their reporting as the entire truth and not take the time to check out the actual facts. Those with a predilection against 12 step or have a perceived resentment of course will jump on the partial truths and outright lies of those with a similar bias.

How else can you describe a mindset where people believe AA has a 5% success rate even though no peer reviewed study has ever concluded this. Instead, the studies show a 67%-75% success rate among alcoholics who get serious about the program; people in denial, however, don’t let facts gets in the way of their flawed world view.

AA has a 67-75% success rate

Critics of the 12-step programs attempt to discredit the longitudinal studies, which show a 67-75% success rate for AA:

• They don’t like the fact that those high success rate numbers only come from people who actively go to AA meetings.  However, this is consistent with AA’s own claim that “Rarely have we seen a person fail who has thoroughly followed our path” (emphasis mine)
• They claim the numbers are invalid because of “self selection bias”; in other words, they believe the people who are clean and sober because of AA would have gotten clean and sober anyway in an alternate universe where AA doesn’t exist. A 2014 paper demonstrates this actually isn’t the case.
• They complain that the studies do not include dropouts.  However, the drop out rate is small enough to not significantly skew the numbers, especially since, most of the time, the dropouts undoubtedly 1) Didn’t work the AA program and 2) Didn’t get sober. 
• They make other complaints about the methodology of the studies. However, this ignores the fact that these numbers have been pretty consistent across multiple longitudinal studies. Fiorentine 1999: 74.8% success rate. Moos and Moos 2006: 67% success rate 16 years later. The Natural History of Alcoholism Revisited by George E. Vaillant has a table on page 197 showing that 19 out of 100 alcoholics attended 300 or more meetings over a 10-year period, of those 19, 14 were sober: 74% success rate. Witbrodt 2012 saw similar numbers: “high 12-step attendance (abstinence rates averaging 75%)”. AA’s own Big Book has the same abstinence rate using 1955 numbers: “Of alcoholics who came to A.A. and really tried, 50% got sober at once and remained that way; 25% sobered up after some relapses”

The fact is this: We have multiple studies showing the same number: 75% success rate among people who keep coming back to AA. The science is undeniable: Meeting makers make it.
 

Brandsma 1980: The AA treatment

The 1980 book Outpatient Treatment of Alcoholism (Brandsma 1980) has been quoted by anti-AA critics as showing AA as being ineffective.  In this post, I will look at the “AA” treatment which the subjects of the book underwent. Here is how the book describes it (“SHARP” here is an office the Brandsma research team set up for conducting their research; they tried to get people in the community to attend, but it ended up that most people got treatment there because the court mandated them to go):

An AA treatment group was started at SHARP by contacting the local AA organization. They sent two volunteers, a man (with a B.S.) and a woman (with a high school education), to begin our new group. They had 14 and 10 years experience in AA, respectively. The AA group was open to all who would attend and was conducted by one or the other of these people for approximately 1 year until January, 1973. By that time (the beginning of the research in terms of assigning patients), a member of the group (not included in the research project) had become very adept at running meetings. This minority group member, who had a 9th grade education, had had no AA experience before modeling himself after the two initial leaders of this group. This person remained the leader for the rest of the time of the research project (with a 3-week hiatus when an experienced AA counselor filled in), and continues to be involved in AA today. He received no remuneration for this, but was very cooperative in providing attendance reports for the project.

The “AA” treatment is described in more detail in another section of the book:

Our group met once per week and tended to focus on persons present for the first time, but not to the exclusion of other members. The 12 steps of AA were used as the content focus for discussions. The men were made aware of the availability of “sponsor-friends” and were encouraged to ally themselves with one or more of their own choosing. Generally our patients seems to neglect availing themselves of this opportunity, and the assignment of sponsors was not followed up on in this group. Beyond this description we are unable to be more specific. [...]

The nature of scientific investigation did force some small changes in the usual AA norms. First, most of our clients came coerced from the court system and all were then randomly assigned to treatments. Thus they were not “free to choose” whether to have treatment or the type of treatment. However, a person’s first visit to AA usually results from some form of “coercion,” e.g. a wife or boss. Second, participation was not anonymous because we kept attendance records, although this was an unobtrusive procedure. If a man’s attendance was poor or nonexistent, he would be contacted at least once by a project social worker, reminded of the conditions of his parole, and encouraged to return. This type of “coercion” (in contrast to the “buddy system”) is different than usual AA procedures. Our group did not develop a high degree of cohesiveness or a formalized buddy system as some do, nor did we offer other “services,” as does Al-Anon. However, given our population, situation, and purposes, we believe this was a representative treatment effort within the AA framework.

So, as we can see, almost no one in the Brandsma treatment got a sponsor, everyone in the group was a “court card” attender, and, even worse, they would get a threatening phone call if they didn’t show up at the meeting.

The Brandsma 1980 “AA” treatment (and, yes, Kaskutas 2009 is slightly inaccurate in her description of the “AA” treatment offered in the Brandsma study; the meetings were open to the public, but it’s unknown whether the meetings were listed in a local AA directory) does not sound like a meeting where people were effectively working the AA program. Kaskutas’s concern that “a potentially important therapeutic ingredient of AA—the experience of longer-term members—would not have been present in the AA condition” is spot on: The leader of the meeting had barely a year sober and the other attendees were “court slip” members.

Based on the description of the “AA” treatment, any conclusions Brandsma 1980 makes about AA efficacy are suspect.