Brandsma 1980: The AA treatment

The 1980 book Outpatient Treatment of Alcoholism (Brandsma 1980) has been quoted by anti-AA critics as showing AA as being ineffective.  In this post, I will look at the “AA” treatment which the subjects of the book underwent. Here is how the book describes it (“SHARP” here is an office the Brandsma research team set up for conducting their research; they tried to get people in the community to attend, but it ended up that most people got treatment there because the court mandated them to go):

An AA treatment group was started at SHARP by contacting the local AA organization. They sent two volunteers, a man (with a B.S.) and a woman (with a high school education), to begin our new group. They had 14 and 10 years experience in AA, respectively. The AA group was open to all who would attend and was conducted by one or the other of these people for approximately 1 year until January, 1973. By that time (the beginning of the research in terms of assigning patients), a member of the group (not included in the research project) had become very adept at running meetings. This minority group member, who had a 9th grade education, had had no AA experience before modeling himself after the two initial leaders of this group. This person remained the leader for the rest of the time of the research project (with a 3-week hiatus when an experienced AA counselor filled in), and continues to be involved in AA today. He received no remuneration for this, but was very cooperative in providing attendance reports for the project.

The “AA” treatment is described in more detail in another section of the book:

Our group met once per week and tended to focus on persons present for the first time, but not to the exclusion of other members. The 12 steps of AA were used as the content focus for discussions. The men were made aware of the availability of “sponsor-friends” and were encouraged to ally themselves with one or more of their own choosing. Generally our patients seems to neglect availing themselves of this opportunity, and the assignment of sponsors was not followed up on in this group. Beyond this description we are unable to be more specific. [...]

The nature of scientific investigation did force some small changes in the usual AA norms. First, most of our clients came coerced from the court system and all were then randomly assigned to treatments. Thus they were not “free to choose” whether to have treatment or the type of treatment. However, a person’s first visit to AA usually results from some form of “coercion,” e.g. a wife or boss. Second, participation was not anonymous because we kept attendance records, although this was an unobtrusive procedure. If a man’s attendance was poor or nonexistent, he would be contacted at least once by a project social worker, reminded of the conditions of his parole, and encouraged to return. This type of “coercion” (in contrast to the “buddy system”) is different than usual AA procedures. Our group did not develop a high degree of cohesiveness or a formalized buddy system as some do, nor did we offer other “services,” as does Al-Anon. However, given our population, situation, and purposes, we believe this was a representative treatment effort within the AA framework.

So, as we can see, almost no one in the Brandsma treatment got a sponsor, everyone in the group was a “court card” attender, and, even worse, they would get a threatening phone call if they didn’t show up at the meeting.

The Brandsma 1980 “AA” treatment (and, yes, Kaskutas 2009 is slightly inaccurate in her description of the “AA” treatment offered in the Brandsma study; the meetings were open to the public, but it’s unknown whether the meetings were listed in a local AA directory) does not sound like a meeting where people were effectively working the AA program. Kaskutas’s concern that “a potentially important therapeutic ingredient of AA—the experience of longer-term members—would not have been present in the AA condition” is spot on: The leader of the meeting had barely a year sober and the other attendees were “court slip” members.

Based on the description of the “AA” treatment, any conclusions Brandsma 1980 makes about AA efficacy are suspect.

Some history behind Pendery 1982

Let me finish off 2018 by taking about how the seminal work refuting the notion that alcoholics can moderate their drinking again, Pendery 1982, came to be published.

Pendery 1982 (i.e. M.L. Pendery, I.M. Maltzman, and L.J. West. "Controlled Drinking by Alcoholics? New Findings and a Reevaluation of a Major Affirmative Study" Science 217) is a very key paper in the history of alcoholism studies; the release of the paper was the last nail in the coffin that alcoholics can engage in sustained controlled drinking again. It humiliated the controlled drinking for alcoholics proponents; the study undeniably shows that alcoholics who try moderate drinking end up either dead, drinking like an alcoholic, or abstaining from alcohol.

The paper has a very interesting history which can be seen in two articles from 1982, when this very important paper was published:

• Alcoholism study under new attack. This article shows how the 1970s Sobell studies resulted in alcoholism experts from that era believing that alcoholics could learn to drink in a responsible manner, and how Pendery 1982 refuted that notion.
• Showdown nears in feud over alcohol studies. This article shows how hard Pendery and Maltzman had to work to make their paper a reality: “At every step of the way the Sobells have tried to block the investigation by Dr. Pendery and Dr. Maltzman. [...] The Sobells refused to hand over their list of the participants [...] Undaunted by the Sobells' resistance, the Pendery group tracked down a list of the patients' names at a county alcoholism center [...] The Sobells retaliated with a suit to block the use of the names, but a Federal District court dismissed their action in April 1977 [...] It took Dr. Pendery, using thousands of dollars of her own money and whatever time she could squeeze in among her other obligations, several more years to track down patients”

However, despite all this resistance, the paper was published and the notion that alcoholics could control their drinking again was no longer mainstream addiction research. 60 Minutes broadcast a segment describing the Pendery paper; Al Gore, long before he became vice president, wanted to investigate the Sobells for fraud.

Ultimately, the addiction experts who supported Sobell’s point of view would appear to have never let go of Pendery 1982. As recently as 2015, in a poorly researched anti-AA polemic, Glaser inaccurately describes the Sobell study. The Sobell study was not accurate; while an initial 1982 panel felt the Sobells were reasonably accurate in their research, a later 1984 Federal investigation (you know, with subpoena powers) pointed out that the Sobells were “careless in preparing their manuscripts for publication”. Indeed, some of the “controlled drinking” subjects in the study sued the Sobells.

Since people who think alcoholics can drink moderately again bring this up: How the alcoholics given abstinence treatment fared is a red herring. The Sobell study was done in the early 70s, some four decades before we starting finding twelve step facilitation treatments which effectively got more people going to 12-step meetings and abstaining from alcohol.

The fact of the matter is this: With one exception, Pendery 1982 shows us that the supposedly moderately drinking alcoholics in the Sobell study were either dead, engaging in out of control drinking, or were abstaining from alcohol 10 years later.

More people are realizing that AA is effective

Just yesterday, someone wrote an article for a small regional magazine (which comes off as like those magazines which were free on every street corner back in the days before the Internet took over publishing) regurgitation the same old “AA only has a 5% success rate” nonsense which has long been refuted.

However, instead of the comments agreeing with the bogus 5% success figures, the comments pointed out how inaccurate those figures are. At the beginning of this year, with the exact same online magazine, I was the only commentator with accurate figures for AA’s success.

This indicates that the tide is really starting to turn; the days of being able to post an article claiming AA has only a 5% success rate without getting devoured in the comments section are behind us.

Charlie Sheen, clean and sober in AA

Charlie Sheen, back in 2011, claimed that Alcoholics Anonymous has only a 5% success rate. Thankfully, over the years, he realized that those numbers are questionable.

He now has one year clean and sober in Alcoholics Anonymous. I'm very happy that he was able to let go of outdated ideas like the idea that science shows AA is ineffective (it doesn't), and instead use the program to become clean and sober.

What is AA's real success rate?

The question is this: What is AA's real success rate? We can never know the true number for two key reasons:

• AA does not keep track of members.
• "AA success" is a poorly defined term, which can have several actual meanings

While AA itself did not keep track of members, Moos and Moos 2006 did. It found the following: Of the people who were serious about AA, 67% of them were sober 16 years later.

But what about the dropouts? Not everyone who did the study did the 16-year followup, so we can't know the true success rate number, but looking at only the people who did follow up gives us a reasonable number (Dodes 2014 claims that Moos/Moos 2006 has completely invalid numbers because of these dropouts, but Dodes appears to have had no interest in an honest assessment of AA's success). Overall, 46% of the people in the Moos and Moos 2006 survey (again, only those who did the 16-year followup) were sober, regardless of their level of AA attendance.

Now we are getting in to the realm of philosophical questions: Do we count someone who was not serious about AA but still stayed sober as a success or failure for AA? Should we count people who weren't serious about meetings when calculating AA's success rate? Polemics like Dodes's The Sober Truth consider every single person who was not serious about AA a failure when making artificially low numbers for AA's success, even if the person got sober after all.

There are a lot of ways we can come up with a "success rate" number based on Moos and Moos 2006:

• There were 461 people in the survey who did a 16-year followup
• Of those, 25% (115) took the AA program seriously. Of those 115, 77 (67% of serious AA members, 17% overall) were sober, and 38 (33% of serious AA members, 8.2% overall) were not sober. We could use these numbers to argue that AA has "only" a 17% success rate, but that argument is dishonest because it discounts the 16% of people (72) who had some level of exposure to AA and still stayed sober.
• Overall, 214 (46%) got sober.  Since pretty much any alcoholic out there at least knows of AA's existence, we could make a case this is a 46% success rate.
• If we look at just people who had a least a little bit of AA treatment, this is 58% (269 of 461 total) of the people in the survey; of those 55% (149) were sober 16 years later. Contrast this with the 42% (192) who didn't try AA at all, 34% or 14% overall (65) were sober 16 years later.

Point being, there are a lot of ways to measure AA success, but to consider people who didn't take AA seriously but still got sober as "failures" (as Dodes and others have done) is downright dishonest.

Alcoholics Anonymous's own success claims
 
Alcoholics Anonymous has, since 1939, made the following claim about AA's success "Rarely have we seen a person fail who has thoroughly followed our path." This claim was expanded in the mid-1950s: "Of alcoholics who came to A.A. and really tried, 50% got sober at once and remained that way; 25% sobered up after some relapses, and among the remainder, those who stayed on with A.A. showed improvement."

Alcoholics Anonymous has never claimed it would help people who did not take the program seriously. Any claim about AA's success should only look at the people who take AA seriously; once we discount the people who did not take AA seriously, the numbers we get are in line with the numbers AA gave us in the mid-1950s (75% success rate among those whose who "really tried" AA).

It is true that AA has a high drop out rate, so I contend this: Have we found a treatment which is effective for alcoholics who, for whatever reason, refuse to work the AA program? Zemore 2019 makes a strong case that people in other programs can also stay clean and sober, as long as the goal is abstinence from alcohol. So, yes, I think people who will not work the AA program can still be clean and sober, but I also think it's dishonest to label all such people "failures" when measuring AA success.

Brandsma 1980 redux

Here’s a paragraph I wrote criticizing Bradsma 1980 when I restored that study to the Effectiveness of Alcoholics Anonymous article (using footnotes):

This study had poor methodology. There was no effort to stop the people in the control group from attending Alcoholics Anonymous meetings, and the "Alcoholics Anonymous" treatment patients underwent in the Brandsma study did not use actual Alcoholics Anonymous meetings. "The control condition allowed for participation in actual AA meetings, while those in the AA condition attended a weekly AA-like meeting administered by the study (that was not an actual AA meeting)"

That might not have been neutral, so another editor tried to make that paragraph more balanced:

Notably, there was no effort to stop the people in the control group from attending Alcoholics Anonymous meetings, and the "Alcoholics Anonymous" treatment patients underwent in the Brandsma study did not use community Alcoholics Anonymous meetings. A later critical analysis noted: "the control condition allowed for participation in actual AA meetings, while those in the AA condition attended a weekly AA-like meeting administered by the study (that was not an actual AA meeting)"

I feel this wording undermines the significant criticism Kaskutas 2009 has about Brandsma 1980, so I reworded it again, hopefully more neutral:

There was no effort to stop the people in the control group from attending Alcoholics Anonymous meetings. The "Alcoholics Anonymous" treatment patients underwent in the Brandsma study did not use community Alcoholics Anonymous meetings; a later analysis says that there are "concerns with the Brandsma trial which call its experimental results into question" because "the control condition allowed for participation in actual AA meetings, while those in the AA condition attended a weekly AA-like meeting administered by the study (that was not an actual AA meeting)"

This study has been heavily abused

I think this is pretty neutral, especially in light of just how problematic Brandsma 1980 is, especially when quoted out of context in anti-AA polemics. As just one example of this, here is how Dodes’s The Sober Truth describes Brandsma 1980:

The investigators found “significantly more binge drinking at the 3-month follow-up” among the people assigned to the AA-oriented meetings. As the year mark approached, the researchers noted, “All of the lay-RBT clients reported drinking less during the last 3 months. This was significantly better than the AA or the control groups at the 0.005 level [meaning the finding was highly statistically significant].” The final data led the researchers to conclude: “In this analysis the AA group was five times more likely to binge than the control group and nine times more likely than the lay-RBT group. The AA group average was 2.4 binges in the last 3 months.”

Brandsma, who passed away in 2008, is probably rolling in his grave that his legacy is this particular study that he did in 1979-1980 being misrepresented. Scott Alexander, in his blog, does not view how this study has been abused very kindly:

Brandsma (1980) is the study beloved of the AA hate groups, since it purports to show that people in Alcoholics Anonymous not only don’t get better, but are nine times more likely to binge drink than people who don’t go into AA at all.

There are a number of problems with this conclusion. First of all, if you actually look at the study, this is one of about fifty different findings. [...]

Second of all, the increased binge drinking was significant at the 6 [this is an error; it was at three months we saw the increased binge drinking] month followup period. It was not significant at the end of treatment, the 3 month [Scott probably means 6-month] followup period, the 9 month followup period, or the 12 month followup period. Remember, taking a single followup result out of the context of the other followup results is a classic piece of Dark Side Statistics and will send you to Science Hell.

Scott also has the same issues with this study Kaskutas 2009 had:

Brandsma didn’t use a real AA group, because the real AA groups make people be anonymous which makes it inconvenient to research stuff. He just sort of started his own non-anonymous group, let’s call it A, with no help from the rest of the fellowship, and had it do Alcoholics Anonymous-like stuff. On the other hand, many members of his control group went out into the community and…attended a real Alcoholics Anonymous, because Brandsma can’t exactly ethically tell them not to. So technically, there were more people in AA in the no-AA group than in the AA group.

Point being, this study has been criticized for its methodological problems, abused by people more interested in attacking AA than in objective facts, and has possibly done more harm than good in finding out the truth of Alcoholics Anonymous’s efficacy.

The 1990 triennial survey: 26%, not 5% one year retention rate

There is an old myth that a 1990 AA Triennial survey shows a 5% one-year retention rate for Alcoholics Anonymous. This is a false belief: That graph, frequently brought up by anti-steppers with little interest in objective truth, actually shows a 26% retention rate, as pointed out in the article "Alcoholics Anonymous (AA) Recovery Outcome Rates".

The graph is not a membership retention graph; it is a simple count of, among the members in their first year, how many were around a given number of months.  If AA had a 100% retention rate, the graph would have shown 8.3% (i.e. 100% divided by 12) of members (instead of the 5% it shows) in their 12th month of sobriety.

The wording in the survey itself says it is a graph of "the number of members that have been around a given number of months"; it was not a graph of per-month membership retention.

This graph does not show an 80% first month drop out rate and it does not show a 5% one-year retention rate.  It shows a 70% first-month retention rate and a 26% one-year retention rate.

Here are a number of books and papers which claimed the 1990 Triennial survey showed a 5% retention rate before "Alcoholic Anonymous Recovery Outcome Rates" was published:

• McIntire, Don (2000). "How Well Does A.A. Work?". Alcoholism Treatment Quarterly. 18 (4): 1. doi:10.1300/J020v18n04_01. "at 12 months the figures are 5% remaining and 95% departed."
• Carl G. Lukefield, "Behavioral Therapy for Rural Substance Abusers", 2000 has this quote: "By 12 months, 90 percent have dropped out (McCrady & Miller, 1993)"
• Charles Bufe, AA: Cult or Cure 1997 has this quote Quote: "AA produced a large monograph, “Comments on A.A.’s Triennial Surveys,” that analyzed the results of all five surveys done to that point. [...] AA has a 95% new-member dropout rate during the first year of attendance."

However, after "Alcoholic Anonymous Recovery Outcome Rates" was published, a number of sources all of a sudden decided that the 1990 Membership retention survey showed a 26% one-year retention rate, which just happens to be the number in that paper. Even anti-AA polemics like Dodes' (poorly-argued) The Sober Truth now use the 26% figure. For example:

• Lance Dodes, M.D.; Zachary Dodes (2014). The Sober Truth: Debunking the Bad Science Behind 12-Step Programs and the Rehab Industry. ISBN 978-0807033159. "AA itself has published a comparable figure in a set of comments on its own thirteen-year internal survey, stating that only 26 percent of people who attend AA stay for longer than one year"
• Adrian, Manuella (2012). "Can Failure Carefully Observed Become a Springboard to Success?". Substance Use & Misuse. 47 (13–14): 1384. doi:10.3109/10826084.2012.706178. "the Alcoholics Anonymous triennial surveys from 1977 through 1989 found that one quarter (26%) of those who first attend an AA meeting are still attending after 1 year"

Point being, the conclusion in "Alcoholic Anonymous Recovery Outcome Rates" that those old surveys showed a 26% one-year retention rate are considered reliable among treatment experts.

As Slate Star Codex puts it: "Almost everyone’s belief about AA’s retention rate is off by a factor of five because one person long ago misread a really confusing graph and everyone else copied them without double-checking."

Moderation for alcoholics: Maybe not

Let’s take a look at this article (original link); this article is an opinion from 2016 on what the science says about Alcoholics Anonymous. First of all, it accurately shows that the 2006 Cochrare review on AA effectiveness is out of date:

Ferri, Amato, and Davoli’s conclusion in a 2006 meta-analysis published in the Cochrane Review [13] has been widely quoted (see e.g., [14]): “No experimental studies unequivocally demonstrated the effectiveness of AA or [Twelve-Step Facilitation] TSF approaches for reducing alcohol dependence or problems” [13].
[...]
Several studies do support some efficacy of TS programs of recovery [15-19]. AA participation is associated with fewer drinks and more abstinent days [15-17], and recent studies show that AA attendance improves sobriety even while controlling for self-selection bias [18]. While these studies do not show unequivocal evidence of success—and are not evidence of sufficient effectiveness to recommend AA/TS programs for everyone—they do support inclusion of TS in the set of appropriate interventions.
[...]
13. Ferri M, Amato L, Davoli M. Alcoholics Anonymous and other 12-step programmes for alcohol dependence. Cochrane Database Syst Rev. 2006;(3):CD005032.
14. Frakt A. What is known about the effectiveness of AA? Incidental Economist. December 20
15. Walitzer KS, Dermen KH, Barrick C. Facilitating involvement in Alcoholics Anonymous during out-patient treatment: a randomized clinical trial. Addiction. 2009;104(3):391-401.
16. Witbrodt J, Ye Y, Bond J, Chi F, Weisner C, Mertens J. Alcohol and drug treatment involvement, 12-step attendance and abstinence: 9-year cross-lagged analysis of adults in an integrated health plan. J Subst Abuse Treat. 2014;46(4):412-419.
17. Moos RH, Moos BS. Participation in treatment and Alcoholics Anonymous: a 16-year follow-up of initially untreated individuals. J Clin Psychol. 2006;62(6):735-750.
18. Humphreys K, Blodgett JC, Wagner TH. Estimating the efficacy of Alcoholics Anonymous without self-selection bias: an instrumental variables re-analysis of randomized clinical trials. Alcohol Clin Exp Res. 2014;38(11):2688-2694.
19. Kaskutas LA. Alcoholics Anonymous effectiveness: faith meets science. J Addict Dis. 2009;28(2):145-157.

So far so good; this shows that AA effectiveness for a subset of alcoholics is pretty much undeniable at this point, and it even links to an experimentatal study (Walitzer 2009) showing this. At this point, the article jumps the shark by giving us this inaccurate whopper without footnotes:

Additionally, TS [Twelve Step] programs promote the goal of abstinence, but moderation is a better goal for some people.

This is a really bold claim, so we would expect about seven footnotes showing research supporting this claim, like we had with the claim that AA helps some alcoholics. But, no, this claim is made without a single footnote.

I will link to two articles which make a strong case this claim is downright false:

• Return to Moderate Drinking is Still a Lie (original article)
• Here’s What Happened When I Tried Moderation Management (original article)

After the disaster that was Sobell 1973, where the moderate drinkers who supposedly did well were either drunk, dead, or abstinent 10 years later, I will not believe any study showing alcoholics moderately drinking again without a 10-year followup. The reason why we need a 10-year followup is because alcoholics are really good at being dishonest about their alcoholic drinking being “moderate”, and it takes up to a decade for the drinking to get so bad the alcoholic can no longer deny their drinking is out of control.