Brandsma 1980: The AA treatment

The 1980 book Outpatient Treatment of Alcoholism (Brandsma 1980) has been quoted by anti-AA critics as showing AA as being ineffective.  In this post, I will look at the “AA” treatment which the subjects of the book underwent. Here is how the book describes it (“SHARP” here is an office the Brandsma research team set up for conducting their research; they tried to get people in the community to attend, but it ended up that most people got treatment there because the court mandated them to go):

An AA treatment group was started at SHARP by contacting the local AA organization. They sent two volunteers, a man (with a B.S.) and a woman (with a high school education), to begin our new group. They had 14 and 10 years experience in AA, respectively. The AA group was open to all who would attend and was conducted by one or the other of these people for approximately 1 year until January, 1973. By that time (the beginning of the research in terms of assigning patients), a member of the group (not included in the research project) had become very adept at running meetings. This minority group member, who had a 9th grade education, had had no AA experience before modeling himself after the two initial leaders of this group. This person remained the leader for the rest of the time of the research project (with a 3-week hiatus when an experienced AA counselor filled in), and continues to be involved in AA today. He received no remuneration for this, but was very cooperative in providing attendance reports for the project.

The “AA” treatment is described in more detail in another section of the book:

Our group met once per week and tended to focus on persons present for the first time, but not to the exclusion of other members. The 12 steps of AA were used as the content focus for discussions. The men were made aware of the availability of “sponsor-friends” and were encouraged to ally themselves with one or more of their own choosing. Generally our patients seems to neglect availing themselves of this opportunity, and the assignment of sponsors was not followed up on in this group. Beyond this description we are unable to be more specific. [...]

The nature of scientific investigation did force some small changes in the usual AA norms. First, most of our clients came coerced from the court system and all were then randomly assigned to treatments. Thus they were not “free to choose” whether to have treatment or the type of treatment. However, a person’s first visit to AA usually results from some form of “coercion,” e.g. a wife or boss. Second, participation was not anonymous because we kept attendance records, although this was an unobtrusive procedure. If a man’s attendance was poor or nonexistent, he would be contacted at least once by a project social worker, reminded of the conditions of his parole, and encouraged to return. This type of “coercion” (in contrast to the “buddy system”) is different than usual AA procedures. Our group did not develop a high degree of cohesiveness or a formalized buddy system as some do, nor did we offer other “services,” as does Al-Anon. However, given our population, situation, and purposes, we believe this was a representative treatment effort within the AA framework.

So, as we can see, almost no one in the Brandsma treatment got a sponsor, everyone in the group was a “court card” attender, and, even worse, they would get a threatening phone call if they didn’t show up at the meeting.

The Brandsma 1980 “AA” treatment (and, yes, Kaskutas 2009 is slightly inaccurate in her description of the “AA” treatment offered in the Brandsma study; the meetings were open to the public, but it’s unknown whether the meetings were listed in a local AA directory) does not sound like a meeting where people were effectively working the AA program. Kaskutas’s concern that “a potentially important therapeutic ingredient of AA—the experience of longer-term members—would not have been present in the AA condition” is spot on: The leader of the meeting had barely a year sober and the other attendees were “court slip” members.

Based on the description of the “AA” treatment, any conclusions Brandsma 1980 makes about AA efficacy are suspect.

Brandsma 1980 redux

Here’s a paragraph I wrote criticizing Bradsma 1980 when I restored that study to the Effectiveness of Alcoholics Anonymous article (using footnotes):

This study had poor methodology. There was no effort to stop the people in the control group from attending Alcoholics Anonymous meetings, and the "Alcoholics Anonymous" treatment patients underwent in the Brandsma study did not use actual Alcoholics Anonymous meetings. "The control condition allowed for participation in actual AA meetings, while those in the AA condition attended a weekly AA-like meeting administered by the study (that was not an actual AA meeting)"

That might not have been neutral, so another editor tried to make that paragraph more balanced:

Notably, there was no effort to stop the people in the control group from attending Alcoholics Anonymous meetings, and the "Alcoholics Anonymous" treatment patients underwent in the Brandsma study did not use community Alcoholics Anonymous meetings. A later critical analysis noted: "the control condition allowed for participation in actual AA meetings, while those in the AA condition attended a weekly AA-like meeting administered by the study (that was not an actual AA meeting)"

I feel this wording undermines the significant criticism Kaskutas 2009 has about Brandsma 1980, so I reworded it again, hopefully more neutral:

There was no effort to stop the people in the control group from attending Alcoholics Anonymous meetings. The "Alcoholics Anonymous" treatment patients underwent in the Brandsma study did not use community Alcoholics Anonymous meetings; a later analysis says that there are "concerns with the Brandsma trial which call its experimental results into question" because "the control condition allowed for participation in actual AA meetings, while those in the AA condition attended a weekly AA-like meeting administered by the study (that was not an actual AA meeting)"

This study has been heavily abused

I think this is pretty neutral, especially in light of just how problematic Brandsma 1980 is, especially when quoted out of context in anti-AA polemics. As just one example of this, here is how Dodes’s The Sober Truth describes Brandsma 1980:

The investigators found “significantly more binge drinking at the 3-month follow-up” among the people assigned to the AA-oriented meetings. As the year mark approached, the researchers noted, “All of the lay-RBT clients reported drinking less during the last 3 months. This was significantly better than the AA or the control groups at the 0.005 level [meaning the finding was highly statistically significant].” The final data led the researchers to conclude: “In this analysis the AA group was five times more likely to binge than the control group and nine times more likely than the lay-RBT group. The AA group average was 2.4 binges in the last 3 months.”

Brandsma, who passed away in 2008, is probably rolling in his grave that his legacy is this particular study that he did in 1979-1980 being misrepresented. Scott Alexander, in his blog, does not view how this study has been abused very kindly:

Brandsma (1980) is the study beloved of the AA hate groups, since it purports to show that people in Alcoholics Anonymous not only don’t get better, but are nine times more likely to binge drink than people who don’t go into AA at all.

There are a number of problems with this conclusion. First of all, if you actually look at the study, this is one of about fifty different findings. [...]

Second of all, the increased binge drinking was significant at the 6 [this is an error; it was at three months we saw the increased binge drinking] month followup period. It was not significant at the end of treatment, the 3 month [Scott probably means 6-month] followup period, the 9 month followup period, or the 12 month followup period. Remember, taking a single followup result out of the context of the other followup results is a classic piece of Dark Side Statistics and will send you to Science Hell.

Scott also has the same issues with this study Kaskutas 2009 had:

Brandsma didn’t use a real AA group, because the real AA groups make people be anonymous which makes it inconvenient to research stuff. He just sort of started his own non-anonymous group, let’s call it A, with no help from the rest of the fellowship, and had it do Alcoholics Anonymous-like stuff. On the other hand, many members of his control group went out into the community and…attended a real Alcoholics Anonymous, because Brandsma can’t exactly ethically tell them not to. So technically, there were more people in AA in the no-AA group than in the AA group.

Point being, this study has been criticized for its methodological problems, abused by people more interested in attacking AA than in objective facts, and has possibly done more harm than good in finding out the truth of Alcoholics Anonymous’s efficacy.

12-step class vs. 12-step fellowship

There are a few randomized controlled trials out there, such as Brandsma 1980 [1] and Stahlbrandt 2007 [2] where the "Alcoholics Anonymous" experimental condition was questionable. With both of these experiments, there was no difference between subjects who had the 12-step-like condition and the control group.

What happened?

The 12-step condition was not an actual AA meeting.

With Brandsma 1980, as Kaskutas 2009 [3] describes it, the subjects undergoing "AA" treatment weren't actually going to AA meetings:

The description of the AA condition states that the steps were used for discussion content, the group focused on newcomers, and they told patients about sponsors, but it is not clear whether the meetings were led by AA members, whether crosstalk was allowed, whether the meeting leader shared their story as part of the meeting, or whether the meeting format was what one would encounter at an actual AA meeting. The meetings may not have been open to other AA members in the community, and not been listed in the AA meeting directory, which would mean that a potentially important therapeutic ingredient of AA--the experience of longer-term members--would not have been present in the AA condition.

With Stahlbrandt 2007, here is how the supposed "AA" meetings were run:

The TSI intervention was a 3-hour formal lecture, given by
therapists trained in the 12-step method.

Like Brandsma 1980, they did not see a difference between the supposed AA condition and the control group.

Point being, if an experimental condition is supposed to be an AA meeting, the numbers will not be reliable unless the AA condition is, in fact, an actual meeting. Which it wasn't with these two studies.

More recent studies, such as Litt et al. 2009, where we give the experimental group treatment which gets them going to more real AA meetings, show that the more people go to meetings, the more days they will be sober. It is unfortunate that these poorly done surveys have incorrectly given the wrong impression about 12 step efficacy.

[1] Brandsma, Jeffery M; Maultsby, Maxie C; Welsh, Richard J (1980). Outpatient Treatment of Alcoholism: a review and comparative study. Baltimore, MD: University Park Press. ISBN 0-8391-1393-5

[2] Stahlbrandt, Henriettæ; Johnsson, Kent O.; Berglund, Mats (2007). "Cluster Randomized Trial". Alcoholism: Clinical and Experimental Research. 31 (3): 458–66. doi:10.1111/j.1530-0277.2006.00327.x. PMID 17295731

[3] Kaskutas, Lee Ann (2009). "Alcoholics Anonymous Effectiveness: Faith Meets Science". Journal of Addictive Diseases. 28 (2): 145–157. doi:10.1080/10550880902772464. PMC 2746426. PMID 19340677.

The most common anti-AA talking points

There are three key studies anti-steppers like to bring out when claiming the steps do not work:

• Ditman 1967. This is one of the earliest studies on AA, done before we knew how to properly study AA. It was a RCT -- a "randomized controlled trial." A RCT doesn't work with AA because 1) The people assigned to AA do not necessarily work the steps as suggested by oldtimers in AA meetings 2) The people not assigned to AA who are serious about getting sober end up going to the program anyway.
• Brandsma 1980. Like Ditman 1967, this was a RCT, with the same problems. In addition, there is an opt-repeated claim that this study shows AA results in increased binge drinking. However 1) The study looked at about three dozen different findings at multiple follow-up periods, but this result was only seen in this one finding at one follow-up period 2) This result has not been replicated in recent studies [1] even though 3) The study is 36 years old.
• The 1990 AA Triennial survey. This was not a scientific survey; this was an informal survey AA sent to members. Of the people in their first year of recovery, about 5% were in their 12th month of recovery, which is lower than the 8.3% rate we expect to see if AA had a 100% retention rate. Statistical extrapolations show that this survey shows a 29% retention rate; the question I ask is this: Is it AA's fault that 71% of newcomers decide to not come back to an AA meeting?

Note that the most recent information anti-steppers tend to bring out is over a quarter century old. What they don't like to talk about is the evidence that AA works: The more someone works AA, (That's from 2006 -- this century) the more likely they are to stay sober (That's from 2009). They dismiss this evidence with some hand-waving about "self selection", arguing that the people who stay sober by going to AA would have stayed sober anyway. While this is one possible explanation why people who go to AA stay sober, there is no evidence that this is the case. Indeed, there is a study from 2014 (under two years ago) showing that it's not just self-selection that keeps alcoholics in AA sober. 

Now, this study showing that it's not just self selection is just one study, and we need to have further studies to confirm that AA itself (and not just self-selection) helps keep alcoholics sober, but that's the direction the evidence points to right now. I would worry about these results being in just one study if the study was 36 years old, but this study is not even two years old.

Bottom line: If you have a drinking problem, and want to get and stay sober, AA is an effective way to do just that. 

[1] Edit: I need to point out Ogborne 1982, but this study is most likely just observing the fact that AA members are more likely to be honest about the negative consequences of their drinking.

A reasonable overview of the AA research out there

Here is a reasonable, if not great overview of the various studies about Alcoholics Anonymous' effectiveness:

http://slatestarcodex.com/2014/10/26/alcoholics-anonymous-much-more-than-you-wanted-to-know/

The biggest defect this essay has is that it too easily dismisses longitudinal studies, claiming they are completely ineffective because of selection bias. Unfortunately, because of the peculiar nature of AA, a proper randomized controlled trial is not feasible (the people who get sober in the control group and/or non-AA-treatment group will, as likely as not, end up going to AA meetings), so longitudinal studies are a reasonable research mechanism, albeit with the thorny self selection bias issue.

PMC4285560, which was published around the same time that blog entry was made, uses sophisticated statistical methods to minimize self selection bias; once that is done, we see that there is a specific effect from the AA program that keeps people sober.

This blog entry's strength is that it has good summaries of studies which are only available in books or behind paywalls. For example, reading its summary of Brandsma 1980, a long-time chestnut anti-steppers love to bring out, shows that the study does not actually reveal a pattern of increased binge drinking among AA members (a claim which is frequently parroted by anti-steppers, such as in the "Rational" wiki).

Brandsma 1980: Not peer reviewed

While researching which studies to discuss in the Effectiveness of Alcoholics Anonymous Wikipedia article, I went over to the Rational Wiki Alcoholics Anonymous article to see if I could add some peer reviewed information showing that AA is not effective from the other side.

Unfortunately, the Rational Wiki did not have any good science in it. The study they refer to with allegations that AA actually increases binge drinking has the ambiguous name "Brandsma et al. 1980" Research revealed that this supposedly scientific, albeit dated, survey showing how AA is worse than no treatment would have a DOI or PMCID, the way peer-reviewed studies do. It doesn't. 

A Google search of the study showed the primary people referring it were unreliable anti-steppers (including, yes, people on the fringe of academia like Stanton Peele). Apparently, this study got put on the Orange Papers site, where the usual cadre of anti-steppers mirrored it from.

Finally, I got some solid information about this article in, of all places, an Amazon review. To summarize: This study was not peer reviewed, was published by one University Park Press, which mainly prints university textbooks -- read: not published in a journal, and its methodology is, at best, suspect.

It is referred to in journal papers, but usually negatively. For example, PMC2746426 mentions "concerns with the Brandsma trial which call its experimental results into question", and PMC3602358 says that studies like this are "significantly limited in their methods or interpretability," PMC3549307 (by the same authors) says the same thing.

Why is it that anti-steppers need to use a 35-year old non-peer reviewed study to find "science" to back up their anti-AA talking points? Because the real science shows that AA actually works.

Let me rephrase that: The objective science shows the people who go to more AA meetings are more likely to stay sober. The science has not yet concluded that it's AA's 12 step program that results in the statistically undeniable positive outcomes for AA members, but research is going in that direction.