I’m unsure about this meta analysis because it’s presumably based on AA publications.
The thing is That numerous prior AA publications have treated “falling off the wagon” or missing one of the steps to mean that you weren’t doing AA properly, and so they dont report it as a failure.
Essentially they end up getting close to just filtering out all the treatment failures, which inflates their success rate.
Based on the OP and two other articles (see below), I believe your presumption is mistaken. It's hard to say for sure, because the list of studies included in the meta-analysis hasn't been published yet (that I can find). (Edit: it looks like the study is available at https://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.CD...) But the articles make it clear that this was a rigorous meta-analysis done by qualified researchers. Surely they didn't just toss in AA publications uncritically. What they're saying is that, after careful review, the studies that have been popularized in the past as showing that AA was less effective are the ones that turn out to be wrong. From the NYT link:
The 2006 Cochrane Collaboration review was based on just eight studies, and ended with a call for more research to assess the program’s efficacy. In the intervening years, researchers answered the call. The newer review also applied standards that weeded out some weaker studies that drove earlier findings.
In the last decade or so, researchers have published a number of very high-quality randomized trials and quasi-experiments. Of the 27 studies in the new review, 21 have randomized designs. Together, these flip the conclusion.
The measure of success should be relapse after treatment, but AA doesn't have the concept of stopping, and more importantly does not consider relapse after ceasing attendance to be a failure of their 12 step program.
This ignores the moral corruptness of praying on, and essentially mandating indoctrination, of people in need of help.
The debunking you're quoting has presumably been superseded by the much higher-quality randomized studies reported on in the article. At least, if that's not the case, then the researchers who've published the new meta-analysis lack even basic competence, which seems unlikely.
That old 2015 article is out of date, and the brand new Cochrane review presents evidence which downright contradicts a number of its findings. Notably, that old article claims AA has a 5% success rate, but the new Cochrane review sees a 41.8% success rate for people who got treatment which got them in the rooms of AA.
And here I was just thinking the finding makes sense to the extent that AA is (probably) more involved than (most) psychotherapy. To extend my point, I think when people consider the benefits of things like therapy, that--because their attention is on the therapy at that moment--they don't realize that if therapy is 1 hour a week, there are 167 hours left in the person's week that can either help or harm them with respect to their goal.
The thing is That numerous prior AA publications have treated “falling off the wagon” or missing one of the steps to mean that you weren’t doing AA properly, and so they dont report it as a failure.
Essentially they end up getting close to just filtering out all the treatment failures, which inflates their success rate.