Hi friend,

Slingshot has released new data on Ash, their AI product.

The data comes as an update to an existing research paper and sheds new light on how people use AI for their mental health, and how it works.

The original paper — a single-arm, naturalistic pilot study of the company’s foundation AI model — demonstrated that people using Ash experienced reductions in depression and anxiety that were sustained at 10 weeks. It also showed improvements in behavioural activation, loneliness and social connection — a finding that surprised many, given the general rhetoric around AI use.

While some welcomed the research as insight into how AI is used in real-world settings, others highlighted the limitations of the single-arm study, which cannot establish whether Ash itself was responsible for the observed improvements. Although the paper did not make causal claims, some felt that the broader communication around the findings could have more clearly reflected these limitations.

Now, the researchers have published an update to the paper. This version adds 12-month data, analysis of the interventions Ash implements with users and an assessment of Ash on SIM-VAIL, a safety and efficacy benchmark. As we attempt to understand the role AI can play in mental healthcare, this study offers new data and poses some important questions.

The original study recruited US adults with a score of at least 10 on either the PHQ-9 measure of depression or the GAD-7 measure of anxiety. Participants used Ash through text or voice and completed assessments at baseline, weeks 2, 4, 6 and 10. Researchers also measured loneliness, behavioural activation, social support, social interaction, goal attainment and therapeutic alliance.

The study found that;

Of course, because there was no control group, the study could not establish that Ash caused these improvements. We know that many people experience improvements in depression and anxiety symptoms without any intervention. So while the findings from this study were interesting, they were primarily evidence of the feasibility and real-world use of purpose-built AI for mental health support.

The latest version of the research extends the earlier work in three ways.

The study still does not have a control, so the results don't tell us whether Ash is effective as a therapy, or even how much of the improvement observed was caused by Ash, but there are still many interesting things we can learn from the paper.

At 10 weeks, depression and anxiety scores were substantially lower than at baseline, with a within-person effect size of d=0.93 for depression and d=0.79 for anxiety. At the 12-month follow-up, those reductions were maintained: d=0.98 for depression and d=0.76 for anxiety. The improvements in loneliness, behavioural activation and social interaction were also sustained at 12 months.

Again, there are limitations in what this can really tell us about the efficacy of Ash. As one example, only 62% of the initial cohort completed the measures at 12 months, and we don’t know what happened to the remaining 38%. Without a control group, we cannot separate the effects of Ash from spontaneous improvement, regression to the mean, other treatment, or any other changes in participants' lives.