Machine-learning of supportive counseling skills

Zhang, X., Goldberg, S. B., Baldwin, S. A., Tanana, M. J., Weitzman, L. M., Narayanan, S. S., Atkins, D. C., & Imel, Z. E. (2025). Association of machine-learning-rated supportive counseling skills with psychotherapy outcome. Journal of Consulting and Clinical Psychology, 93(2), 110–119. https://doi.org/10.1037/ccp0000935

Rationale

This study addresses a need to better understand the relationship between specific supportive counseling skills and client outcomes.

Previous research on supportive counseling, which includes skills like active listening and reflection, has shown its significant therapeutic value, sometimes achieving approximately 75% of the effect of more specialized treatments.

However, studies examining the impact of individual supportive counseling skills, such as empathy, open questions, and reflections, have produced mixed results.

This inconsistency may stem from limitations in previous research designs, including small sample sizes, a focus on overall associations rather than therapist-specific variations, and a lack of standardized coding systems for skill assessment.

To address these limitations, this study employed a machine-learning-based approach to assess therapist skills at scale, allowing for a more nuanced analysis of the relationship between these skills and client outcomes.

Method

Procedure

• Data collection: The study utilized a dataset of 2,974 therapy sessions from a university counseling center, including audio recordings and client symptom level data.
• Skill assessment: Therapists’ use of empathy, open questions, and reflections was assessed using a machine learning system.
• Outcome measurement: Client symptom levels were measured before and after treatment using the Counseling Center Assessment of Psychological Symptoms Instruments (CCAPS).
• Data analysis: Multilevel modeling was used to examine the relationship between therapist skills and client outcomes, separating between-therapist and within-therapist effects.

Sample

• The sample consisted of 610 clients and 48 therapists.
• Demographic information for clients and therapists is provided in Supplemental Tables S1 and S2.

Measures

• Empathy: A session-level score (1-5) representing the therapist’s understanding of the client’s perspective.
• Open Questions: An utterance-level label indicating questions that encourage the client to provide more information.
• Reflections: An utterance-level label indicating therapist statements that reflect or paraphrase the client’s words.
• CCAPS: A questionnaire measuring client psychological symptoms, used to assess treatment outcomes.

Statistical Measures

• Multilevel modeling was used to analyze the data.
• The model separated between-therapist and within-therapist effects.
• Standardized coefficients were used to compare effect sizes.

Results

• Hypothesis 1a: Higher between-therapist empathy would be associated with larger symptom reduction. Result: Unsupported
• Hypothesis 1b: Higher within-therapist empathy would be associated with larger symptom reduction. Result: Unsupported
• Hypothesis 2a: Higher between-therapist use of open questions would be associated with larger symptom reduction. Result: Supported
• Hypothesis 2b: Higher within-therapist use of open questions would be associated with larger symptom reduction. Result: Unsupported
• Hypothesis 3a: Higher between-therapist use of reflections would be associated with larger symptom reduction. Result: Supported
• Hypothesis 3b: Higher within-therapist use of reflections would be associated with larger symptom reduction. Result: Unsupported

Insight

The study’s key finding is that therapists who used more open questions and reflections, on average, had clients with better treatment outcomes.

However, this association was not observed within therapists, meaning that a therapist’s use of these skills relative to their own average did not predict individual client outcomes.

This study is particularly informative because it clarifies the relationship between supportive counseling skills and client outcomes.

It suggests that while these skills are important, their impact may be more related to the therapist’s general approach than to specific interactions with individual clients.

The findings extend previous research by highlighting the importance of considering therapist-level effects when studying the impact of specific skills.

They also suggest that the effectiveness of open questions and reflections may vary depending on client characteristics and therapeutic context.

Future research could explore the reasons behind the observed between-therapist effects, investigate the role of client characteristics in moderating the relationship between skills and outcomes, and examine the potential benefits of training programs focused on increasing therapists’ use of open questions and reflections.

Implications

The findings have implications for both practitioners and policymakers.

They suggest that training programs for therapists should emphasize the importance of open questions and reflections in facilitating client improvement.

Additionally, supervisors could use these findings to provide feedback to therapists on their use of these skills.

Policymakers could consider incorporating these findings into the development of treatment guidelines and quality assurance measures.

However, it is important to acknowledge the challenges of implementing these findings, such as the need for further research to establish causal relationships and the potential for oversimplification of the complex interplay of factors that contribute to therapeutic success.

Strengths

• Large sample size
• Use of a standardized, machine-learning-based skill assessment system
• Separation of between-therapist and within-therapist effects

Limitations

• The machine learning model’s accuracy for certain skills (e.g., reflections) was lower than for others.
• The model only predicted a single skill code per therapist statement, potentially missing important information.
• The study relied on a correlational design, limiting the ability to draw causal conclusions.

Reference

Zhang, X., Goldberg, S. B., Baldwin, S. A., Tanana, M. J., Weitzman, L. M., Narayanan, S. S., Atkins, D. C., & Imel, Z. E. (2025). Association of machine-learning-rated supportive counseling skills with psychotherapy outcome. Journal of Consulting and Clinical Psychology, 93(2), 110–119. https://doi.org/10.1037/ccp0000935