University at Buffalo - The State University of New York
Skip to Content

News and views for UB faculty and staff

Research News

RIA awarded grant to develop telephone-based skill assessment tool


Published March 28, 2013

There is a growing trend to make clinical training available to alcohol and drug abuse treatment professionals via the Internet and through distance learning. It is cost effective and can deliver training in current evidence-based practices to audiences for whom access may be limited.

While ensuring that trainees can correctly apply the clinical skills taught during training is crucial, evaluation of whether or not clinical skills are correctly applied can be challenging because it is typically done through role playing. The assessment of role playing requires feedback, as well as reviewing the application of that feedback.

When training is delivered via the Internet and/or through distance learning, clinical skills application assessment might require such tools as video conferencing, which can be cost-prohibitive for many organizations.

UB’s Research Institute on Addictions (RIA) has been awarded a $267,469 National Institute on Alcohol Abuse and Alcoholism grant to refine a telephone-based clinical-assessment tool for evaluating the training of drug and alcohol counselors—a tool that is built on existing data—and then conduct a psychometric evaluation of its effectiveness.

“The idea for the grant came about because of a challenge we encountered on a previous study—the ‘Behavioral Interventions with Couples Project,’ or BIC project,” says Christopher Barrick, RIA research scientist and principal investigator on the new grant.

According to Barrick, the previous study involved expert clinicians in the field of substance abuse who were trained in behavioral couples therapy. The research compared the effects of being trained at an in-person workshop versus distance learning via video conference. The objective was to assess gains in knowledge and clinical skill following the workshop. Assessing knowledge was straightforward—trainees were asked to respond to a Web-based questionnaire.  

“Assessing clinical skill was more complicated,” says Barrick.

“How could we conduct a role play, a traditional method of clinical skill assessment, with our distance learning trainees, many of whom were in the greater Rochester area? Sending research staff to conduct in-person role plays would have been too time-consuming and costly,” he says. 

“To address this need, we developed a method that used interactive voice recording (IVR). Essentially, trainees telephoned in and the IVR system acted like a sophisticated voice mail system, capturing unrehearsed responses to clinical questions. It was a good start, but the method needs refinement. That's the goal of this current research grant.”
Barrick says specific goals of the project are to refine and extend the existing assessment tool developed in the BIC project, and conduct a psychometric evaluation to examine the generalizability, alternate form reliability and construct validity of the tool.  

Barrick notes that his co-investigator at RIA, research scientist Neil McGillicuddy, has had experience developing similar instruments. “I'm excited to collaborate with him on this project,” he adds. 

Barrick says this particular project focuses on developing a methodology that could be used in a variety of fields.  

“It just happens that there is a trend toward making trainings available to substance abuse treatment clinicians via the Internet and other distance learning methods, so this is a good fit with the broader needs of the field at the moment.

“There is a lot of great treatment and intervention research currently going on in the substance abuse field. Unfortunately, there has been an ongoing problem of transferring that work into community practice. This project is part of this broad area of research that looks at better ways to make the big investment in treatment and intervention research pay off and gets those results into the hands of people who can use it,” he says.

The grant is scheduled to run from December 2012 to December 2014.