Imagine you’ve created a survey to assess changes in competence among participants in a CME activity (pre- vs. post-participation). Let’s say there’re a total of eight questions in your survey (each one asking participants to self-report their utilization of a specific clinical task related to the CME activity content on a 5-point scale). One week after the CME activity is completed, you’re staring at an Excel spreadsheet containing all of the pre- and post-activity responses. Now what? How do you use these data to determine whether your activity was effective? And how do you compare these data to the results of other outcome assessments?
If this was an infomercial, I’d now break into scene (colorfully sweatered) and introduce the latest advancement in outcome measurement: Cohen’s d. It slices, it dices…it reduces all of your outcomes data into a single metric that summarizes the overall effectiveness of your CME. But wait, there’s more…you can use Cohen’s d to compare the effectiveness of your CME across activities. That’s right. Want to know how effective this year’s conference was relative to last year’s? Cohen’s d. Want to know how effective your live CME activity was relative to its repurposed enduring material? Cohen’s d. Want to know the effectiveness of your overall CME program by topic? Or by format? That’s right, Cohen’s d.
Although this may sound like an innovation, Cohen’s d has been around for decades. It’s even been used in CME; however, this has largely been restricted to academic publications. But the claims made above are true. Cohen’s d can answer the following questions: 1) Was my CME effective? 2) How effective was it? And 3) how effective was my CME relative to other CME activities? Better still, it’s remarkably simple to calculate – if you can calculate a mean and a standard deviation (not by hand, of course, use Excel), then you can calculate a Cohen’s d.
I’m going to dedicate two more blog posts to Cohen’s d. This one provides step-by-step instructions of how to calculate it and interpret the results. The other will demonstrate how you can use Cohen’s d to assess the effectiveness of your CME beyond the individual activity (i.e., by topic, format, or year). And I promise, you’ll be amazed at how easy this is to do.
Subscribe in the top right corner to get notified when new blog posts are added to follow along.
2 Comments to Calculating Effect Size for Your CME Activities