What’s the Question? Why Asking the Right Question the Right Way Matters to Effective CME
I have memories of courses that were meant to teach me how – and how not – to write test questions. And while I’ve since used that knowledge to write everything from themed trivia questions to CME/CE assessments, no matter what type of multiple-question test you are creating – be it trivia, high school history, or for a continuing medical education activity – how the questions are written can greatly impact the type and quality of feedback you receive.
As an accredited/approved provider by ACCME, ACPE, and CBRN, we at CME Outfitters are committed to providing quality continuing medical education. The surveys and tests we implement help determine the efficacy of our activities and ensure our learners get the most from the education we provide.
Toward that end, there are some simple rules to writing effective surveys or tests that always apply:
- Avoid True/False questions, as they do not accurately reflect learner knowledge.
- Use a question format, not, for example, a fill-in-the-blank format.
- Keep answer lengths similar; additionally, avoid your correct answer being the longest or shortest choice.
- Balance your answer placement, keeping in mind that positions B and C are often overused for the correct option.
- If there is a logical order – such as numerical – that order should be used.
In addition to those most basic rules, there are several other guidelines that your questions and answers should follow, some of which are more directly related to CME/CE:
- Avoid both imprecise and absolute terms whenever possible. Terms like seldom, sometimes, and associated with might have different meanings for different learners, and savvy examinees know that few ideas or solutions are absolute or universally true.
- Wrong response options should be plausible distractors; they should of course be in the same therapeutic area, as well as falling into the same category as the correct answer. For example, if the correct answer is a diagnosis, all the wrong answers should also be diagnoses.
- When it is necessary to use a Likert-type question, use an even number of response options. This keeps participants from choosing the middle, or ‘average’, option.
- Patient-care scenarios make for the most engaging questions.
- Perhaps most importantly, all questions should have a clear link to the activity’s learning objective(s) and test at the same level of learning as the objectives are designed to assess.
It is also good to consider adding open-ended questions to your surveys, as they have several advantages, including:
- Allowing respondents to include more information, such as feelings, attitudes, and understanding of the subject,
- Requiring that the respondent read the full question rather than simply filling in an answer,
- And providing answers that you can use for secondary analysis (including, but not limited to, demographic information).
These are just some of the question writing rules that exist not just in our industry, but across all types of education. It might seem like a lot of information to take in, but as an accredited/approved provider by ACCME, ACPE, and CBRN, we are dedicated to providing education that drives positive outcomes for both our participants and their patients. The surveys and tests we create for CME activities help us to measure the effectiveness of our education, and their results help feed new educational content when we examine professional practice gaps and develop needs assessments. This ensures that our education is making a difference in clinicians’ and patients’ lives.
Contact us at firstname.lastname@example.org to learn more about the educational activities we can design for you.
About the Author
Follow me on Twitter: @taralynnewalker