Automatic Fails in MyDispense Assessments: To Use or Not To Use?

Key Points:

• Although controversial, automatic fails may be used in MyDispense assessments to emphasize important components of prescription processing
• MyDispense users can designate items as “critical criteria” within assessments

What is an automatic fail?

When learners do not demonstrate specific, essential criteria during an assessment, they may automatically “trip” a failing grade on the assessment. Essentially serving as “circuit breakers”, these select criteria are considered critical items required to pass the assessment.  In one of my high-stakes assessments, I use blood pressure accuracy as an essential criterion.  Thus, if a student demonstrates perfect technique taking a patient’s blood pressure but does not obtain an accurate reading, he/she trips the circuit breaker and automatically fails the assessment due to inaccuracy of the reading.  While the student may appear to receive a passing score based on the rubric used to assess technique, the student ultimately receives a failing score and must remediate the assessment to ensure both technique and accuracy are appropriate.  An important criterion, such as blood pressure accuracy in this example, is also sometimes referred to as a “kill point”.

In MyDispense assessment exercises, certain criteria designated as “critical criteria” can be used to place emphasis on important aspects of MyDispense cases.  Outcome, patient fact finding, and/or prescriber fact finding can be selected as these critical criteria.  I use critical criteria as required components to pass end-of-semester, high-stakes MyDispense assessments.  For example, if a student omits asking a patient about medication allergies and then dispenses amoxicillin to a patient with a severe allergic reaction to penicillins, the student automatically fails the assessment despite correctly processing all other aspects of the prescription (e.g. correct SIG, route, refills, etc.).  When this happens, students are required to meet with me to discuss the error, implications of the error in practice, and ways to prevent the error from occurring in future MyDispense exercises as well as real life.   Students then have an opportunity to remediate and must demonstrate competency in order to pass both the assessment and my skills laboratory course.

Why use automatic fails in MyDispense?

The concept of automatic fails is a pedagogical method with which not everyone agrees.  The harsh penalty of failure based on a single or small number of assessment items is undoubtedly one in which can result in student frustration, decreased confidence, and anxiety, not to mention an increase in instructor workload to address failures and remediate students if applicable.  However, automatic fails can be a powerful tool to demonstrate the importance of particular items imperative to successful outcomes and safe patient care.  As a simulation platform, MyDispense provides a risk-free environment and thus a fantastic place for learners to make mistakes.  However, without automatic fails, it can be especially difficult identifying students who make these critical mistakes, yet perform everything else well. This provides us the opportunity to help our students think about the seriousness of the mistakes they are making within MyDispense and how these mistakes may impact real patients.  What if the patient with whom the mistake was made was the student’s mother, father, sibling, grandparent, or friend? Having those discussions with students often “drives the point home” and highlights the value of the “failing grade with opportunity to remediate” versus harming a real person.

The use of automatic fails can be an impactful learning opportunity that improves patient safety by instilling the requirement for patient assessment and accuracy demanded of prescription processing, which is why I have chosen to implement critical criteria in MyDispense assessments. I believe skills-based assessments need to mirror actual practice as much as possible, and potentially adverse outcomes should be taken seriously.  Students who previously did not pass a MyDispense exercise and are now on rotations or working as licensed pharmacists have contacted me to report that since failing, they have not forgotten to ask a patient about allergies, verify a medication dose if unsure, or complete other items of importance when processing prescriptions.  If this has translated to improved patient safety and/or care, the difficult conversations with defensive students and extra time resulting from the automatic fail were most worth the effort.

How can critical criteria be designated in MyDispense?

1. When building an exercise, enable assessments:

2. Select “enable critical criteria”:

3. Critical criteria not attempted by a learner is indicated by a red highlight within Exam Manager:

As an alternative, one could weight the points for critical criteria such that if a learner misses the criteria, they would not receive a passing score.

Tips from the Trenches: Based on my experience using these methods in MyDispense, I have put together the following short list of helpful tips.

• Numerous practice opportunities containing critical criteria used in assessments should be provided to students before automatic fails are implemented.
• Considerable advance planning must be given to the impact of automatic failure and associated next steps. 
• Before the assessment, communicate to students that automatic failure is a possibility and what will happen in the event a student does not pass – be very transparent!  
• Prepare to have discussions with students who may be defensive or upset. Focus the conversation early on the potential impact their action (or lack thereof) may have had on a living person.
• Use automatic fails as an opportunity to discuss real world implications of the students’ mistakes, both to patient safety and professionally.
• Provide an opportunity to remediate so that students demonstrate competency, have additional practice, and have increased assurance in their ability to successfully process prescriptions.

If you have questions about this topic or others related to MyDispense, feel free to contact me at woodyard@purdue.edu.  Best wishes with your MyDispense endeavors!