Assessment - FAQ
What is the standard setting method for Objective Structured Clinical Examinations (OSCEs)?
Currently, Borderline Regression (BLR) is used as the method to set a pass mark in Year 1 and Year 2 Clinical Skills OSCE. In the Year 3 iOSCE, an absolute pass mark of 60% (or 3 out of 5) is used while BLR is the supplementary method to identify students who might benefit from extra support.
BLR calculates individual station pass marks based on the results of a regression analysis, using the overall performance mark as the independent variable and the combined mark of station checklist mark and global rating mark as the dependent variable. The regression line is calculated based on all of the interactions between examiners and students for a particular station. The pass mark is then the point at which the regression line crosses the borderline category. The overall pass mark of OSCEs using the BLR method is calculated as the total of all of the station pass marks plus two Standard Error of Measurement (SEM).
What is a weekly feedback quiz?
Weekly feedback quizzes are short quizzes delivered through Examplify that first year and second year Foundations Curriculum students will complete at home each week. While low stakes, these will help students examine how well they have learned the week’s material. These quizzes will focus on key concepts that must be understood, but some higher-level questions related to comprehension and application of concepts will also be included.
Is there a time limit to complete the weekly feedback quiz?
There is no time "limit" to complete the quiz; however, it is best to do the quiz in one sitting. Students are expected to complete the quizzes before the deadline in order for their results to be uploaded to the Learner Chart.
Do I get the results of the weekly feedback quiz right away?
Yes, you will be able to see your score as soon as you complete the quiz. Also, once your quiz has been uploaded you will receive a detailed feedback report in your ExamSoft account.
Additional weekly feedback quiz attempts
Students have access to two additional attempts of each weekly feedback quiz. These attempts are for learning purposes only and will not appear in the Learner Chart. Students will receive feedback reports in their ExamSoft accounts, showing the learning objectives attached to all questions and the question stems, after completing the quiz and after each subsequent attempt.
What if I forgot to do a weekly feedback quiz, or am sick and cannot write the quiz?
There is no make-up time or approvals of absence or deferrals if students miss a quiz. Missed weekly feedback quizzes will be indicated as incomplete in your Learner Chart.
What is a mastery exercise?
Mastery exercises are mandatory invigilated written assessments for first, second and third year students. Students complete these assessments using Examplify on their own devices. These assessments may include multiple-choice questions, short-answer questions, essay questions or key feature questions.
Are mastery exercises all weighted equally in Foundations Curriculum courses?
No, mastery exercises are weighted according to the amount of material (number of weeks of material) they cover in the first and second years.
Mastery exercise question review for Foundations Curriculum students
Faculty will provide a debrief or ‘teaching point’ for questions answered poorly by the class. Neither the question nor answers will be released. This is to maintain the integrity of the exam bank, and to ensure that students focus on the learning objective addressed by the question or the learning gap identified as opposed to the question itself.
Students who performed poorly on a Mastery Exercise should reach out to the OHPSA academic coach or course director to discuss learning and studying strategies. Rules and regulations are in place to maintain the integrity of the MD Program’s question banks. Hence, viewing of assessments is limited to written assessments that include short answer questions (SAQs) and/or other narrative components, in order to request a re-mark of a written assessment.
How are mastery exercises created for Foundations Curriculum courses?
Multiple choice questions (MCQs) on mastery exercises are written to assess one or more end-of-week learning objective(s). MCQs are developed by content experts who are usually directly involved in teaching. The question stems in the MCQs will include all of the information required to answer the question. There is one correct answer and three incorrect alternatives known as distractors.
Mastery exercise questions are designed to assess not just whether students can remember facts and understand concepts, but also if they can apply the content being taught to a real-life like clinical situation.
Any learning objective can serve as the basis of an assessment question on a mastery exercise, including those objectives that are covered by pre-week and self-learning.
Following each mastery exercise, and prior to approving the students’ results for publication, the course director reviews the performance statistics of each question and the mastery exercise as a whole in accordance with evidence-informed processes endorsed by the Test Committee.
Once the mastery exercise results are approved by the course director students will receive a report summarizing their performance that can be used to guide their further studying.