Assessment - FAQ
Mastery exercise question review for Foundations Curriculum students
Faculty will provide a debrief or ‘teaching point’ for questions answered poorly by the class. Neither the question nor answers will be released. This is to maintain the integrity of the exam bank, and to ensure that students focus on the learning objective addressed by the question or the learning gap identified as opposed to the question itself.
Students who performed poorly on a Mastery Exercise should reach out to the OHPSA academic coach or course director to discuss learning and studying strategies. Rules and regulations are in place to maintain the integrity of the MD Program’s question banks. Hence, viewing of assessments is limited to written assessments that include short answer questions (SAQs) and/or other narrative components, in order to request a re-mark of a written assessment.
How are mastery exercises created for Foundations Curriculum courses?
Multiple choice questions (MCQs) on mastery exercises are written to assess one or more end-of-week learning objective(s). MCQs are developed by content experts who are usually directly involved in teaching. The question stems in the MCQs will include all of the information required to answer the question. There is one correct answer and three incorrect alternatives known as distractors.
Mastery exercise questions are designed to assess not just whether students can remember facts and understand concepts, but also if they can apply the content being taught to a real-life like clinical situation.
Any learning objective can serve as the basis of an assessment question on a mastery exercise, including those objectives that are covered by pre-week and self-learning.
Following each mastery exercise, and prior to approving the students’ results for publication, the course director reviews the performance statistics of each question and the mastery exercise as a whole in accordance with evidence-informed processes endorsed by the Test Committee.
Once the mastery exercise results are approved by the course director students will receive a report summarizing their performance that can be used to guide their further studying.
How is the passing mark of mastery exercises determined for Foundations Curriculum courses? Why are they different for the different courses in the Foundations Curriculum?
Based on the input and review of the the Student Assessment and Standards Committee (SASC), the Foundations Committee, and the Curriculum Committee, we currently use the Hofstee method for standard setting for Mastery Exercises (ME). This approach asked members from SASC and the Foundations Committee to examine ME data from the previous academic year. Using their understanding of the objectives and goals of each course as well as the historical ME data, each committee member was asked to estimate what the acceptable values would be for the minimum and maximum pass mark in each course as well as the minimum and maximum number of students who could possibly be below the pass mark. This does not mean that there will always be the minimum number of students below the pass mark every year. Instead, these judgements are just an approach to help judges communicate their sense of the expected difficulty of each course. Combing all of this information from members of both committees, we plot the average values on to previous ME data for a particular course and identify which score satisfies the best estimates of the committee. This point becomes the new pass mark for the course. The new pass marks are deployed and re-evaluated again at end of the academic year.
The pass mark can be found on the Elentra course page
When will Foundations Curriculum students receive results of their mastery exercise?
Reports will be released to students approximately 1-2 weeks after the completion of mastery exercises.
Following the completion of each mastery exercise, the course director conducts an analysis of the assessment to ensure that all questions meet minimum standards for quality. This includes a review of item difficulty, reliability and clarity. Adjustments will be made to optimize the overall reliability of the assessment prior to the release of reports to students. The 1-2 week timeframe also allows for the small proportion of students who need to write a make up assessment for various reasons (typically scheduled on the next self-learning day), so that the results can be released to the entire class at the same time.
How do I interpret the mastery exercise feedback report to support me in my learning?
The report you receive provides a breakdown by type of question and the specific learning objective assessed in the mastery exercise. You can use the report to determine how you did on certain types of questions (e.g. remembering, understanding, applying or evaluating) and which learning objectives you should revisit to ensure that you understand the material well enough. Each code corresponds to a learning objective which you can find in Elentra.
What happens if there is a university closure on a mastery exercise date?
Your mastery exercise will be rescheduled.
What happens if I fail a mastery exercise?
If a student fails a mastery exercise, they should contact their course director to review their performance on the assessment and determine next steps.
What is a progress test?
The progress test is a comprehensive knowledge-based test designed to assess the targeted exit competency of students in the MD program. Progress tests are completed by Year 1, 2, 3 and 4 students. The best preparation for the test is to engage in the curriculum and stay up to date throughout the program.
Why do we have progress testing?
Progress tests are designed to show your growth and competency acquisition over time. It’s a stable measure that shows you where you are, and where you need to be by the end of the program.
When will progress tests occur?
Progress tests will be given to students at regular intervals throughout the four years of the MD Program. Year 1, 2 and 3 students will have three progress tests every academic year. Year 4 students will have two progress tests every academic year. The schedule can be found in MedSIS.