Assessment - FAQ
What is standard setting?
Standard setting is the process by which pass marks are determined for assessments. This area of education has a great deal of research and evidence to support. There is no single best way to set pass marks for tests just as there is no objective or mathematical formula to decide on the pass mark. At the same time, pass marks are not determined arbitrarily. They are set considering the goals of the curriculum, the level of mastery expected of students, expert opinion, and statistical analysis combined through rigorous methodology. (The MD Program’s curriculum goals can be found here: https://md.utoronto.ca/mdprogramcompetencies).
Who sets the standards for Mastery Exercises in Foundations?
Standard setting is reviewed by three important committees in which students have a voice and representation: the Student Assessment and Standards Committee (SASC), the Foundations Committee, and the Curriculum Committee.
Does the program aim to flag a specific proportion of the class as below the standard and pass mark in Foundations?
The standards we use for the curriculum are not intended to increase the number of students who do not meet the standard; there is no norm referenced standard setting in the MD program (i.e., we do not aim to flag a certain portion of the class as below the standard after each Mastery Exercise (ME) or course). Instead, our pass marks are adjusted on a yearly basis based on the difficulty of the questions and ensuring that the assessments used in the program enable a determination if a student has met the required course objectives and competencies for MD training. It is possible all of the students will be above the pass mark for an ME despite the estimates of the judges. Additionally, quality control metrics are used to evaluate each ME and all assessments on a routine basis including item difficulties, appropriateness of questions for objectives, and overall validity.
How can I find out more about the Hofstee method?
The Hofstee method for standard setting is used for Foundations Curriculum mastery exercises. The Hofstee method is well described in the literature on standard setting. You may also find out more from your student reps on SASC and Foundations. Mckinley DW, Norcini JJ. How to set standards on performance based examinations: AMEE Guide No 85. Med Teach. 2014;36:97–110. (https://www.ncbi.nlm.nih.gov/pubmed/24256050)
How do I know what pass mark will be used for the current academic year?
You can find the information on the Assessment tab of the course’s Elentra page.
What security guidelines must students adhere to for written assessments?
Written Assessment Delivery Process Guidelines
1. Students should bring photo identification (T-card) and are required to display it at the request of the invigilator/examiner. For computer-based assessments, students are responsible for bringing their own device (computer or tablet).
2. Students should only be directed to a back-up room and/or be provided with a back-up device if they have a technical issue that cannot be addressed in the main assessment room. In the case of technical issues, students should be allowed the full scheduled time to complete the assessment. Back-up rooms and/or devices are not a requirement for Clerkships.
3. One sheet of paper will be provided to all students once everyone is seated. The invigilator(s)/examiner(s) will collect this paper from all students at the end of the assessment. Failure to return the paper, whether used or not, will be considered a security breach. No other materials or aids should be used during the assessment unless explicitly authorized by the invigilator/examiner.
4. Bags and books are to be deposited in areas designated by the invigilator/examiner and are not to be taken to the assessment desk/table. All other electronic devices are to be turned off and must remain in the designated area. This includes cell phones, smart watches or other electronic devices.
5. The invigilator/examiner has the authority to assign seats to students in the assessment room. No person will be allowed in an assessment room during an assessment except the students completing the assessment and those supervising the assessment.
6. Students who arrive late to an assessment will be permitted to enter the room. Students who arrive late will have their names and the time they arrived recorded. Students who arrive late will not be allowed any additional time. For Foundations, students who arrive ten (10) or more minutes late for the assessment will be directed to a back-up room.
7. Examplify should be the only application open on the student device, from the start of the assessment until the student has left the assessment room. If a student finishes their assessment and is expected to stay in the room they should do so with their device closed.
8. Students may leave the assessment room no earlier than thirty (30) minutes after the start of the assessment, and under supervisor. Candidates shall remain seated at their desks during the final ten (10) minutes of each assessment, even if they have completed the assessment.
9. If leaving the assessment room for a comfort break, students are not permitted to remove their personal devices, cellphones, and bags from the room except by authority of the invigilator/examiner.
10. Irregularities/errors relating to wording, spelling, punctuation, numbers or notations may not be dealt with during the assessment. Ambiguities will normally be referred to the course director in writing within 24 hours of the assessment.
11. All students will be expected to show their confirmation of upload screen once they have submitted.
12. Students will agree to the following Statement of Acknowledgement for every written assessment.
“I will not share the content of this assessment with anyone other than the Course Director (if the need arises). I will not solicit or provide any information about items, however vague, through verbal or electronic communication including, but not limited to email, text, social media platforms, and chatrooms. I appreciate that doing so will place me in breach of the University of Toronto’s Code of Behaviour on Academic Matters. In such an instance, I acknowledge that I would be liable for academic penalties, under the Code.”