{"id":{"repo_id":"queens","oai_identifier":"oai:queensu.scholaris.ca:1974/13713"},"canonical_url":"https://search.dev.ndltd.org/etd/queens/oai:queensu.scholaris.ca:1974/13713","repository":{"repo_id":"queens","name":"Queens University","base_url":"https://qspace.library.queensu.ca/server/oai/request"},"display":{"title":"Understanding How Raters Communicate in the Context of Medical High-Stakes, Performance-Based Assessments","abstract":"It is well established in the medical education literature that raters represent a significant source of unexplained variance in performance assessments. To better understand the variance associated with raters’ evaluative judgments, this dissertation investigated how raters, in various assessment contexts, communicate and articulate what they observe during performance assessments. Empirical data collected from three different performance assessments: a multiple mini-interview (MMI), an objective structured clinical examination (OSCE), and workplace-based assessments (WBAs) were analyzed to better understand raters’ behaviours. Relationships that exist between different forms of assessment were also examined. The first study established a proof of concept, showing that raters are not always consciously aware of the associations they make during assessments. The second study examined conditions under which raters find it difficult to make a seemingly uncomplicated judgment. Finally, the third study examined relationships that exist between quantitative and qualitative methods used in the evaluation of performance assessments. The empirical findings from this dissertation suggest that although raters may behave idiosyncratically, there exist patterns of behaviour that could be used to attain better information from raters about a learner’s performance and potentially allow raters to be more suitably matched according to their strengths in performance-based assessments. The findings of this research should help to overcome some of the challenges associated with rater-mediated assessments.","abstract_html":"It is well established in the medical education literature that raters represent a significant source of unexplained variance in performance assessments. To better understand the variance associated with raters’ evaluative judgments, this dissertation investigated how raters, in various assessment contexts, communicate and articulate what they observe during performance assessments. Empirical data collected from three different performance assessments: a multiple mini-interview (MMI), an objective structured clinical examination (OSCE), and workplace-based assessments (WBAs) were analyzed to better understand raters’ behaviours. Relationships that exist between different forms of assessment were also examined. The first study established a proof of concept, showing that raters are not always consciously aware of the associations they make during assessments. The second study examined conditions under which raters find it difficult to make a seemingly uncomplicated judgment. Finally, the third study examined relationships that exist between quantitative and qualitative methods used in the evaluation of performance assessments. The empirical findings from this dissertation suggest that although raters may behave idiosyncratically, there exist patterns of behaviour that could be used to attain better information from raters about a learner’s performance and potentially allow raters to be more suitably matched according to their strengths in performance-based assessments. The findings of this research should help to overcome some of the challenges associated with rater-mediated assessments.","abstract_has_math":false,"creators":["Sebok, Stefanie Susane"],"institution":null,"degree_name":null,"degree_level":null,"degree_discipline":null,"degree_department":"Education","school":null,"contributors":[],"advisors":["Klinger, Don A."],"committee_chairs":[],"committee_members":[],"year":2015,"date_issued":"2015-10-01","date_published":"2015-10-01","updated_at":"2026-07-27T20:35:45Z","subjects":["Rater-Based Assessments","Competency-Based Assessment","Psychometrics","Medical Education"],"languages":["eng"],"rights":[],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"http://hdl.handle.net/1974/13713","outbound_label":"Handle","outbound_source":"dc:identifier.uri"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor.department","label":"Department","values":["Education"]},{"key":"dc:contributor.supervisor","label":"Supervisor","values":["Klinger, Don A."]},{"key":"dc:creator","label":"Author","values":["Sebok, Stefanie Susane"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2015-09-27 17:35:28.998"]},{"key":"dc:date.accessioned","label":"Dc Date Accessioned","values":["2015-10-01T23:15:51Z"]},{"key":"dc:date.available","label":"Dc Date Available","values":["2015-10-01T23:15:51Z"]},{"key":"dc:date.issued","label":"Date","values":["2015-10-01"]},{"key":"dc:type","label":"Dc Type","values":["thesis"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Rater-Based Assessments","Competency-Based Assessment","Psychometrics","Medical Education"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language.iso","label":"Language (ISO)","values":["eng"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier.uri","label":"Identifier URI","values":["http://hdl.handle.net/1974/13713"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["Thesis (Ph.D, Education) -- Queen's University, 2015-09-27 17:35:28.998"]},{"key":"dc:description.abstract","label":"Abstract","values":["It is well established in the medical education literature that raters represent a significant source of unexplained variance in performance assessments. To better understand the variance associated with raters’ evaluative judgments, this dissertation investigated how raters, in various assessment contexts, communicate and articulate what they observe during performance assessments. Empirical data collected from three different performance assessments: a multiple mini-interview (MMI), an objective structured clinical examination (OSCE), and workplace-based assessments (WBAs) were analyzed to better understand raters’ behaviours. Relationships that exist between different forms of assessment were also examined. The first study established a proof of concept, showing that raters are not always consciously aware of the associations they make during assessments. The second study examined conditions under which raters find it difficult to make a seemingly uncomplicated judgment. Finally, the third study examined relationships that exist between quantitative and qualitative methods used in the evaluation of performance assessments. The empirical findings from this dissertation suggest that although raters may behave idiosyncratically, there exist patterns of behaviour that could be used to attain better information from raters about a learner’s performance and potentially allow raters to be more suitably matched according to their strengths in performance-based assessments. The findings of this research should help to overcome some of the challenges associated with rater-mediated assessments."]},{"key":"dc:description.degree","label":"Dc Description Degree","values":["PhD"]},{"key":"dc:title","label":"Title","values":["Understanding How Raters Communicate in the Context of Medical High-Stakes, Performance-Based Assessments"]}]}],"canonical_facts":{"dc:contributor.department":["Education"],"dc:contributor.supervisor":["Klinger, Don A."],"dc:creator":["Sebok, Stefanie Susane"],"dc:date":["2015-09-27 17:35:28.998"],"dc:date.accessioned":["2015-10-01T23:15:51Z"],"dc:date.available":["2015-10-01T23:15:51Z"],"dc:date.issued":["2015-10-01"],"dc:description":["Thesis (Ph.D, Education) -- Queen's University, 2015-09-27 17:35:28.998"],"dc:description.abstract":["It is well established in the medical education literature that raters represent a significant source of unexplained variance in performance assessments. To better understand the variance associated with raters’ evaluative judgments, this dissertation investigated how raters, in various assessment contexts, communicate and articulate what they observe during performance assessments. Empirical data collected from three different performance assessments: a multiple mini-interview (MMI), an objective structured clinical examination (OSCE), and workplace-based assessments (WBAs) were analyzed to better understand raters’ behaviours. Relationships that exist between different forms of assessment were also examined. The first study established a proof of concept, showing that raters are not always consciously aware of the associations they make during assessments. The second study examined conditions under which raters find it difficult to make a seemingly uncomplicated judgment. Finally, the third study examined relationships that exist between quantitative and qualitative methods used in the evaluation of performance assessments. The empirical findings from this dissertation suggest that although raters may behave idiosyncratically, there exist patterns of behaviour that could be used to attain better information from raters about a learner’s performance and potentially allow raters to be more suitably matched according to their strengths in performance-based assessments. The findings of this research should help to overcome some of the challenges associated with rater-mediated assessments."],"dc:description.degree":["PhD"],"dc:identifier.uri":["http://hdl.handle.net/1974/13713"],"dc:language.iso":["eng"],"dc:subject":["Rater-Based Assessments","Competency-Based Assessment","Psychometrics","Medical Education"],"dc:title":["Understanding How Raters Communicate in the Context of Medical High-Stakes, Performance-Based Assessments"],"dc:type":["thesis"]},"updated_at":"2026-07-27T20:35:45Z"}