{"id":{"repo_id":"uic","oai_identifier":"oai:figshare.com:article/32995307"},"canonical_url":"https://search.dev.ndltd.org/etd/uic/oai:figshare.com:article/32995307","repository":{"repo_id":"uic","name":"University of Illinois - Chicago","base_url":"https://api.figshare.com/v2/oai"},"display":{"title":"Rater Implicit Gender Bias in a Multi-Center Simulation-Based Resident Milestone Assessment","abstract":"Background: Gender bias in graduate medical education assessments may disadvantage female residents. With simulation increasingly used in high-stakes assessments, such as those used to assess emergency medicine residents, understanding potential bias is essential. Objective: To evaluate the presence of gender bias in Simtastic, a large multi-center simulation-based milestone assessment of second year residents to better understand the presence of gender bias in simulation-based assessments. Methods: This retrospective study analyzed 1011 Simtastic scores of PGY2 residents from 2015–2025. Residents were scored using case-specific checklists and global assessments. The data was analyzed for scoring bias based on gender concordance between rater and resident by comparing global assessments to the checklist score on each case. Hierarchical linear modeling accounted for clustering by residency program, site, and rater. Results: Checklist and global assessments (GA) scores were strongly correlated. However, female residents received lower GA and checklist scores than male residents across all years. Male raters awarded higher scores to male residents, while female raters showed no such difference. Over time, GA scores from male raters converged with those from female raters, suggesting reduced inflation with rater experience. Despite this trend, score disparities for female residents persisted. Conclusions: Although female residents consistently received lower scores in Simtastic, such trends likely reflect systemic bias rather than performance differences. Despite male rater bias diminishing over time, inequities remained. Structured rater training and refinement of checklist design are needed to promote equity in high-stakes simulation-based evaluations.","abstract_html":"Background: Gender bias in graduate medical education assessments may disadvantage female residents. With simulation increasingly used in high-stakes assessments, such as those used to assess emergency medicine residents, understanding potential bias is essential. Objective: To evaluate the presence of gender bias in Simtastic, a large multi-center simulation-based milestone assessment of second year residents to better understand the presence of gender bias in simulation-based assessments. Methods: This retrospective study analyzed 1011 Simtastic scores of PGY2 residents from 2015–2025. Residents were scored using case-specific checklists and global assessments. The data was analyzed for scoring bias based on gender concordance between rater and resident by comparing global assessments to the checklist score on each case. Hierarchical linear modeling accounted for clustering by residency program, site, and rater. Results: Checklist and global assessments (GA) scores were strongly correlated. However, female residents received lower GA and checklist scores than male residents across all years. Male raters awarded higher scores to male residents, while female raters showed no such difference. Over time, GA scores from male raters converged with those from female raters, suggesting reduced inflation with rater experience. Despite this trend, score disparities for female residents persisted. Conclusions: Although female residents consistently received lower scores in Simtastic, such trends likely reflect systemic bias rather than performance differences. Despite male rater bias diminishing over time, inequities remained. Structured rater training and refinement of checklist design are needed to promote equity in high-stakes simulation-based evaluations.","abstract_has_math":false,"creators":["Clare Desmond (24400265)"],"institution":null,"degree_name":null,"degree_level":null,"degree_discipline":null,"degree_department":null,"school":null,"contributors":[],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2026,"date_issued":"2026-05-01T00:00:00Z","date_published":"2026-05-01T00:00:00Z","updated_at":"2026-07-27T21:33:53Z","subjects":["Rater Gender Bias","Gender Bias in Simulation-Based Assessment"],"languages":[],"rights":["In Copyright","Open Access after 2028-05-01"],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"https://doi.org/10.25417/uic.32995307.v1","outbound_label":"DOI","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:creator","label":"Author","values":["Clare Desmond (24400265)"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2026-05-01T00:00:00Z"]},{"key":"dc:relation","label":"Dc Relation","values":["https://figshare.com/articles/thesis/Rater_Implicit_Gender_Bias_in_a_Multi-Center_Simulation-Based_Resident_Milestone_Assessment/32995307"]},{"key":"dc:type","label":"Dc Type","values":["Text","Thesis"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Rater Gender Bias","Gender Bias in Simulation-Based Assessment"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:rights","label":"Dc Rights","values":["In Copyright","Open Access after 2028-05-01"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["10.25417/uic.32995307.v1"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["Background: Gender bias in graduate medical education assessments may disadvantage female residents. With simulation increasingly used in high-stakes assessments, such as those used to assess emergency medicine residents, understanding potential bias is essential. Objective: To evaluate the presence of gender bias in Simtastic, a large multi-center simulation-based milestone assessment of second year residents to better understand the presence of gender bias in simulation-based assessments. Methods: This retrospective study analyzed 1011 Simtastic scores of PGY2 residents from 2015–2025. Residents were scored using case-specific checklists and global assessments. The data was analyzed for scoring bias based on gender concordance between rater and resident by comparing global assessments to the checklist score on each case. Hierarchical linear modeling accounted for clustering by residency program, site, and rater. Results: Checklist and global assessments (GA) scores were strongly correlated. However, female residents received lower GA and checklist scores than male residents across all years. Male raters awarded higher scores to male residents, while female raters showed no such difference. Over time, GA scores from male raters converged with those from female raters, suggesting reduced inflation with rater experience. Despite this trend, score disparities for female residents persisted. Conclusions: Although female residents consistently received lower scores in Simtastic, such trends likely reflect systemic bias rather than performance differences. Despite male rater bias diminishing over time, inequities remained. Structured rater training and refinement of checklist design are needed to promote equity in high-stakes simulation-based evaluations."]},{"key":"dc:title","label":"Title","values":["Rater Implicit Gender Bias in a Multi-Center Simulation-Based Resident Milestone Assessment"]}]}],"canonical_facts":{"dc:creator":["Clare Desmond (24400265)"],"dc:date":["2026-05-01T00:00:00Z"],"dc:description":["Background: Gender bias in graduate medical education assessments may disadvantage female residents. With simulation increasingly used in high-stakes assessments, such as those used to assess emergency medicine residents, understanding potential bias is essential. Objective: To evaluate the presence of gender bias in Simtastic, a large multi-center simulation-based milestone assessment of second year residents to better understand the presence of gender bias in simulation-based assessments. Methods: This retrospective study analyzed 1011 Simtastic scores of PGY2 residents from 2015–2025. Residents were scored using case-specific checklists and global assessments. The data was analyzed for scoring bias based on gender concordance between rater and resident by comparing global assessments to the checklist score on each case. Hierarchical linear modeling accounted for clustering by residency program, site, and rater. Results: Checklist and global assessments (GA) scores were strongly correlated. However, female residents received lower GA and checklist scores than male residents across all years. Male raters awarded higher scores to male residents, while female raters showed no such difference. Over time, GA scores from male raters converged with those from female raters, suggesting reduced inflation with rater experience. Despite this trend, score disparities for female residents persisted. Conclusions: Although female residents consistently received lower scores in Simtastic, such trends likely reflect systemic bias rather than performance differences. Despite male rater bias diminishing over time, inequities remained. Structured rater training and refinement of checklist design are needed to promote equity in high-stakes simulation-based evaluations."],"dc:identifier":["10.25417/uic.32995307.v1"],"dc:relation":["https://figshare.com/articles/thesis/Rater_Implicit_Gender_Bias_in_a_Multi-Center_Simulation-Based_Resident_Milestone_Assessment/32995307"],"dc:rights":["In Copyright","Open Access after 2028-05-01"],"dc:subject":["Rater Gender Bias","Gender Bias in Simulation-Based Assessment"],"dc:title":["Rater Implicit Gender Bias in a Multi-Center Simulation-Based Resident Milestone Assessment"],"dc:type":["Text","Thesis"]},"updated_at":"2026-07-27T21:33:53Z"}