{"id":{"repo_id":"uiuc","oai_identifier":"oai:www.ideals.illinois.edu:2142/122044"},"canonical_url":"https://search.dev.ndltd.org/etd/uiuc/oai:www.ideals.illinois.edu:2142/122044","repository":{"repo_id":"uiuc","name":"University of Illinois - Urbana-Champaign","base_url":"https://www.ideals.illinois.edu/oai-pmh"},"display":{"title":"IRTree selection validity in presence of extreme response styles","abstract":"Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2024-03-01 without embargo terms","abstract_html":"Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2024-03-01 without embargo terms","abstract_has_math":false,"creators":["Quirk, Victoria L."],"institution":"University of Illinois at Urbana-Champaign","degree_name":"M.S.","degree_level":"Thesis","degree_discipline":"Educational Psychology","degree_department":null,"school":null,"contributors":["Kern, Justin L.","Zhang, Jinming"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2023,"date_issued":"2023-12","date_published":"2023-12","updated_at":"2026-07-22T22:25:00Z","subjects":["Item Response Theory","Item Response Trees","Irtree Models","Response Process","Non-content Variability","Extreme Response Styles","Likert Scales","Selection Validity","Classification Accuracy","Adverse Impact"],"languages":["en","eng"],"rights":["Copyright 2023 Victoria Quirk"],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"https://hdl.handle.net/2142/122044","outbound_label":"Handle","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Kern, Justin L.","Zhang, Jinming"]},{"key":"dc:creator","label":"Author","values":["Quirk, Victoria L."]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2023-12","2023-12-04"]},{"key":"dc:type","label":"Dc Type","values":["text"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Educational Psychology"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Thesis"]},{"key":"thesis:degree_name","label":"Degree Name","values":["M.S."]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of Illinois at Urbana-Champaign"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Item Response Theory","Item Response Trees","Irtree Models","Response Process","Non-content Variability","Extreme Response Styles","Likert Scales","Selection Validity","Classification Accuracy","Adverse Impact"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["en","eng"]},{"key":"dc:rights","label":"Dc Rights","values":["Copyright 2023 Victoria Quirk"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["https://hdl.handle.net/2142/122044"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2024-03-01 without embargo terms","The student, Victoria Quirk, accepted the attached license on 2023-11-30 at 13:31.","The student, Victoria Quirk, submitted this Thesis for approval on 2023-11-30 at 13:41.","This Thesis was approved for publication on 2023-12-04 at 11:57.","DSpace SAF Submission Ingestion Package generated from Vireo submission #20088 on 2024-03-01 at 13:15:19","The measurement of psychological constructs is frequently based on self-report tests, often with Likert-type items rated from “Strongly Disagree” to “Strongly Agree.” However, previous research has suggested that responses to these types of items are often not solely a function of the content trait of interest, but also of other systematic response tendencies due to item format. These tendencies, called response styles, can introduce noise into the measurement of a content trait. Previous research also demonstrates demographic group differences in response styles, potentially introducing bias into the scoring of tests. In recent years, a family of item response theory (IRT) models called IRTree models have been developed that can allow researchers to parse out content traits (e.g., personality traits) from noise traits (e.g., response styles). IRTree models, in which IRT models are organized into a decision tree structure, allow for unique parameters at each node of a theorized response process. While there has been research in evaluating the efficacy of IRTree models in modeling response processes in comparison to other models, there are few studies regarding the practical implications of decisions made based on this structured model. In this study, we analyze the selection validity and adverse impact of classification decisions made based on an IRTree model that controls for response style tendencies compared to an IRT model that does not control for response style tendencies. Here, we consider situations where respondents may display an extreme response style, or a tendency to systematically prefer extreme response categories (e.g., “Strongly Disagree” or “Strongly Agree”), though the approach could be expanded to use for any theorized response process. We first present a simulation in which we demonstrate that in cases where extreme response styles do exist, the IRTree model has greater selection validity than the generalized partial credit model (GPCM) while in situations where extreme response styles do not exist, the models perform nearly equally well. We also find that, in situations where extreme response styles do exist, the IRTree model presents far fewer instances of adverse impact. Additionally, we demonstrate that both models are robust to item-level directional invariance (i.e., cases where the threshold to endorse an extreme response category is dependent upon the agree or disagree direction). To consider the practical applicability of our findings, we also present an example using real data collected from the Open-Source Psychometrics Project Fisher Temperament Inventory dataset. We found the IRTree model to have better fit, but little variation in the selection decision made by each model; thus, adverse impact was virtually identical between the two models. We offer considerations for practical application of the IRTree model and future directions for research."]},{"key":"dc:format","label":"Dc Format","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["IRTree selection validity in presence of extreme response styles"]}]}],"canonical_facts":{"dc:contributor":["Kern, Justin L.","Zhang, Jinming"],"dc:creator":["Quirk, Victoria L."],"dc:date":["2023-12","2023-12-04"],"dc:description":["Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2024-03-01 without embargo terms","The student, Victoria Quirk, accepted the attached license on 2023-11-30 at 13:31.","The student, Victoria Quirk, submitted this Thesis for approval on 2023-11-30 at 13:41.","This Thesis was approved for publication on 2023-12-04 at 11:57.","DSpace SAF Submission Ingestion Package generated from Vireo submission #20088 on 2024-03-01 at 13:15:19","The measurement of psychological constructs is frequently based on self-report tests, often with Likert-type items rated from “Strongly Disagree” to “Strongly Agree.” However, previous research has suggested that responses to these types of items are often not solely a function of the content trait of interest, but also of other systematic response tendencies due to item format. These tendencies, called response styles, can introduce noise into the measurement of a content trait. Previous research also demonstrates demographic group differences in response styles, potentially introducing bias into the scoring of tests. In recent years, a family of item response theory (IRT) models called IRTree models have been developed that can allow researchers to parse out content traits (e.g., personality traits) from noise traits (e.g., response styles). IRTree models, in which IRT models are organized into a decision tree structure, allow for unique parameters at each node of a theorized response process. While there has been research in evaluating the efficacy of IRTree models in modeling response processes in comparison to other models, there are few studies regarding the practical implications of decisions made based on this structured model. In this study, we analyze the selection validity and adverse impact of classification decisions made based on an IRTree model that controls for response style tendencies compared to an IRT model that does not control for response style tendencies. Here, we consider situations where respondents may display an extreme response style, or a tendency to systematically prefer extreme response categories (e.g., “Strongly Disagree” or “Strongly Agree”), though the approach could be expanded to use for any theorized response process. We first present a simulation in which we demonstrate that in cases where extreme response styles do exist, the IRTree model has greater selection validity than the generalized partial credit model (GPCM) while in situations where extreme response styles do not exist, the models perform nearly equally well. We also find that, in situations where extreme response styles do exist, the IRTree model presents far fewer instances of adverse impact. Additionally, we demonstrate that both models are robust to item-level directional invariance (i.e., cases where the threshold to endorse an extreme response category is dependent upon the agree or disagree direction). To consider the practical applicability of our findings, we also present an example using real data collected from the Open-Source Psychometrics Project Fisher Temperament Inventory dataset. We found the IRTree model to have better fit, but little variation in the selection decision made by each model; thus, adverse impact was virtually identical between the two models. We offer considerations for practical application of the IRTree model and future directions for research."],"dc:format":["application/pdf"],"dc:identifier":["https://hdl.handle.net/2142/122044"],"dc:language":["en","eng"],"dc:rights":["Copyright 2023 Victoria Quirk"],"dc:subject":["Item Response Theory","Item Response Trees","Irtree Models","Response Process","Non-content Variability","Extreme Response Styles","Likert Scales","Selection Validity","Classification Accuracy","Adverse Impact"],"dc:title":["IRTree selection validity in presence of extreme response styles"],"dc:type":["text"],"thesis:degree_discipline":["Educational Psychology"],"thesis:degree_level":["Thesis"],"thesis:degree_name":["M.S."],"thesis:institution_name":["University of Illinois at Urbana-Champaign"]},"updated_at":"2026-07-22T22:25:00Z"}