{"id":{"repo_id":"uiuc","oai_identifier":"oai:www.ideals.illinois.edu:2142/115512"},"canonical_url":"https://search.dev.ndltd.org/etd/uiuc/oai:www.ideals.illinois.edu:2142/115512","repository":{"repo_id":"uiuc","name":"University of Illinois - Urbana-Champaign","base_url":"https://www.ideals.illinois.edu/oai-pmh"},"display":{"title":"A Bayesian change-point approach to item compromise detection","abstract":"Submission published under a 24 month embargo labeled 'U of I Access', the embargo will last until 2024-05-01","abstract_html":"Submission published under a 24 month embargo labeled 'U of I Access', the embargo will last until 2024-05-01","abstract_has_math":false,"creators":["Du, Yang"],"institution":"University of Illinois at Urbana-Champaign","degree_name":"Ph.D.","degree_level":"Dissertation","degree_discipline":"Educational Psychology","degree_department":null,"school":null,"contributors":["Cromley, Jennifer G","Chang, Hua-Hua","Kern, Justin L","Xia, Yan"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2022,"date_issued":"2022-05","date_published":"2022-05","updated_at":"2026-07-22T22:24:54Z","subjects":["test security","item compromise","continuous tests","Bayesian change-point"],"languages":["en","eng"],"rights":["Copyright 2022 Yang Du."],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"https://hdl.handle.net/2142/115512","outbound_label":"Handle","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Cromley, Jennifer G","Chang, Hua-Hua","Kern, Justin L","Xia, Yan"]},{"key":"dc:creator","label":"Author","values":["Du, Yang"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2022-05","2022-03-10"]},{"key":"dc:type","label":"Dc Type","values":["text","Thesis"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Educational Psychology"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Dissertation"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Ph.D."]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of Illinois at Urbana-Champaign"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["test security","item compromise","continuous tests","Bayesian change-point"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["en","eng"]},{"key":"dc:rights","label":"Dc Rights","values":["Copyright 2022 Yang Du."]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["https://hdl.handle.net/2142/115512"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["Submission published under a 24 month embargo labeled 'U of I Access', the embargo will last until 2024-05-01","The student, Yang Du, accepted the attached license on 2022-03-08 at 12:00.","The student, Yang Du, submitted this Dissertation for approval on 2022-03-08 at 13:30.","This Dissertation was approved for publication on 2022-03-10 at 15:36.","DSpace SAF Submission Ingestion Package generated from Vireo submission #17526 on 2022-11-11 at 11:55:49","Item compromise has been a longstanding issue that threatens test validity and test security in continuous tests. The literature in compromised item detection thus far has been primarily focused on non-Bayesian methods. Most of the existing detection methods monitor the item compromise process at the item level; each item is monitored individually through a summary statistic, such as the correct or incorrect response proportion. What's more, most of the existing detection methods are primarily based on response accuracy and response time information has not been efficiently utilized. In this study, a Bayesian change-point method that incorporates responses and response times is proposed to detect compromised items in continuous tests. In Chapter 1, the background of continuous tests (using computerized tests as an example) and test security is briefly overviewed. The strengths and weaknesses of continuous tests and the challenges that they have imposed on test security will be discussed. Next, the history of utilizing response times in testing as well as why and how response times can help uncover the underlying response process will be presented. Within this context, the problem that this dissertation is trying to resolve, i.e., detecting item compromise in continuous tests with responses and response times, will be clearly stated. The significance of addressing this problem will be discussed as well. In Chapter 2, the methods related to item compromise prevention and detection will be carefully reviewed. With an emphasis on the detection methods, most recent detection methods will be reviewed and synthesized based on the nature of the methods. Meanwhile, the Bayesian change-point framework will also be illustrated. Here, both Phase I and Phase II detection will be demonstrated, with their definitions and application context being explained. Chapter 3 presents a pilot study which aims at employing Bayesian change-point approach to detect the item compromise with responses only. The main purpose of this pilot study is to show that Bayesian change-point approach has great potential in item compromise detection and can be employed to enhance test security. Three simulation studies are conducted to show its performance. Simulation I aims to show the stationary Bayesian change-point model. Specifically, convergence and parameter recovery of the proposed Metropolis-Hastings-within-Gibbs algorithms with respect to the change-point model are evaluated. Simulation II demonstrates the performance of Shiryaev procedure in detecting compromised items in real time while using true person parameters. Simulation III also presents the performance of Shiryaev procedure, but using the EAP estimates of latent ability parameters. In Chapter 4, the proposed Bayesian change-point model with responses and response times and the Shiryaev procedure are explained in detail. The simulation designs in conjunction with evaluation criteria are explained as well. Chapter 5 presents the results of the three simulations. Convergence plot and parameter recovery of the Bayesian change-point model are presented in simulation I. Detection accuracy and efficiency are presented for simulation II. The comparison of the Bayesian change-point method with and without RT is presented in simulation III. Finally, the conclusions of the current study are presented in Chapter 6. Limitations as well as the directions for future studies are also discussed."]},{"key":"dc:format","label":"Dc Format","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["A Bayesian change-point approach to item compromise detection"]}]}],"canonical_facts":{"dc:contributor":["Cromley, Jennifer G","Chang, Hua-Hua","Kern, Justin L","Xia, Yan"],"dc:creator":["Du, Yang"],"dc:date":["2022-05","2022-03-10"],"dc:description":["Submission published under a 24 month embargo labeled 'U of I Access', the embargo will last until 2024-05-01","The student, Yang Du, accepted the attached license on 2022-03-08 at 12:00.","The student, Yang Du, submitted this Dissertation for approval on 2022-03-08 at 13:30.","This Dissertation was approved for publication on 2022-03-10 at 15:36.","DSpace SAF Submission Ingestion Package generated from Vireo submission #17526 on 2022-11-11 at 11:55:49","Item compromise has been a longstanding issue that threatens test validity and test security in continuous tests. The literature in compromised item detection thus far has been primarily focused on non-Bayesian methods. Most of the existing detection methods monitor the item compromise process at the item level; each item is monitored individually through a summary statistic, such as the correct or incorrect response proportion. What's more, most of the existing detection methods are primarily based on response accuracy and response time information has not been efficiently utilized. In this study, a Bayesian change-point method that incorporates responses and response times is proposed to detect compromised items in continuous tests. In Chapter 1, the background of continuous tests (using computerized tests as an example) and test security is briefly overviewed. The strengths and weaknesses of continuous tests and the challenges that they have imposed on test security will be discussed. Next, the history of utilizing response times in testing as well as why and how response times can help uncover the underlying response process will be presented. Within this context, the problem that this dissertation is trying to resolve, i.e., detecting item compromise in continuous tests with responses and response times, will be clearly stated. The significance of addressing this problem will be discussed as well. In Chapter 2, the methods related to item compromise prevention and detection will be carefully reviewed. With an emphasis on the detection methods, most recent detection methods will be reviewed and synthesized based on the nature of the methods. Meanwhile, the Bayesian change-point framework will also be illustrated. Here, both Phase I and Phase II detection will be demonstrated, with their definitions and application context being explained. Chapter 3 presents a pilot study which aims at employing Bayesian change-point approach to detect the item compromise with responses only. The main purpose of this pilot study is to show that Bayesian change-point approach has great potential in item compromise detection and can be employed to enhance test security. Three simulation studies are conducted to show its performance. Simulation I aims to show the stationary Bayesian change-point model. Specifically, convergence and parameter recovery of the proposed Metropolis-Hastings-within-Gibbs algorithms with respect to the change-point model are evaluated. Simulation II demonstrates the performance of Shiryaev procedure in detecting compromised items in real time while using true person parameters. Simulation III also presents the performance of Shiryaev procedure, but using the EAP estimates of latent ability parameters. In Chapter 4, the proposed Bayesian change-point model with responses and response times and the Shiryaev procedure are explained in detail. The simulation designs in conjunction with evaluation criteria are explained as well. Chapter 5 presents the results of the three simulations. Convergence plot and parameter recovery of the Bayesian change-point model are presented in simulation I. Detection accuracy and efficiency are presented for simulation II. The comparison of the Bayesian change-point method with and without RT is presented in simulation III. Finally, the conclusions of the current study are presented in Chapter 6. Limitations as well as the directions for future studies are also discussed."],"dc:format":["application/pdf"],"dc:identifier":["https://hdl.handle.net/2142/115512"],"dc:language":["en","eng"],"dc:rights":["Copyright 2022 Yang Du."],"dc:subject":["test security","item compromise","continuous tests","Bayesian change-point"],"dc:title":["A Bayesian change-point approach to item compromise detection"],"dc:type":["text","Thesis"],"thesis:degree_discipline":["Educational Psychology"],"thesis:degree_level":["Dissertation"],"thesis:degree_name":["Ph.D."],"thesis:institution_name":["University of Illinois at Urbana-Champaign"]},"updated_at":"2026-07-22T22:24:54Z"}