{"id":{"repo_id":"uiuc","oai_identifier":"oai:www.ideals.illinois.edu:2142/116058"},"canonical_url":"https://search.dev.ndltd.org/etd/uiuc/oai:www.ideals.illinois.edu:2142/116058","repository":{"repo_id":"uiuc","name":"University of Illinois - Urbana-Champaign","base_url":"https://www.ideals.illinois.edu/oai-pmh"},"display":{"title":"Detecting examinees with item pre-knowledge in computer based large-scale assessment","abstract":"Submission published under a 24 month embargo labeled 'U of I Access', the embargo will last until 2024-08-01","abstract_html":"Submission published under a 24 month embargo labeled 'U of I Access', the embargo will last until 2024-08-01","abstract_has_math":false,"creators":["Demirkaya, Onur"],"institution":"University of Illinois at Urbana-Champaign","degree_name":"Ph.D.","degree_level":"Dissertation","degree_discipline":"Educational Psychology","degree_department":null,"school":null,"contributors":["Zhang, Jinming","Chang, Hua-hua","Anderson, Carolyn J","Jiang, Ge"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2022,"date_issued":"2022-08","date_published":"2022-08","updated_at":"2026-07-22T22:24:55Z","subjects":["gradient test","item revisit","response time","change point analysis","person fit","Item pre-knowledge","compromised items","test security","likelihood ratio test","score test"],"languages":["en","eng"],"rights":["Copyright 2022 Onur Demirkaya"],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"https://hdl.handle.net/2142/116058","outbound_label":"Handle","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Zhang, Jinming","Chang, Hua-hua","Anderson, Carolyn J","Jiang, Ge"]},{"key":"dc:creator","label":"Author","values":["Demirkaya, Onur"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2022-08","2022-07-07"]},{"key":"dc:type","label":"Dc Type","values":["text","Thesis"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Educational Psychology"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Dissertation"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Ph.D."]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of Illinois at Urbana-Champaign"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["gradient test","item revisit","response time","change point analysis","person fit","Item pre-knowledge","compromised items","test security","likelihood ratio test","score test"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["en","eng"]},{"key":"dc:rights","label":"Dc Rights","values":["Copyright 2022 Onur Demirkaya"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["https://hdl.handle.net/2142/116058"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["Submission published under a 24 month embargo labeled 'U of I Access', the embargo will last until 2024-08-01","The student, Onur Demirkaya, accepted the attached license on 2022-07-06 at 21:53.","The student, Onur Demirkaya, submitted this Dissertation for approval on 2022-07-06 at 22:48.","This Dissertation was approved for publication on 2022-07-07 at 16:07.","DSpace SAF Submission Ingestion Package generated from Vireo submission #18161 on 2022-11-15 at 19:17:08","Item pre-knowledge, one of the most frequently encountered aberrant testing behaviors in high-stakes assessment, jeopardizes test score integrity, validity, and fairness by artificially inflating the test scores of its beneficiaries, examinees who have access to some test items before taking the test. To maintain test score integrity, there is increasing interest in developing statistical methods to detect item pre-knowledge by utilizing examinee responses and response process information. One of those methods is the constrained likelihood ratio test, which is the only frequentist approach specifically designed to detect item pre-knowledge (Sinharay & Johnson, 2019). This method assumes that a set of compromised items is precisely known by investigators. However, different subsets of items can be compromised for different groups of examinees, and statistical detection procedures cannot confidently identify all compromised items. Therefore, in Chapter 2, two new frequentist statistics, namely the constrained score test and the constrained gradient test, which rely on responses and response times to detect item pre-knowledge were proposed and their performance was investigated in situations of uncertainty about suspicious items through simulation studies for both adaptive and linear tests. A real data analysis was conducted to show the usefulness of the statistics in the operational setting. The proposed statistics were found to be as powerful as the existing statistic. With the availability of process data collected in computer-based assessments, research on detecting item pre-knowledge using both item scores and response times has progressed. Examinees’ item-revisit patterns can also be utilized as an additional source of information. In Chapter 3, a novel statistic was proposed for detecting item pre-knowledge when compromised items are known by utilizing the hierarchical speed accuracy revisits model. By simultaneously evaluating abnormal changes in the latent abilities, speeds, and revisit propensities of examinees, the statistic was found to provide greater statistical power and stronger substantive evidence that an examinee had indeed benefitted from item pre-knowledge than existing statistics. Change point analysis (CPA), a statistical process control method, has recently been applied to psychometric problems including the detection of aberrant behaviors. Existing CPA methods, however, have not been specifically designed to detect item pre-knowledge but can be applied only after reordering the test items based on the information of compromised item status, which is not always readily available. In Chapter 4, therefore, the use of item usage or exposure rate was proposed to reorder data along with one residual-based and three modified likelihood-based statistics relying on both response and response time information to detect item pre-knowledge. Comprehensive simulation studies conducted with both linear and adaptive tests showed that the proposed statistics were powerful enough to flag aberrant examinees exhibiting different change point locations when there was an association between item usage and item status."]},{"key":"dc:format","label":"Dc Format","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["Detecting examinees with item pre-knowledge in computer based large-scale assessment"]}]}],"canonical_facts":{"dc:contributor":["Zhang, Jinming","Chang, Hua-hua","Anderson, Carolyn J","Jiang, Ge"],"dc:creator":["Demirkaya, Onur"],"dc:date":["2022-08","2022-07-07"],"dc:description":["Submission published under a 24 month embargo labeled 'U of I Access', the embargo will last until 2024-08-01","The student, Onur Demirkaya, accepted the attached license on 2022-07-06 at 21:53.","The student, Onur Demirkaya, submitted this Dissertation for approval on 2022-07-06 at 22:48.","This Dissertation was approved for publication on 2022-07-07 at 16:07.","DSpace SAF Submission Ingestion Package generated from Vireo submission #18161 on 2022-11-15 at 19:17:08","Item pre-knowledge, one of the most frequently encountered aberrant testing behaviors in high-stakes assessment, jeopardizes test score integrity, validity, and fairness by artificially inflating the test scores of its beneficiaries, examinees who have access to some test items before taking the test. To maintain test score integrity, there is increasing interest in developing statistical methods to detect item pre-knowledge by utilizing examinee responses and response process information. One of those methods is the constrained likelihood ratio test, which is the only frequentist approach specifically designed to detect item pre-knowledge (Sinharay & Johnson, 2019). This method assumes that a set of compromised items is precisely known by investigators. However, different subsets of items can be compromised for different groups of examinees, and statistical detection procedures cannot confidently identify all compromised items. Therefore, in Chapter 2, two new frequentist statistics, namely the constrained score test and the constrained gradient test, which rely on responses and response times to detect item pre-knowledge were proposed and their performance was investigated in situations of uncertainty about suspicious items through simulation studies for both adaptive and linear tests. A real data analysis was conducted to show the usefulness of the statistics in the operational setting. The proposed statistics were found to be as powerful as the existing statistic. With the availability of process data collected in computer-based assessments, research on detecting item pre-knowledge using both item scores and response times has progressed. Examinees’ item-revisit patterns can also be utilized as an additional source of information. In Chapter 3, a novel statistic was proposed for detecting item pre-knowledge when compromised items are known by utilizing the hierarchical speed accuracy revisits model. By simultaneously evaluating abnormal changes in the latent abilities, speeds, and revisit propensities of examinees, the statistic was found to provide greater statistical power and stronger substantive evidence that an examinee had indeed benefitted from item pre-knowledge than existing statistics. Change point analysis (CPA), a statistical process control method, has recently been applied to psychometric problems including the detection of aberrant behaviors. Existing CPA methods, however, have not been specifically designed to detect item pre-knowledge but can be applied only after reordering the test items based on the information of compromised item status, which is not always readily available. In Chapter 4, therefore, the use of item usage or exposure rate was proposed to reorder data along with one residual-based and three modified likelihood-based statistics relying on both response and response time information to detect item pre-knowledge. Comprehensive simulation studies conducted with both linear and adaptive tests showed that the proposed statistics were powerful enough to flag aberrant examinees exhibiting different change point locations when there was an association between item usage and item status."],"dc:format":["application/pdf"],"dc:identifier":["https://hdl.handle.net/2142/116058"],"dc:language":["en","eng"],"dc:rights":["Copyright 2022 Onur Demirkaya"],"dc:subject":["gradient test","item revisit","response time","change point analysis","person fit","Item pre-knowledge","compromised items","test security","likelihood ratio test","score test"],"dc:title":["Detecting examinees with item pre-knowledge in computer based large-scale assessment"],"dc:type":["text","Thesis"],"thesis:degree_discipline":["Educational Psychology"],"thesis:degree_level":["Dissertation"],"thesis:degree_name":["Ph.D."],"thesis:institution_name":["University of Illinois at Urbana-Champaign"]},"updated_at":"2026-07-22T22:24:55Z"}