{"id":{"repo_id":"uiuc","oai_identifier":"oai:www.ideals.illinois.edu:2142/98159"},"canonical_url":"https://search.dev.ndltd.org/etd/uiuc/oai:www.ideals.illinois.edu:2142/98159","repository":{"repo_id":"uiuc","name":"University of Illinois - Urbana-Champaign","base_url":"https://www.ideals.illinois.edu/oai-pmh"},"display":{"title":"Using response times in CAT","abstract":"This Dissertation was approved for publication on 2017-05-30 at 16:05.","abstract_html":"This Dissertation was approved for publication on 2017-05-30 at 16:05.","abstract_has_math":false,"creators":["Kern, Justin L"],"institution":"University of Illinois at Urbana-Champaign","degree_name":"Ph.D.","degree_level":"Dissertation","degree_discipline":"Psychology","degree_department":null,"school":null,"contributors":["Chang, Hua-Hua","Anderson, Carolyn J.","Culpepper, Steven A.","Douglas, Jeffrey A.","Zhang, Jinming"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2017,"date_issued":"2017-09-29T17:45:24Z","date_published":"2017-09-29T17:45:24Z","updated_at":"2026-07-22T22:24:35Z","subjects":["Computer adaptive testing","Response times","Item response theory","Item selection criteria","Bayesian estimation"],"languages":["en"],"rights":["Copyright 2017 Justin L. Kern"],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"http://hdl.handle.net/2142/98159","outbound_label":"Handle","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Chang, Hua-Hua","Anderson, Carolyn J.","Culpepper, Steven A.","Douglas, Jeffrey A.","Zhang, Jinming"]},{"key":"dc:creator","label":"Author","values":["Kern, Justin L"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2017-09-29T17:45:24Z","2020-03-03T10:15:32Z","2017-05-30","2017-08"]},{"key":"dc:type","label":"Dc Type","values":["text"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Psychology"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Dissertation"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Ph.D."]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of Illinois at Urbana-Champaign"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Computer adaptive testing","Response times","Item response theory","Item selection criteria","Bayesian estimation"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["en"]},{"key":"dc:rights","label":"Dc Rights","values":["Copyright 2017 Justin L. Kern"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["http://hdl.handle.net/2142/98159"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["This Dissertation was approved for publication on 2017-05-30 at 16:05.","DSpace SAF Submission Ingestion Package generated from Vireo submission #11184 on 2017-09-29 at 10:45:37","Made available in DSpace on 2017-09-29T17:45:24Z (GMT). No. of bitstreams: 2 KERN-DISSERTATION-2017.pdf: 978404 bytes, checksum: fba9e72d0914396a6f9db2f14d4d6f35 (MD5) LICENSE.txt: 4208 bytes, checksum: c5471fb7d7eebacaeb7628f0f13c66f0 (MD5) Previous issue date: 2017-05-30","Embargo set by: Colleen Fallaw for item 103466 Lift date: 2019-09-29T17:48:06Z Reason: Author requested U of Illinois access only (OA after 2yrs) in Vireo ETD system","Embargo set by: Seth Robbins for item 103466 Lift date: 2020-03-02T19:56:41Z Reason: Author requested U of Illinois access only (OA after 2yrs) in Vireo ETD system","Embargo set by: Seth Robbins for item 103466 Lift date: 2020-03-02T19:59:52Z Reason: Author requested U of Illinois access only (OA after 2yrs) in Vireo ETD system","Embargo set by: Seth Robbins for item 103466 Lift date: 2020-03-02T20:02:46Z Reason: Author requested U of Illinois access only (OA after 2yrs) in Vireo ETD system","U of I Only Restriction Lifted for Item 103466 on 2020-03-03T10:15:32Z.","\"Many areas of psychology and education place a high premium on measurement, using psychometric theory to measure constructs, such as cognitive ability, personality, and attitudes. Some of the more well-known measurement theories used are classical test theory (CTT), structural equation modeling (SEM), and item response theory (IRT). For the practical test construction needs of psychology and education, IRT is the most heavily used, and has been ever since Lord and Novick (1968) published their book, Statistical Theories of Mental Test Scores. One of the biggest advances in IRT has been the advent of computerized adaptive testing (CAT). First introduced as tailored tests by Lord (1980), CATs have increasingly gained in popularity as the cost of computation has gone down. As suggested by the term \"\"tailored tests,\"\" every person who takes an adaptive test takes a test form unique to the person. The test is constructed item-by-item by matching items' difficulty levels to the ability level of that particular person. The promise of CAT is that by constructing a test in this way items that do not contribute much to the overall effectiveness of the measurement are left out, which can shorten the test substantially while still maintaining a high level of measurement accuracy. The efficiency of CAT has not gone unnoticed. The Armed Services Vocational Aptitude Test Battery (ASVAB), which is used for measuring vocationally relevant abilities was originally introduced as a paper-and-pencil test in 1968, and became operational as a CAT in 1996; the ASVAB was the first large-scale, high-stakes operational CAT. Numerous adaptive tests have gone into operational use for use in selection including the Graduate Management Admission Test (GMAT), the Adjustable Competence Evaluation (ACE), the Business Language Testing Service (BULATS) Computer Test, the IBM Selection Tests, among others. Additionally, many licensure exams currently in use—including the Uniform CPA Examination (for certified personal accountants), and the National Council Licensure Examinations (for nurses)—are adaptive. Furthermore, the recently signed Every Student Succeeds Act has recommended a greater use of adaptive testing in the American educational system, allowing states to develop and administer CATs. Computer-based tests, such as CATs, allow for easy collection of response times. With the abundance of essentially-free data, methods and applications for using response time data have become en vogue, though they are still in their infant stage. As such, no large-scale assessments are currently using response times as an active part of the test. Because the data is essentially-free, it is reasonable to believe that their use is simply the next step in the evolution of computer-based tests. Indeed, it only seems natural that CATs be modified to take advantage of response time information, especially since it is well-known that response accuracy and response time are related (Sternberg, 1999). Some applications include cheating detection (van der Linden, 2009a), shortening the time needed to take a test (Choe & Kern, 2014; Fan, Wang, Chang, & Douglas, 2012), and item selection (van der Linden, 2008). The goal of this dissertation is to introduce CAT and some of the current issues surrounding its use, to introduce response times in measurement, and several new methods for using response times in adaptive testing. In the first chapter, a quick review of IRT will be given, including its historical roots, its assumptions, and some examples of commonly used IRT models. Following this will be a brief overview of the basic components of CAT, including item selection, ability (or trait) estimation, and item constraints. I will then discuss response times in measurement and their current role in CAT. In the second chapter, I will describe an already-completed study on estimating person ability and speededness jointly. In Chapter 3, I investigate the efficacy of using the MAP estimator developed in Chapter 2 when selecting items using a generalized time-weighted maximum information criterion (GMICT). In Chapter 4, I introduce a new item selection technique based on the ideas of Bayesian item selection that incorporates the response time model directly. A modified version of this criterion using the ideas from the GMICT is also investigated. In Chapter 5, I introduce a time-weighted Kullback-Leibler information technique and investigate its effectiveness. Finally, I conclude in Chapter 6 with some remarks about how these techniques fit in in the current literature on response times, scoring, and adaptive testing.\"","Submission published under a 24 month embargo labeled 'U of I Access', the embargo will last until 2019-08-01","The student, Justin Kern, accepted the attached license on 2017-05-30 at 11:55.","The student, Justin Kern, submitted this Dissertation for approval on 2017-05-30 at 11:59."]},{"key":"dc:format","label":"Dc Format","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["Using response times in CAT"]}]}],"canonical_facts":{"dc:contributor":["Chang, Hua-Hua","Anderson, Carolyn J.","Culpepper, Steven A.","Douglas, Jeffrey A.","Zhang, Jinming"],"dc:creator":["Kern, Justin L"],"dc:date":["2017-09-29T17:45:24Z","2020-03-03T10:15:32Z","2017-05-30","2017-08"],"dc:description":["This Dissertation was approved for publication on 2017-05-30 at 16:05.","DSpace SAF Submission Ingestion Package generated from Vireo submission #11184 on 2017-09-29 at 10:45:37","Made available in DSpace on 2017-09-29T17:45:24Z (GMT). No. of bitstreams: 2 KERN-DISSERTATION-2017.pdf: 978404 bytes, checksum: fba9e72d0914396a6f9db2f14d4d6f35 (MD5) LICENSE.txt: 4208 bytes, checksum: c5471fb7d7eebacaeb7628f0f13c66f0 (MD5) Previous issue date: 2017-05-30","Embargo set by: Colleen Fallaw for item 103466 Lift date: 2019-09-29T17:48:06Z Reason: Author requested U of Illinois access only (OA after 2yrs) in Vireo ETD system","Embargo set by: Seth Robbins for item 103466 Lift date: 2020-03-02T19:56:41Z Reason: Author requested U of Illinois access only (OA after 2yrs) in Vireo ETD system","Embargo set by: Seth Robbins for item 103466 Lift date: 2020-03-02T19:59:52Z Reason: Author requested U of Illinois access only (OA after 2yrs) in Vireo ETD system","Embargo set by: Seth Robbins for item 103466 Lift date: 2020-03-02T20:02:46Z Reason: Author requested U of Illinois access only (OA after 2yrs) in Vireo ETD system","U of I Only Restriction Lifted for Item 103466 on 2020-03-03T10:15:32Z.","\"Many areas of psychology and education place a high premium on measurement, using psychometric theory to measure constructs, such as cognitive ability, personality, and attitudes. Some of the more well-known measurement theories used are classical test theory (CTT), structural equation modeling (SEM), and item response theory (IRT). For the practical test construction needs of psychology and education, IRT is the most heavily used, and has been ever since Lord and Novick (1968) published their book, Statistical Theories of Mental Test Scores. One of the biggest advances in IRT has been the advent of computerized adaptive testing (CAT). First introduced as tailored tests by Lord (1980), CATs have increasingly gained in popularity as the cost of computation has gone down. As suggested by the term \"\"tailored tests,\"\" every person who takes an adaptive test takes a test form unique to the person. The test is constructed item-by-item by matching items' difficulty levels to the ability level of that particular person. The promise of CAT is that by constructing a test in this way items that do not contribute much to the overall effectiveness of the measurement are left out, which can shorten the test substantially while still maintaining a high level of measurement accuracy. The efficiency of CAT has not gone unnoticed. The Armed Services Vocational Aptitude Test Battery (ASVAB), which is used for measuring vocationally relevant abilities was originally introduced as a paper-and-pencil test in 1968, and became operational as a CAT in 1996; the ASVAB was the first large-scale, high-stakes operational CAT. Numerous adaptive tests have gone into operational use for use in selection including the Graduate Management Admission Test (GMAT), the Adjustable Competence Evaluation (ACE), the Business Language Testing Service (BULATS) Computer Test, the IBM Selection Tests, among others. Additionally, many licensure exams currently in use—including the Uniform CPA Examination (for certified personal accountants), and the National Council Licensure Examinations (for nurses)—are adaptive. Furthermore, the recently signed Every Student Succeeds Act has recommended a greater use of adaptive testing in the American educational system, allowing states to develop and administer CATs. Computer-based tests, such as CATs, allow for easy collection of response times. With the abundance of essentially-free data, methods and applications for using response time data have become en vogue, though they are still in their infant stage. As such, no large-scale assessments are currently using response times as an active part of the test. Because the data is essentially-free, it is reasonable to believe that their use is simply the next step in the evolution of computer-based tests. Indeed, it only seems natural that CATs be modified to take advantage of response time information, especially since it is well-known that response accuracy and response time are related (Sternberg, 1999). Some applications include cheating detection (van der Linden, 2009a), shortening the time needed to take a test (Choe & Kern, 2014; Fan, Wang, Chang, & Douglas, 2012), and item selection (van der Linden, 2008). The goal of this dissertation is to introduce CAT and some of the current issues surrounding its use, to introduce response times in measurement, and several new methods for using response times in adaptive testing. In the first chapter, a quick review of IRT will be given, including its historical roots, its assumptions, and some examples of commonly used IRT models. Following this will be a brief overview of the basic components of CAT, including item selection, ability (or trait) estimation, and item constraints. I will then discuss response times in measurement and their current role in CAT. In the second chapter, I will describe an already-completed study on estimating person ability and speededness jointly. In Chapter 3, I investigate the efficacy of using the MAP estimator developed in Chapter 2 when selecting items using a generalized time-weighted maximum information criterion (GMICT). In Chapter 4, I introduce a new item selection technique based on the ideas of Bayesian item selection that incorporates the response time model directly. A modified version of this criterion using the ideas from the GMICT is also investigated. In Chapter 5, I introduce a time-weighted Kullback-Leibler information technique and investigate its effectiveness. Finally, I conclude in Chapter 6 with some remarks about how these techniques fit in in the current literature on response times, scoring, and adaptive testing.\"","Submission published under a 24 month embargo labeled 'U of I Access', the embargo will last until 2019-08-01","The student, Justin Kern, accepted the attached license on 2017-05-30 at 11:55.","The student, Justin Kern, submitted this Dissertation for approval on 2017-05-30 at 11:59."],"dc:format":["application/pdf"],"dc:identifier":["http://hdl.handle.net/2142/98159"],"dc:language":["en"],"dc:rights":["Copyright 2017 Justin L. Kern"],"dc:subject":["Computer adaptive testing","Response times","Item response theory","Item selection criteria","Bayesian estimation"],"dc:title":["Using response times in CAT"],"dc:type":["text"],"thesis:degree_discipline":["Psychology"],"thesis:degree_level":["Dissertation"],"thesis:degree_name":["Ph.D."],"thesis:institution_name":["University of Illinois at Urbana-Champaign"]},"updated_at":"2026-07-22T22:24:35Z"}