Extending the Utility of Ant Colony Optimization Through the Incorporation of an Intraclass Correlation Coefficient to Assess for Rater Consistency

<p>Ant Colony Optimization (ACO) is a flexible algorithm designed to solve complex combinatorial problems. While the method was derived from the behavior of ants by researchers in the field of computer science, its application to solving complex combinatorial problems is widespread in a growing number of fields in behavioral science, including psychometrics. Over the last two decades, psychometricians have adapted ACO to measurement model specification problems with the intention of generating measurement models that express measurement model fit and reliability within the standards of what is considered acceptable. Additionally, psychometricians have used ACO to generate shortened versions of existing measures while preserving the integrity of other psychometric properties (e.g., model fit, validity, and reliability). The current study sought to extend the utility of ACO by incorporating the intra-class correlation coefficient (ICC) as an optimization criterion in seeking an optimal (or near-optimal) measurement model solution. The introduction of ICC to ACO procedures is intended to address data that features multiple observations of the same targets from multiple raters and was the first of its kind. The study featured a new measure designed to capture the quality of telephonically delivery crisis intervention services, the Multidimensional Crisis Monitoring Form (MCMF-3). Scores from the ACO-derived measurement models were further tested to examine the relationship between the quality of universal screening questions asked by crisis workers, and the subsequent quality of their approach to the crisis management process as captured by the MCMF-3. Implications for future research and use of AI-driven techniques in behavioral health measurement practices are discussed in response to the study results.</p>

Name thesis:degree_name

Ph.D.

Level thesis:degree_level

Dissertation

Year dc:date.available

2024

Author dc:creator

• Leveling, Mark

Contributors dc:contributor

• Yixiao Dong
• Peter Organisciak
• Nick Cutforth
• Stacey Freedenthal

• Ant colony optimization
• Crisis services
• Educational measurement
• Intraclass correlation coefficient
• Item response theory
• Suicide prevention
• Education
• Educational Assessment, Evaluation, and Research
• Other Psychology
• Other Statistics and Probability
• Physical Sciences and Mathematics
• Psychology
• Statistical Models
• Statistics and Probability

Statement dc:rights

• <p>Copyright is held by the author. User is responsible for all copyright compliance.</p>

Language dc:language

English (eng)

Repository record dc:identifier

https://digitalcommons.du.edu/etd/2401

OAI identifier oai:identifier

oai:digitalcommons.du.edu:etd-3384