This study examines the proposed Reliability Generalization (RG) method for studying reliability. RG employs the application of meta-analytic techniques similar to those used in validity generalization studies to examine reliability coefficients. This study explains why RG does not provide a proper research method for the study of reliability, including describing how reliability is not a singular metric but a family of coefficients that are not interchangeable, along with other issues, such as sample and test administration. This research used Monte Carlo simulations designed to illustrate how the same instrument, administered repeatedly, can result in different reliability coefficients and to show that variation in reliability coefficients is due to sampling error; results illustrate that the reliability of a test will vary across test administrations based on the size and composition of the sample and how the sample was selected (randomly versus non-randomly).
","abstract_html":"<p>This study examines the proposed Reliability Generalization (RG) method for studying reliability. RG employs the application of meta-analytic techniques similar to those used in validity generalization studies to examine reliability coefficients. This study explains why RG does not provide a proper research method for the study of reliability, including describing how reliability is not a singular metric but a family of coefficients that are not interchangeable, along with other issues, such as sample and test administration. This research used Monte Carlo simulations designed to illustrate how the same instrument, administered repeatedly, can result in different reliability coefficients and to show that variation in reliability coefficients is due to sampling error; results illustrate that the reliability of a test will vary across test administrations based on the size and composition of the sample and how the sample was selected (randomly versus non-randomly).</p>","abstract_has_math":false,"creators":["Smith, Julie Marilyn"],"institution":null,"degree_name":"Ph.D.","degree_level":"Open Access Dissertation","degree_discipline":"Education Evaluation and Research","degree_department":null,"school":null,"contributors":["Shlomo S. Sawilowsky"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2011,"date_issued":"2011-01-01T08:00:00Z","date_published":"2011-01-01T08:00:00Z","updated_at":"2026-07-24T05:58:57Z","subjects":["Measurement, Monte Carlo, Psychometrics, Reliability, Reliability generalization, Sampling error","Educational Assessment, Evaluation, and Research"],"languages":[],"rights":[],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"https://digitalcommons.wayne.edu/oa_dissertations/396","outbound_label":"Repository record","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Shlomo S. Sawilowsky"]},{"key":"dc:creator","label":"Author","values":["Smith, Julie Marilyn"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date.available","label":"Dc Date Available","values":["2011-01-01T08:00:00Z"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Education Evaluation and Research"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Open Access Dissertation"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Ph.D."]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Measurement, Monte Carlo, Psychometrics, Reliability, Reliability generalization, Sampling error","Educational Assessment, Evaluation, and Research"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["https://digitalcommons.wayne.edu/oa_dissertations/396"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description.abstract","label":"Abstract","values":["This study examines the proposed Reliability Generalization (RG) method for studying reliability. RG employs the application of meta-analytic techniques similar to those used in validity generalization studies to examine reliability coefficients. This study explains why RG does not provide a proper research method for the study of reliability, including describing how reliability is not a singular metric but a family of coefficients that are not interchangeable, along with other issues, such as sample and test administration. This research used Monte Carlo simulations designed to illustrate how the same instrument, administered repeatedly, can result in different reliability coefficients and to show that variation in reliability coefficients is due to sampling error; results illustrate that the reliability of a test will vary across test administrations based on the size and composition of the sample and how the sample was selected (randomly versus non-randomly).
"]},{"key":"dc:title","label":"Title","values":["Reliability generalization: lapsus linguae"]}]}],"canonical_facts":{"dc:contributor":["Shlomo S. Sawilowsky"],"dc:creator":["Smith, Julie Marilyn"],"dc:date.available":["2011-01-01T08:00:00Z"],"dc:description.abstract":["This study examines the proposed Reliability Generalization (RG) method for studying reliability. RG employs the application of meta-analytic techniques similar to those used in validity generalization studies to examine reliability coefficients. This study explains why RG does not provide a proper research method for the study of reliability, including describing how reliability is not a singular metric but a family of coefficients that are not interchangeable, along with other issues, such as sample and test administration. This research used Monte Carlo simulations designed to illustrate how the same instrument, administered repeatedly, can result in different reliability coefficients and to show that variation in reliability coefficients is due to sampling error; results illustrate that the reliability of a test will vary across test administrations based on the size and composition of the sample and how the sample was selected (randomly versus non-randomly).
"],"dc:identifier":["https://digitalcommons.wayne.edu/oa_dissertations/396"],"dc:subject":["Measurement, Monte Carlo, Psychometrics, Reliability, Reliability generalization, Sampling error","Educational Assessment, Evaluation, and Research"],"dc:title":["Reliability generalization: lapsus linguae"],"thesis:degree_discipline":["Education Evaluation and Research"],"thesis:degree_level":["Open Access Dissertation"],"thesis:degree_name":["Ph.D."]},"updated_at":"2026-07-24T05:58:57Z"}