An Evaluation of the Effect of Treatment Integrity Errors and Observation Conditions on the Accuracy and Reliability of Data Collection

Within the applied domain of behavior analysis (ABA), it is common practice to conduct measurement assessment (e.g., IOA). These measures are often collected as part of an ongoing evaluation of behavioral services and applied research. In other words, critical decisions related to the presumed success or failure of specific ABA interventions and procedures in applied practice and research are based on data that are collected during the course of service delivery and applied research (Vollmer et al., 2008). However, several factors in relation to staff-delivered consequences in the context of applied practice and research may impact the accuracy and reliability of collected data, including the treatment integrity of the services delivered along with the observation conditions of the data collection period. Data that are dependent on staff-delivered consequences, instead of the client behaviors, may result in making data-based decisions that can detrimentally impact the client and the success of the intervention (Vollmer et al., 2008). As such, it is crucial that data that are collected in the context of service delivery and research are both accurate and reliable in relation to the client's/participant's behaviors. The present research evaluated the effects of treatment integrity errors and observation conditions on the accuracy and reliability of data. Participants collected data on scripted videos of role-played client sessions, where we conducted two experiments manipulating: (a) treatment integrity and (b) observation conditions. Results showed that treatment integrity errors alone did not consistently impact data accuracy and data reliability, but that treatment integrity errors when paired with obscured observation conditions detrimentally impacted both the accuracy and reliability of collected data.

Level thesis:degree_level

Doctorate Degree

Year dc:date.issued

2025

Author dc:creator

• Ragulan, Swathi Cheema

Advisor dc:contributor.advisor

• Contreras, Bethany P

Committee members dc:contributor.committeemember

• Lewon, Matthew P
• Petursdottir, Anna I
• Kahng, SungWoo
• Hanna, Alexis

• Accuracy
• Data collection
• Observation conditions
• Reliability
• Treatment integrity

Language dc:language.iso

en_US, English

Repository record dc:identifier.uri

https://scholarwolf.unr.edu/handle/11714/11554

OAI identifier oai:identifier

oai:scholarwolf.unr.edu:11714/11554