Statistical Software Usability for Novice Doctoral Students in the Social Sciences

Many social science doctoral students need to be able to conduct quantitative data analysis. Despite the wide availability of statistical software tools, these students struggle to master statistical data analysis and exhibit low ability in using these tools. This thesis tackles the possibility that the design of statistical software systems may explain much of the difficulty that students encounter when conducting statistical data analysis. It is argued that these tools may not be well-designed for non-statisticians or novice and infrequent users. To explore this issue, three interrelated studies were carried out. The first stage of the research aimed to establish a New Zealand-wide baseline for the severity and nature of the problems related to the use of statistical software by social science doctoral students. To this end, an online survey was used to explore attitudinal factors related to statistics and statistical software (Study 1). Doctoral students were generally in agreement that using technology/statistical software benefits statistics learning; however, they reported lower confidence in conducting statistical analysis using statistical software. They demonstrated low levels of competence in understanding statistical concepts, software fluency, and the use of statistical software. Study 2 aimed at evaluating the suitability for novices of four software tools (SPSS, R within RStudio Desktop, R Commander, & jamovi) from a human-computer interaction perspective. The features studied included technical properties such as the user interface design, the range of statistical operations included in the software, and data handling features including preparation, manipulation, and visualization. Usage properties were also studied, including the speed/number of steps to a solution, ease of command/use, and efficiency. The results indicated that usability factors from an interaction perspective are likely to be especially important for students gaining an introductory knowledge of statistics. Of the software evaluated, jamovi exhibited the strongest potential for novices or infrequent users. The design features of jamovi, such as immediate bidirectional responsiveness, minimalist and ready to publish output, and permanent records of analysis, make it an easy-to-use statistical software suitable for non-experts. Study 3 was an experiment that explored the usability of the software systems and user performance when solving a linear regression problem. The results indicated that jamovi elicited better task performance followed by SPSS. To gain a deeper insight into the usability of jamovi and SPSS and to identify the key design features that help or hinder users' task performance, eye tracking metrics and indicators were examined. Performance was evaluated by (1) quality of task completion and (2) how long participants took to find the correct menu commands to initiate the linear regression, as well as what they gazed upon and for how long. Eye tracking was conducted using the Tobii Pro Spectrum eye tracker. Upon task completion, each participant "thought aloud" as they watched a video of their own performance. The results showed that performance quality was greater in jamovi than SPSS (75% vs. 42%); however, this was not beyond chance (χ2(1) = 2.74, p = .10). Mean score for task completion, however, shows a very large effect size in favor of jamovi (d = 0.998 [95% CI: 0.15-1.85]). jamovi required less effort of participants with respect to speed, fixation counts, and gaze duration than SPSS (d > 0.60). Heat maps of gaze time and location showed that in jamovi the more restricted range of functions supported success compared to the abundance of options provided in SPSS. Thematically analyzed stimulated recall interviews confirmed that simple menus, rather than complex embedded and multiple functions, helped beginners. Further, the automatic display of results on the same screen in jamovi was considered a substantial advantage. These findings reinforce previous claims regarding the low confidence of research students in non-STEM fields to do and use statistics. The findings of this thesis provide conceptual and empirical evidence as to the importance of the design features of statistical software tools, particularly for end-users such as doctoral-level social science students who are not proficient in statistics or statistical software. Certain helpful features of a software system can compensate for students' low confidence by making it easier to conduct data analysis. This thesis reinforces the eye-mind assumption by illustrating what a user fixates their eye on and how their visual attention on the statistical software interface contributes to statistical task performance by the use of better-designed software. It is evident that software can, on its own, contribute to the successful completion of statistical research.

Name thesis:degree_name

PhD

Level thesis:degree_level

Doctoral

Discipline thesis:degree_discipline

Education

Grantor dc:publisher

ResearchSpace@Auckland

Year dc:date.issued

2022

Author dc:creator

• Abbasnasab Sardareh, Sedigheh

Advisors dc:contributor.advisor

• Brown, Gavin T. L.
• Denny, Paul

Statement dc:rights

• Items in ResearchSpace are protected by copyright, with all rights reserved, unless otherwise indicated.

Licence dc:rights.uri

• https://researchspace.auckland.ac.nz/docs/uoa-docs/rights.htm

Handle dc:identifier.uri

https://hdl.handle.net/2292/62687

OAI identifier oai:identifier

oai:researchspace.auckland.ac.nz:2292/62687