A psychometric assessment of the Irish Civil Service employee engagement scale

This study takes a quantitative approach with the aim to understand whether the employee engagement scale applied in the Civil Service Employee Engagement Survey (CSEES) is an appropriate psychometric measure of employee engagement in the Irish Civil Service. For the purposes of this study, an appropriate employee engagement scale is defined as one that is psychometrically valid, reliable and robust and can provide meaningful quantitative insights into the Irish Civil Service population. The analysis in this study provides insights into psychometric properties of the CSEES employee engagement scale are investigated through the application of several robust psychometric statistical methods. The descriptive statistics and antecedent analysis are reviewed and assessed for the capacity to provide meaningful insights for the Irish Civil Service. The primary focus of the study is the psychometric analyses of the CSEES employee engagement measure. and the antecedents of employee engagement in the Irish Civil Service by grade, gender, and length of service. The CSEES is a census survey with over 60,000 responses across the three iterations of the survey that are available for research purposes. The employee engagement scale used within the CSEES is a modified form of the Utrecht Work Engagement Scale (UWES) (Schaufeli & Bakker, 2004). Psychometric and statistical methods applied are measures of association including Pearson’s correlation coefficients, measures of internal consistency including Cronbach’s alpha, McDonald’s omega and Fornell- Larcker criterion, measures of dimensionality and structure including exploratory and confirmatory analysis and measures of cumulative modelling including multilinear regression analysis, item response theory and differential item functioning. The amount of research of employee engagement within the public sector (Vigoda-Gadot et al., 2013) and the psychometric research of employee engagement scales across all sectors, both at an organisational and demographic level (Bailey et al., 2017) is limited (Kulikowski, 2017). This study contributes to this novel intersection of the research area. The study finds that the CSEES employee engagement scale is psychometrically robust and provides useful insights into the employee engagement of the Irish Civil Service. It is an appropriate measure of employee engagement and identifies five findings of note suggesting further development and research on the CSEES employee engagement would be worthwhile. Firstly, the CSEES employee engagement scale is a robust, valid, and reliable measure of employee engagement in the Irish Civil Service. All items in the scale performed very well when different psychometric methods were applied, apart from one which was “At my work I always persevere, even when things do not go well”. Secondly, IRT analysis suggests that the employee engagement scores in the CSEES may be overinflated, with low to moderate item discrimination and difficulty inherent in the CSEES employee engagement scale. This result suggests that civil servants with lower or more varied employee engagement levels can achieve high employee engagement scores in the CSEES. Thirdly, the CSEES employee engagement scale is unidimensional, in contrast to the UWES which was developed as a multidimensional or three factor scale (Schaufeli & Bakker, 2002). The CSEES employee engagement scale contains items from two of the three UWES dimensions, including dedication and vigour items but not absorption items. Fourthly, the differential item functioning (DIF) analysis found that no DIF effect was likely in the items. However, nominal variation in the outputs by gender suggest that some DIF effect may exist by gender. Fifthly, meaningfulness was the antecedent with the highest strength of association with employee engagement by grade, gender, and length of service by a sizeable margin. The top five antecedents by demographic comprised of six other attitudinal scales including organisational support and job skills match. Although this study does find the CSEES employee engagement scale to be appropriate, a review of the CSEES employee engagement scale is recommended, potentially reducing the number of items, and including items with higher levels of discrimination and more moderate and varied levels of difficulty. Given the small number of antecedents of employee engagement by demographic, the survey itself could also be reduced in scale without any loss of information or relevant insights. This approach would reduce response burden for those surveyed and result in a more accessible survey with an accurate view of the employee engagement of Irish civil servants.

Grantor dc:publisher

University College Cork

Year dc:date.issued

2025

Author dc:creator

• Crilly, Sarah

Advisors dc:contributor.advisor

• Linehan, Carol
• Murphy, Raegan

• Work engagement
• Employee engagement
• Civil service
• Psychometric analysis
• Item response theory
• Factor analysis

Statement dc:rights

• © 2025, Sarah Crilly.

Licence dc:rights.uri

• https://creativecommons.org/licenses/by-nc-nd/4.0/

Language dc:language.iso

en

Handle dc:identifier.uri

https://hdl.handle.net/10468/18420

OAI identifier oai:identifier

oai:cora.ucc.ie:10468/18420