Engaged employees are better employees. Studies have shown they are more productive, take fewer sick days and have fewer work-related injury or illness. The U.S. Merit Systems Protection Board (MPSB) has found that departments and programs with more engaged employees perform better. Measuring employee engagement is important so that management can take steps to boost overall employee engagement to prescribed levels or to take actions if it begins to slide.
As jobs change, new technology and other factors are introduced into the workplace researchers need to ensure their measurement criteria are up-to-date. Improving the employee engagement scale for civil servants in the Federal workplace was one of the primary goals of a recent study, Text Mining Narrative Responses to Develop Engagement Scale Items.
Until now, developing the engagement scale was primarily based on existing literature and responses to several dozen five-option Agree/Disagree questions. There was no systematic assessment of open-ended questions that were included in the survey. One main factor for this was the size of the survey. In 2005 it was administered to 70,000 employees of whom almost 37,000 responded.
To improve or make the engagement scale more precise researchers identified and analyzed relevant open-ended questions using modern text mining techniques and Provalis Research WordStat text mining software. The purpose of text mining the open-ended questions was to discover additional differences between engaged and less engaged employees that were not being captured in the list of questions. By finding new differences, the team could write rating items that captured them and improve the ability to measure engagement. An improved measure of engagement increases the ability of the scale to measure engagement and its relationship with organizational performance and other variables
The paper discusses how researchers went about their tasks using phrase extraction, applying an exclusion list, stemming or lemmatization and other text mining techniques to improve the engagement scale. You can read the full paper here
[PDF] Text Mining Narrative Survey Responses to Develop Engagement Scale Items
