Test Construction Project
With the prevalence of mental health issues in the adult working population, interventions to promote mindfulness and meditation in the workplace have been implemented by many organizations. However, after thorough research, we found that there was no existing scale that measured an individual’s agency to practice mental health self-care behaviors while at work. Therefore, this project was deployed to create a validated measure of Mental Health Self-Care Agency in the Workplace. The following page outlines the process of creating this scale and includes the final assessment and test manual.
Project was completed in collaboration with Dylan Sakamoto, Claire Henderson, Isabella Di Lauro, and Reyna White
Mental Health Self-Care Agency in the Workplace
Construct Identification and Definition
The goal of this measure was to assess the agency one has to practice specific behaviors that have been supported to improve mental health in employees so that appropriate interventions or training can be properly utilized to ensure employee mental health. There was a noticeable gap in the literature on self-care agency that focuses particularly on mental health. Understanding the impact of mental health issues on individuals and organizations, it seemed imperative to examine workplace habits that establish good mental health
​
Mindfulness and meditation were shown to have significant effects on reducing stress and burnout in addition to improving performance in the workplace and as such they were chosen as the specific constructs to be used to represent mental health in our scale. We operationalized the definition of both factors specifically to:
• Mindfulness: The act of being mentally present when doing tasks and being aware of one’s thoughts.
• Meditation: A mental practice that encourages attention and awareness towards a mental state of calm concentration and positive emotions.
Test Domain
Because these work stressors differ from those encountered in personal life and knowing that our group members are all entering the field of Industrial-Organizational Psychology, we chose to specify the context of our scale to the workplace.
​
While our construct is specific to mindfulness and meditation as representatives of mental health, it falls under the umbrella construct of self-care agency. The first step we took to ensure our construct is differentiated from the related construct of self-care agency was to develop clear and specific operationalized definitions of mindfulness and meditation.
Item Writing
A 5-point Likert Scale format was decided upon in order to minimize the expertise required to administer the test and to make final analyses easier with quantifiable data.
​
In line with our Rational Test Development approach, items were written with our specific construct and factors in mind, using previous research on Self-care Agency and our operationalized definitions as reference. Guidelines for good item writing as put forth in our textbook were followed, including practices such as avoiding double-barreled and leading items in addition to writing simple items that measure present behavior to create an initial scale of 21 items for Mindfulness and 17 items for Meditation.
​
SME Evaluations
We worked with a set of 7 SMEs that have varying levels of theoretical knowledge of and practical experience in Mindfulness and Meditation to provide a robust range of expertise for our scale.
​
After SMEs rated each potential scale item on relevance to the construct and provided feedback on the constructs, ratings were examined to determine a cut score. This process ensured tha items maintained in the final scale were rated above the average SME rating. This left our scale at a total of 20 items, 10 each under Mindfulness and Meditation.
Pilot Testing
There was a total of 168 valid participants in our pilot sample after filtering out responses that were less than 95% complete, all recruited through convenience sampling.
Participants were presented with the 20 item scale to be rated on a 5-point agreement Likert Scale (strongly disagree to strongly agree). Additionally, participants completed 15-items from the Appraisal of Self-Care Agency Revised (ASAS-R) to evaluate construct validity. Demographic information was collected from all participants including age, gender identity, race, and current work hours.
Data Analyses
Following the Rational Test Development process, an exploratory factor analysis (EFA) on the dimensionality of the scale was completed to gain insight on the operationalization of the underlying construct. The results of the EFA led to the elimination of several items based on correlation and extraction values of several run analyses. This led to an clean loading of items into two factors that aligned with our sub-constructs of mindfulness and meditation.
​
Additionally, Reliability and Validation tests were completed to ensure the final scale was internally reliable and had adequate evidence to support the Construct, Content, and Criterion-Related Validity of items.
Final Scale
The final scale definition is: The Mental Health Self-care Agency scale measures the specific behaviors and intentions of practicing mental health related self-care in the domains of mindfulness and meditation, specifically in the context of the workplace.
The final version of the scale had a total of 14 items in two distinct factors. 9 items fall under the construct of Mindfulness and 5 items hang under Meditation
Reflection
Successes
Throughout my academic career, I have frequently used validated measures in research to assess behaviors, attitudes, and perceptions. To be able to create my own validated measure was an invaluable experience to not only gain competence in test construction, but to further my understanding of the operationalization of psychological concepts. By working with Subject Matter Experts to validate the scale, I was able to practice my professional collaboration skills that can transfer into my career in Training and Development.
Future Considerations
When constructing future scales and test items, there are some considerations I have learned from this project. In addition to writing quality test items following recommended practices, the importance of thorough evaluation of each item by test writers and Subject Matter Experts cannot be overstated. In future test developments, I would recruit as many SMEs as possible to ensure strong content validity. Similarly, exploratory factor analysis is a process that requires not only a thorough understanding of each statistical tests, but also deep theoretical knowledge of your constructs.
Skills Developed
-
Technical writing
-
Quantitative Data Analysis
-
Communication
-
SME Collaboration
-
Data Collection