Psychology Ph.D. Dissertations

Revisiting Item Semantics in Measurement: A New Perspective Using Modern Natural Language Processing Embedding Techniques

Date of Award

2023

Document Type

Dissertation

Degree Name

Doctor of Philosophy (Ph.D.)

Department

Psychology/Industrial-Organizational

First Advisor

Michael Zickar (Committee Chair)

Second Advisor

Neil Baird (Other)

Third Advisor

Richard Anderson (Committee Member)

Fourth Advisor

Samuel McAbee (Committee Member)

Abstract

Language understanding plays a crucial role in psychological measurement and so it is important that semantic cues should be studied for more effective and accurate measurement practices. With advancements in computer science, natural language processing (NLP) techniques have emerged as efficient methods for analyzing textual data and have been used to improve psychological measurement. This dissertation investigates the application of NLP embeddings to address fundamental methodological challenges in psychological measurement, specifically scale development and validation. In Study 1, a word embedding-based approach was used to develop a corporate personality measure, which resulted in a three-factor solution closely mirroring three dimensions out of the Big Five framework (i.e., Extraversion, Agreeableness, and Conscientiousness). This research furthers our conceptual understanding of corporate personality by identifying similarities and differences between human and organizational personality traits. In Study 2, the sentence-based embedding model was applied to predict empirical pairwise item response relationships, comparing its performance with human ratings. This study also demonstrated the effectiveness of fine-tuned NLP models for classifying item pair relationships into trivial/low or moderate/high empirical relationships, which provides preliminary validity evidence without collecting human responses. The research seeks to enhance psychological measurement practices by leveraging NLP techniques, fostering innovation and improved understanding in the field of social sciences.

Link to Full Text

Share

COinS