New Guidelines for Obtaining Optimal Scale Scores
By far one of the most challenging aspects of any empirical research application is how to best obtain valid and reliable scale scores of the theoretical constructs under study. The field of psychometrics has given rise to a myriad of methods for designing assessments, evaluating dimensionality, and estimating person-specific scores for subsequent analysis. The traditional approach to scoring is to calculate a simple mean of a set of items. However, this method is characterized by several well-known limitations that can combine to reduce both the reliability and the validity of the resulting score. More advanced methods draw on the item response theory and factor analysis traditions to obtain factor scores in which item unreliability is an explicit part of the model and items can differentially contribute to the calculation of the scale score. However, these approaches do not allow for the inclusion of the effects of individual-difference measures such as age, gender, ethnicity, or any other covariate of potential interest. Our group has worked on this problem for a number of years, and we recently published a paper that empirically demonstrates the improvement in score estimation when exogenous covariates are included in the scoring model. We also published a second paper that proposes a specific set of steps for implementing these models in practice, and a third paper that contrasts this approach to more conventional alternatives measurement models. We believe these papers add to the broader literature that continues to work toward the improvement of score estimation in applied research settings.