Tianjun Sun, Ph.D. - Though Forced, Still Valid: Psychometric

Journal article

Bo Zhang, Tianjun Sun, F. Drasgow, O. Chernyshenko, Christopher D. Nye, Stephen E. Stark, L. White
Organizational Research Methods, 2020

Semantic Scholar DOI

Cite

APA Click to copy
Zhang, B., Sun, T., Drasgow, F., Chernyshenko, O., Nye, C. D., Stark, S. E., & White, L. (2020). Though Forced, Still Valid: Psychometric Equivalence of Forced-Choice and Single-Statement Measures. Organizational Research Methods.

Chicago/Turabian Click to copy
Zhang, Bo, Tianjun Sun, F. Drasgow, O. Chernyshenko, Christopher D. Nye, Stephen E. Stark, and L. White. “Though Forced, Still Valid: Psychometric Equivalence of Forced-Choice and Single-Statement Measures.” Organizational Research Methods (2020).

MLA Click to copy
Zhang, Bo, et al. “Though Forced, Still Valid: Psychometric Equivalence of Forced-Choice and Single-Statement Measures.” Organizational Research Methods, 2020.

BibTeX Click to copy

@article{bo2020a,
  title = {Though Forced, Still Valid: Psychometric Equivalence of Forced-Choice and Single-Statement Measures},
  year = {2020},
  journal = {Organizational Research Methods},
  author = {Zhang, Bo and Sun, Tianjun and Drasgow, F. and Chernyshenko, O. and Nye, Christopher D. and Stark, Stephen E. and White, L.}
}

Abstract

Forced-choice (FC) measures are gaining popularity as an alternative assessment format to single-statement (SS) measures. However, a fundamental question remains to be answered: Do FC and SS instruments measure the same underlying constructs? In addition, FC measures are theorized to be more cognitively challenging, so how would this feature influence respondents’ reactions to FC measures compared to SS? We used both between- and within-subjects designs to examine the equivalence of the FC format and the SS format. As the results illustrate, FC measures scored by the multi-unidimensional pairwise preference (MUPP) model and SS measures scored with the generalized graded unfolding model (GGUM) showed strong equivalence. Specifically, both formats demonstrated similar marginal reliabilities and test-retest reliabilities, high convergent validities, good discriminant validities, and similar criterion-related validities with theoretically relevant criteria. In addition, the formats had little differential impact on respondents’ general emotional and cognitive reactions except that the FC format was perceived to be slightly more difficult and more time-saving.