Advisor

Morse, T. David

Committee Member

Elder, Anastasia

Committee Member

Forde, M. Connie

Committee Member

McMillen, L. David

Committee Member

Morse, W. Linda

Date of Degree

5-1-2008

Document Type

Dissertation - Open Access

Degree Name

Doctor of Philosophy

College

College of Education

Abstract

This study investigates the psychometric issues and viability of cellphone-based-testing, a novel test administration modality whereby test-takers use a cellphone to respond to items on a web-based assessment. The study explored mode-dependent differences in scores from a web-based version of the Self-Monitoring Scale (SMS) administered across two modalities: desktop computer and cellphone. The selection of the SMS was based on several pre-established criteria. The instrument was simple and brief. Its text-based items included true/false response categories. Its rights of use fell under public domain and it had been previously validated for online administration. The study includes a comprehensive overview of recent literature related to the topic of psychometric equivalency and incorporates numerous methodological approaches to determine test score equivalence, including: comparisons of central tendency, dispersions, and rank order; the Kolmogorov-Smirnov test of equal score distributions; the Pitman procedure for detecting differences in reliability coefficients; a confirmatory factor analysis of the equality of factor structures using LISREL; and an analysis of differential item functioning based on item response theory using BILOG-MG. The study employed a counterbalanced repeated measures design whereby 234 participants took an 18-item web-based version of the SMS using a desktop computer and/or a cellphone. The psychometric equivalency of scores from the two modes of administration was analyzed. All statistical comparisons provided overwhelming support for one general conclusion: There were no mode-dependent differences in scores on the web-based version of the SMS when administered by desktop computer versus cellphone. The study also explored participants’ attitudes toward using cellphones as a test-taking tool. The participants correctly anticipated that their scores would not be affected by using a cellphone, but they categorically rated the cellphone as less enjoyable, more difficult, and more cumbersome than a desktop computer. However, one cannot ignore the tendency of our modern society for being obsessed with information on demand. As cellphone technology continues to improve and the text-messaging generation begins to influence the field of educational and psychological measurement, cellphone-based-testing will likely become an accepted standard for both academic and clinical practice.

URI

https://hdl.handle.net/11668/15057

Share

COinS