Theses and Dissertations

Issuing Body

Mississippi State University

Advisor

Morse, T. David

Committee Member

Elder, Anastasia

Committee Member

Forde, M. Connie

Committee Member

McMillen, L. David

Committee Member

Morse, W. Linda

Date of Degree

5-3-2008

Document Type

Dissertation - Open Access

Major

Educational Psychology

Degree Name

Doctor of Philosophy

College

College of Education

Department

Department of Counseling, Educational Psychology and Special Education

Abstract

This study investigates the psychometric issues and viability of cellphone-based-testing, a novel test administration modality whereby test-takers use a cellphone to respond to items on a web-based assessment. The study explored mode-dependent differences in scores from a web-based version of the Self-Monitoring Scale (SMS) administered across two modalities: desktop computer and cellphone. The selection of the SMS was based on several pre-established criteria. The instrument was simple and brief. Its text-based items included true/false response categories. Its rights of use fell under public domain and it had been previously validated for online administration. The study includes a comprehensive overview of recent literature related to the topic of psychometric equivalency and incorporates numerous methodological approaches to determine test score equivalence, including: comparisons of central tendency, dispersions, and rank order; the Kolmogorov-Smirnov test of equal score distributions; the Pitman procedure for detecting differences in reliability coefficients; a confirmatory factor analysis of the equality of factor structures using LISREL; and an analysis of differential item functioning based on item response theory using BILOG-MG. The study employed a counterbalanced repeated measures design whereby 234 participants took an 18-item web-based version of the SMS using a desktop computer and/or a cellphone. The psychometric equivalency of scores from the two modes of administration was analyzed. All statistical comparisons provided overwhelming support for one general conclusion: There were no mode-dependent differences in scores on the web-based version of the SMS when administered by desktop computer versus cellphone. The study also explored participants’ attitudes toward using cellphones as a test-taking tool. The participants correctly anticipated that their scores would not be affected by using a cellphone, but they categorically rated the cellphone as less enjoyable, more difficult, and more cumbersome than a desktop computer. However, one cannot ignore the tendency of our modern society for being obsessed with information on demand. As cellphone technology continues to improve and the text-messaging generation begins to influence the field of educational and psychological measurement, cellphone-based-testing will likely become an accepted standard for both academic and clinical practice.

URI

https://hdl.handle.net/11668/15057

Download

Share

COinS