An anatomy of assessment: A qualitative study of conceptual influences upon placement essay raters' scoring decisions at a large urban community college
Abstract
One of the assumptions implicit in large-scale holistic writing evaluation is that building essay raters' consensus increases scoring reliability and deters the effects of various "differences" among raters. In smaller local settings, however, the evaluation of placement test writing may involve particular constraints upon raters' communal experience of the task. At Community College of Philadelphia (CCP), the underlying conceptions and assumptions of placement essay raters play a major role in defining and shaping the placement essay rating session. This study examines the ways in which CCP placement essay raters' conceptions of key elements implicated in the rating experience collectively influence raters' scoring decisions and, consequently, affect the outcomes for student placement into college-level and pre-college-level introductory writing and reading courses. The data consisted of three sets of think-aloud protocols collected as each of four essay rater-subjects scored placement essays; audiotaped interviews of the rater-subjects and selected CCP academic administrators, faculty members, and students; and one audiotaped recording of an actual "live" essay rating for comparison. The results show that, while individuals in the CCP academic community hold conflicting views of the educational enterprise, it is a distinctly socially constructed valuation of placement essay scoring specifically and student services generally that identifies the institutional character. Across the five items in the conceptual framework derived through this research, placement essay raters demonstrate the consistency with which a culturally-defined notion of the student-writer in writing evaluation supersedes conventional emphases upon the function and practice of assessment itself. These results can be shown to have implications for problematic issues in contemporary education.
Subject Area
Language arts|Educational tests & measurements|Educational evaluation|Community college education
Recommended Citation
Nace, John Edward, "An anatomy of assessment: A qualitative study of conceptual influences upon placement essay raters' scoring decisions at a large urban community college" (1992). Dissertations available from ProQuest. AAI9308633.
https://repository.upenn.edu/dissertations/AAI9308633