Investigation of a nonlinear transformation to improve score linking for low- and high -performing groups of students
Educational progress is often measured by comparing the performance of successive cohorts of students on educational assessments over time, but valid inferences of progress require that scores on a newer assessment are comparable to scores on a previously administered assessment. The process by which scores on two assessments are statistically adjusted to achieve comparability is referred to as score linking or equating. Scores for two successive administrations of the National Assessment of Educational Progress (NAEP) are linked through a linear transformation that sets the means and standard deviations of the two score distributions to be equal. When the shapes of the two score distributions differ beyond these first two moments, a linear transformation may not adequately link scores, particularly those in the tails of the distributions. An inaccurate linking function could possibly obscure important changes in the performance of low- and high-performing groups of students over time. This study examined the possibility of improving upon linear linking by using a nonlinear (quadratic) function that matched the mean, standard deviation, and skewness of the NAEP score distributions to be linked. Graphical and statistical comparisons of the shapes of the score distributions, including several evaluative indices proposed by Jaeger (1981) to choose between linear and nonlinear linking, showed some evidence of dissimilarity for three of the four assessments examined. Assessment outcomes, including mean scale scores, score distribution percentiles, and achievement level percentages, obtained from linear and nonlinear linking were then compared for ten low-and high-performing groups of students. Differences in mean scale scores and extreme percentiles (5th, 10th, 90th and 95 th) were observed, but the magnitude of those differences was small (the largest was a decrease of .12 standard deviation in the fifth percentile of a subgroup score distribution) and were of little practical significance. Although a quadratic linking function failed to produce significantly different outcomes for the low-and high-performing groups studied, a cubic function that also accounts for kurtosis in the score distributions may be a better fit. Therefore, further study is necessary to judge the adequacy of linear linking in NAEP.
Moran, Rebecca A, "Investigation of a nonlinear transformation to improve score linking for low- and high -performing groups of students" (2007). Dissertations available from ProQuest. AAI3271794.