Recent Developments in Reliability Analysis

For many researchers, the literature of reliability coefficients seems bewildering although the methodological problem in which they are embedded is reasonably clear: Since we can never know what it is that we claim to see independent of our seeing it, or, translated into the language of science, since we can not test hypotheses about reality without first generating the observations or data to talk about, the accuracy by which primary data "represent" an unobserved nature remains unascertainable in principle (Krippendorff, 1991). Yet, to assure that the data that go into scientific inquiries are not accidental, it is important to demonstrate that the data-generating procedures are reproducible under varying circumstances and by several observers. All reliability measures are intended to express the degree to which several observers, several measuring instruments, or several interrogations of the same units of analysis yield the same descriptive accounts, category assignments, quantitative measures or data for short.