| Sign In to gain access to subscriptions and/or personal tools. |
DOI: 10.1177/1094428105275376 Interrater Agreement Reconsidered: An Alternative to the rwg IndicesWestern Kentucky University
Virginia Polytechnic Institute and State University For continuous constructs, the most frequently used index of interrater agreement (r wg(1))can be problematic. Typically, rwg(1) is estimated with the assumption that a uniform distribution represents no agreement. The authors review the limitations of this uniform nullr wg(1) index and discuss alternative methods for measuring interrater agreement. A new interrater agreement statistic,a wg(1),is proposed. The authors derive thea wg(1)statistic and demonstrate thatawg(1) is an analogue to Cohens kappa, an interrater agreement index for nominal data. A comparison is made between agreement estimates based on the uniformr wg(1)and a wg(1), and issues such as minimum sample size and practical significance levels are discussed. The authors close with recommendations regarding the use ofr wg(1)/rwg(J) when a uniform null is assumed,r wg(1)/rwg(J) indices that do not assume a uniform null,awg(1) / a wg(J)indices, and generalizability estimates of interrater agreement.
Key Words: interrater agreement interrater reliability kappa
This article has been cited by other articles:
|
||||||||||||||||||||||||||||||||||||||||||||||||

