This Article Full Text (PDF) References Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in Web of Science Alert me to new issues of the journal Add to Saved Citations Download to citation manager Request Permissions Request Reprints Add to My Marked Citations Citing Articles Citing Articles via HighWire Citing Articles via Web of Science (20) Citing Articles via Google Scholar Citing Articles via Scopus Google Scholar Articles by LeBreton, J. M. Articles by Lindell, M. K. Search for Related Content Social Bookmarking                         What's this?

Recent Issues Regarding rWG, rWG, rWG(J), and rWG(J)

Wayne State University

Lawrence R. James

Georgia Institute of Technology

Michael K. Lindell

Texas A&M University

A recent article by Lindell and Brandt raised two concerns regarding the use of James, Demaree, and Wolf’s interrater agreement indices r_WG and r_WG(J). First, they noted that the multi-item _rWG(J) equation is mathematically equivalent to inserting r_WG into the Spearman-Brown prophecy formula and questioned whether applying a formula developed for a reliability index was also appropriate for an agreement index. Second, they questioned the appropriateness of James et al.’s suggestion of replacing obtained negative values of r_WG with zeros. This article addresses these concerns by demonstrating that r_WG(J) can be derived independently from the Spearman-Brown prophecy formula and that negative values of r_WG can be avoided by reparameterizing the structural equation underlying the data when there is systematic disagreement between subgroups of raters.

Key Words: interrater agreement • interrater disagreement • Spearman-Brown • rWG

Organizational Research Methods, Vol. 8, No. 1, 128-138 (2005)
DOI: 10.1177/1094428104272181


CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    Twitter    What's this?

This article has been cited by other articles:

				X. Dumay Origins and Consequences of Schools' Organizational Culture for Student Achievement Educational Administration Quarterly, October 1, 2009; 45(4): 523 - 555. [Abstract] [PDF]

				O. Ludtke and A. Robitzsch Assessing Within-Group Agreement: A Critical Examination of a Random-Group Resampling Approach Organizational Research Methods, July 1, 2009; 12(3): 461 - 487. [Abstract] [PDF]

				D. J. Pasisz and G. M. Hurtz Testing for Between-Group Differences in Within-Group Interrater Agreement Organizational Research Methods, July 1, 2009; 12(3): 590 - 613. [Abstract] [PDF]

				A. Cohen, E. Doveh, and I. Nahum-Shani Testing Agreement for Multi-Item Scales With the IndicesrWG(J) and AD M(J) Organizational Research Methods, January 1, 2009; 12(1): 148 - 164. [Abstract] [PDF]

				D. A. Newman and H.-P. Sin How Do Missing Data Bias Estimates of Within-Group Agreement? Sensitivity of SD WG, CVWG, rWG(J), rWG(J) * , and ICC to Systematic Nonresponse Organizational Research Methods, January 1, 2009; 12(1): 113 - 147. [Abstract] [PDF]

				J. M. LeBreton and J. L. Senter Answers to 20 Questions About Interrater Reliability and Interrater Agreement Organizational Research Methods, October 1, 2008; 11(4): 815 - 852. [Abstract] [PDF]

				D. J. Beal and J. F. Dawson On the Use of Likert-Type Scales in Multilevel Data: Influence on Aggregate Variables Organizational Research Methods, October 1, 2007; 10(4): 657 - 672. [Abstract] [PDF]

				Q. M. Roberson, M. C. Sturman, and T. L. Simons Does the Measure of Dispersion Matter in Multilevel Research? A Comparison of the Relative Performance of Dispersion Indexes Organizational Research Methods, October 1, 2007; 10(4): 564 - 588. [Abstract] [PDF]

				F. Luthans and C. M. Youssef Emerging Positive Organizational Behavior Journal of Management, June 1, 2007; 33(3): 321 - 349. [Abstract] [PDF]

				C. E. Lance, M. M. Butts, and L. C. Michels The Sources of Four Commonly Reported Cutoff Criteria: What Did They Really Say? Organizational Research Methods, April 1, 2006; 9(2): 202 - 220. [Abstract] [PDF]