This Article Full Text (PDF) All Versions of this Article: 1094428108319128v1 1094428108319128v2 12/3/590    most recent References Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Alert me to new issues of the journal Add to Saved Citations Download to citation manager Request Permissions Request Reprints Add to My Marked Citations Citing Articles Citing Articles via Scopus Google Scholar Articles by Pasisz, D. J. Articles by Hurtz, G. M. Social Bookmarking                         What's this?

Testing for Between-Group Differences in Within-Group Interrater Agreement

California State University, Sacramento, djp07d{at}fsu.edu

Gregory M. Hurtz

California State University, Sacramento

Users of r_wg may find situations where they want to compare the levels of within-group agreement across two or more groups. F tests are presented that compare two values of r_wg, rwg(J), or r*_wg(J), and recommendations are made for comparing pairwise group values for three or more groups with a control for alpha inflation. More robust and flexible methods based on O'Brien's test are then recommended when sufficient raw data are available. The meaning of findings that agreement statistics are either the same or different across groups is discussed in light of simultaneous comparisons of mean differences. Examples are shown to demonstrate the robust methods using employee responses to questions regarding climate perceptions of organizational support for employee development activities. Other scenarios where the F test may prove useful are discussed, and directions for future research are provided.

Key Words: agreement • interrater agreement • rwg • measurement design

This version was published on July 1, 2009

Organizational Research Methods, Vol. 12, No. 3, 590-613 (2009)
DOI: 10.1177/1094428108319128


CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    Twitter    What's this?