Advanced Search

Journal Navigation

Journal Home

Subscriptions

Archive

Contact Us

Table of Contents

SAGETRACK

Sign In to gain access to subscriptions and/or personal tools.
Organizational Research Methods
This Article
Right arrow Full Text (OnlineFirst PDF)
Right arrow All Versions of this Article:
1094428108317406v1
12/3/461    most recent
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Alert me to new issues of the journal
Right arrow Add to Saved Citations
Right arrow Download to citation manager
Right arrowRequest Permissions
Right arrow Request Reprints
Right arrow Add to My Marked Citations
Citing Articles
Right arrow Citing Articles via Google Scholar
Right arrow Citing Articles via Scopus
Google Scholar
Right arrow Articles by Lüdtke, O.
Right arrow Articles by Robitzsch, A.
Right arrow Search for Related Content
Social Bookmarking
Add to CiteULike   Add to Complore   Add to Connotea   Add to Del.icio.us   Add to Digg   Add to Reddit   Add to Technorati   Add to Twitter  
What's this?

Article

Assessing Within-Group Agreement: A Critical Examination of a Random-Group Resampling Approach

Oliver Lüdtke and Alexander Robitzsch

Max Planck Institute for Human Development

* To whom correspondence should be addressed.


  Abstract
The measure of within-group agreement most frequently encountered in organizational psychology is the rWG index. The rWG index is determined by comparing the observed group variance among raters with an expected random variance. The most critical issue in calculating the rWG is the choice of an appropriate random distribution that would be expected to follow from raters making their ratings at random. A data-driven approach that uses random-group resampling (RGR) procedures to determine the expected random variance has been proposed. In the present study, the application of the RGR procedure will be illustrated with reference to students' ratings of their mathematics instruction and critically compared with a recently proposed simulation-based approach. It will be shown mathematically that the probability of obtaining statistically significant within-group agreement when applying the RGR procedure strongly depends on the intraclass correlation as well as on the group sizes. Finally, implications for applying the RGR procedure to assess within-group agreement in multilevel data will be discussed.

First published on April 29, 2008, doi:10.1177/1094428108317406

Organizational Research Methods 2009;12:461.

A more recent version of this article appeared on July 1, 2009

Add to CiteULike CiteULike   Add to Complore Complore   Add to Connotea Connotea   Add to Del.icio.us Del.icio.us   Add to Digg Digg   Add to Reddit Reddit   Add to Technorati Technorati   Add to Twitter Twitter    What's this?