Organizational Research Methods current issue

Scale Coarseness as a Methodological Artifact: Correcting Correlation Coefficients Attenuated From Using Coarse Scales

Aguinis, H., Pierce, C. A., Culpepper, S. A. — Wed, 23 Sep 2009 09:23:46 PDT

Scale coarseness is a pervasive yet ignored methodological artifact that attenuates observed correlation coefficients in relation to population coefficients. The authors describe how to disattenuate correlations that are biased by scale coarseness in primary-level as well as meta-analytic studies and derive the sampling error variance for the corrected correlation. Results of two Monte Carlo simulations reveal that the correction procedure is accurate and show the extent to which coarseness biases the correlation coefficient under various conditions (i.e., value of the population correlation, number of item scale points, and number of scale items). The authors also offer a Web-based computer program that disattenuates correlations at the primary-study level and computes the sampling error variance as well as confidence intervals for the corrected correlation. Using this program, which implements the correction in primary-level studies, and incorporating the suggested correction in meta-analytic reviews will lead to more accurate estimates of construct-level correlation coefficients.

The Neglected Science and Art of Quasi-Experimentation: Why-to, When-to, and How-to Advice for Organizational Researchers

Grant, A. M., Wall, T. D. — Wed, 23 Sep 2009 09:23:46 PDT

Although quasi-experiments can facilitate causal inferences by combining good internal validity with high external validity, organizational scholars underutilize them. In this article, the authors aim to encourage the novel use of quasi-experimentation by identifying five of its key benefits: (a) strengthening causal inference when random assignment and controlled manipulation are not possible or ethical; (b) building better theories of time and temporal progression; (c) minimizing ethical dilemmas of harm, inequity, paternalism, and deception; (d) facilitating collaboration with practitioners; and (e) using context to explain conflicting findings. We offer advice and illustrative examples to guide future research, and provide recommendations for gaining access to organizations to open doors for collaborating on quasi-experiments.

Decimal Dust, Significant Digits, and the Search for Stars

Bedeian, A. G., Sturman, M. C., Streiner, D. L. — Wed, 23 Sep 2009 09:23:46 PDT

The practice of rounding statistical results to two decimal places is one of a large number of heuristics followed in the social sciences. In evaluating this heuristic, the authors conducted simulations to investigate the precision of simple correlations. They considered a true correlation of .15 and ran simulations in which the sample sizes were 60, 100, 200, 500, 1,000, 10,000, and 100,000. They then looked at the digits in the correlations’ first, second, and third decimal places to determine their reproducibility. They conclude that when n < 500, the habit of reporting a result to two decimal places seems unwarranted, and it never makes sense to report the third digit after the decimal place unless one has a sample size larger than 100,000. Similar results were found with rhos of .30, .50, and .70. The results offer an important qualification to what is otherwise a misleading practice.

Testing Multilevel Mediation Using Hierarchical Linear Models: Problems and Solutions

Zhang, Z., Zyphur, M. J., Preacher, K. J. — Wed, 23 Sep 2009 09:23:46 PDT

Testing multilevel mediation using hierarchical linear modeling (HLM) has gained tremendous popularity in recent years. However, potential confounding in multilevel mediation effect estimates can arise in these models when within-group effects differ from between-group effects. This study summarizes three types of HLM-based multilevel mediation models, and then explains that in two types of these models confounding can be produced and erroneous conclusions may be derived when using popularly recommended procedures. A Monte Carlo simulation study illustrates that these procedures can underestimate or overestimate true mediation effects. Recommendations are provided for appropriately testing multilevel mediation and for differentiating within-group versus between-group effects in multilevel settings.

A Proposal for Operationalizing the Pace and Scope of Organizational Change in Management Studies

Street, C. T., Gallupe, R. B. — Wed, 23 Sep 2009 09:23:46 PDT

Organizational change is an important construct for management theorists, yet organizational research is being hampered by inconsistent and incompatible operationalizations of the construct. This article presents a proposal for improving clarity about how the types and characteristics of organizational change can be operationalized and measured. In particular, the scope and pace of organizational change are examined and a common approach is developed to measure the impacts of these two factors on patterns of organizational change.

Test Bias, Differential Prediction, and a Revised Approach for Determining the Suitability of a Predictor in a Selection Context

Meade, A. W., Fetzer, M. — Wed, 23 Sep 2009 09:23:46 PDT

The most commonly used and accepted model of assessing bias in a selection context is that proposed by Cleary in which predictor-criterion regression lines are tested for both slope and intercept equality. With this approach, any difference in intercepts or slopes is considered an indication of bias. We argue that differing regression lines intercepts is indicative of differential prediction but not test bias. We describe several fundamentally different potential causes of differences in groups’ regression line intercepts, many of which are unrelated to test properties. We argue that differential prediction because of such sources should not preclude the use of the test in selection contexts. We propose a new procedure to potentially identify the source of regression line differences and illustrate this framework using a job incumbent sample.

A Tale of Three Perspectives: Examining Post Hoc Statistical Techniques for Detection and Correction of Common Method Variance

Richardson, H. A., Simmering, M. J., Sturman, M. C. — Wed, 23 Sep 2009 09:23:46 PDT

Many researchers who use same-source data face concerns about common method variance (CMV). Although post hoc statistical detection and correction techniques for CMV have been proposed, there is a lack of empirical evidence regarding their efficacy. Because of disagreement among scholars regarding the likelihood and nature of CMV in self-report data, the current study evaluates three post hoc strategies and the strategy of doing nothing within three sets of assumptions about CMV: that CMV does not exist, that CMV exists and has equal effects across constructs, and that CMV exists and has unequal effects across constructs. The implications of using each strategy within each of the three assumptions are examined empirically using 691,200 simulated data sets varying factors such as the amount of true variance and the amount and nature of CMV modeled. Based on analyses of these data, potential benefits and likely risks of using the different techniques are detailed.

Book Review: Creswell, J., & Plano Clark, V. (2007). Designing and Conducting Mixed Methods Research. Thousand Oaks, CA: Sage

Yu, C. H. — Wed, 23 Sep 2009 09:23:46 PDT

Book Review: Van de Ven, A. (2007). Engaged scholarship: A guide for organizational and social research. New York: Oxford University Press

Adelman, M., Spivack, A. — Wed, 23 Sep 2009 09:23:46 PDT

Book Review: Dunn-Rankin, P., Knezek, G. A., Wallace, S., & Zhang, S. (2004). Scaling methods (2nd ed.). Mahwah, NJ: Lawrence Erlbaum

Hurley-Hanson, A. E. — Wed, 23 Sep 2009 09:23:47 PDT

Ad Hoc Reviewers From July 1, 2007, Through June 1, 2009

Wed, 23 Sep 2009 09:23:47 PDT