Abstract
Purpose - This study aims to explore the objectivity in third-party ratings. Third-party ratings are often based on some form of aggregation of various experts' opinions with the assumption that the potential judgment biases of the experts cancel each other out. While psychology research has suggested that experts can be unintentionally biased, management literature has not considered the effect of expert bias on the objectivity of third-party ratings. Thus, this study seeks to address this issue.
Design/methodology/approach - Ranking data from the US News and World Report between 1993 and 2008, institution-related variables and, to represent sports prominence, NCAA football and basketball performance variables are leveraged in testing our hypotheses. A mediating-model is tested using regression with panel-corrected standard errors.
Findings - This study finds that the judgments of academicians and recruiters, concerning the quality of universities, have been biased by the prominence of a university's sports teams and that the bias introduced to these experts mediates the aggregated bias in the resultant rankings of MBA programs. Moreover, it finds that experts may inflate rankings by up to two positions.
Practical implications - This study is particularly relevant for university officials as it uncovers how universities can tangibly manipulate the relative perception of quality through sports team prominence. For third-party rating systems, the reliability of ratings based on aggregated expert judgments is called into question.
Originality/value - This study addresses a significant gap in the literature by examining how a rating system may be unintentionally biased through the aggregation of experts' judgments. Given the heavy reliance on third-party rating systems by both academics and the general population, addressing the objectivity of such ratings is crucial.