Content-Oblivious Quality Measures and the Control of Academia (by Oded Goldreich)

The essay examines the common practice of relying on content-oblivious quantitative measures for the evaluation of the quality of academic research. It starts with a definition of these measures, while distinguishing the raw bibliographic data and the way it is processed (to obtain a numerical value). It exposes the hidden decisions that determine which pieces of data are used and how they are processed (i.e., which statistics is taken of it). Indeed, content-oblivious quantitative measures are obtained by an automatic processing of superficial parameters of scholarly work, and their claim for objectivity hides the arbitrariness of the decisions on which they are based, and does not allow to discuss these decisions.

The direct consequence of relying on content-oblivious quantitative measures is reaching uninformed decisions, since professional judgment that relates to the content is replaced by a superficial measure whose relevance to the questions at stake is highly questionable. The indirect consequences of relying on these measures are even more dangerous. This practice neglects the actual content while fetishizing quantity, it replaces the academic vocation by preparation for accounting, it oppresses intellectual curiosity, encourages manipulations, and reproduces power relations within scientific disciplines and between them. The content-oblivious quantitative measures are compatible with shifting the focus of evaluation from verifying the satisfaction of threshold criteria to ranking and forming a rigid hierarchy.

The essay presents several explanations for the popularity of these content-oblivious quantitative measures: They fit the neo-liberal order and its preference for standardized regulation procedures; they carry a seductive promise of objectivity (which is extremely tempting to modern science); they serve opportunism (in the form of intellectual laziness and escaping responsibility); they empower the academic-managerial class by providing it with mechanisms for control of the academic works (by subjecting scientific knowledge to managerial knowledge); they facilitate diffusion of business attitudes to the academic world and the domination of scientific content by (bibliometric) technology. All these phenomena are related to the rise of a new political and scientific order in which a tighter control of academia plays an important role.

Back to Oded Goldreich's homepage or to Politics and Academia