Measurement is the assignment of numbers to objects or events according to a rule. The rule should be such that the numbers provide reliable information about the objects or events. But the rules applicable in the field are different from the rules used in the laboratory. Methodologies appropriate for field measurement have to include instructions of how to replace control of the measurand and environment by control of the representing model, and how to deal with unscientific observations. Investigations of several measurement practices in different social field sciences show that for such methodologies expert judgment is indispensable. The statistical model can replace the laboratory to a certain extent, but not completely. The knowledge gap between an empirical model, however accurate it is, and the complex social field phenomenon has to be bridged by the intuitions of a field expert. But expert judgments are subjective and personal, and so tend to disagree and are not equally good. To have measurement outside the laboratory as objective as possible, one needs to find a consensus of the multiple expert judgments in a way that accounts for the performance of the individual experts. But to account for the quality of these performances in social science, instead of evaluating and comparing individual scientists, one has to evaluate and compare the expertise of institutions, each of which employs a team of experts and is the proprietor of the empirical model.

Consensus, Expert judgment, Field science, Laboratory science, Measurement, Model,
Erasmus School of Philosophy

Boumans, M. (2015). Science Outside the Laboratory: Measurement in Field Science and Economics. doi:10.1093/acprof:oso/9780199388288.001.0001