Quantification in process. Putting data into measurement categories

Abstracts

How to quantify irregular migration at a European level? Discussions on data and measurement categories within the CIREFI

Pauline Adam, Université libre de Bruxelles and Ecole Normale Supérieure

In this paper I ask what happens (conceptually), and what happened (empirically), with the establishment of the first European quantification process on irregular migration. Drawing on the thinking tools of the socio-history of quantification, I explore the statistical activities of CIREFI (1992-2010), the first EU body ever established to collect statistical data on irregular border crossings and migration on a EU-ropean scale. As the CIREFI was not directly producing statistics at source but collecting data from the Member States delegations, statistical categories were therefore defined theoretically. Therefore, I consider the harmonisation of irregular migration data at a European level as an ”harmonisation of outputs” as investigated by Alain Desrosières (Desrosières, 2008). For these reasons, I will have a special look at the discussions and tools that enable the dissemination of “top-down” definitions and the way Member States are interpreting those categories according to their national administrative, political and legal framework. Thanks to CIREFI archives research, I study the controversies, failures and partial achievements involved in this effort, the practical tools used to collect and store the data, the “CIREFI experts” conclusions on categories and common definitions. I argue that the discussions on common definitions and categories never end within the CIREFI from its creation to its end. In fact, the CIREFI conventions continue to be discussed whereas the measure operations are already taking place. The two steps of the quantification process are entangled.

Telling the minister a quantified story about re-industrialization: how to put data into political categories

Camille Beaurepaire, Ecole Normale Supérieure

Politicians use numbers routinely. When they are interviewed in the media, when they announce or denounce such or such policy – these numbers that they use are woven into political narratives. Among them, ministers occupy a special place because they can rely on the figures calculated by members of their administration. How do the statisticians-economists working in this administration respond to requests from ministers to put data into the political categories and narratives they are used to? This paper draws on a three-year ethnographic survey inside the French economic administration, within a service specialized in economic and industrial statistics. To tell a quantified story about re-industrialization to their minister, the statisticians-economists of this service use the various data at their disposal, whether they are of official or private origin. Three different data journeys about the quantification of re-industrialization can be contrasted. What data are chosen and how they are modified to fit the category depends on the constraints imposed upstream by the political authority.

Open data in higher education and research: a 'strategically opaque transparency'

Pauline Boyer, CNRS - SAGE

Following strong international mobilisation, the French state has implemented key policies to promote open government data. This agenda-setting relies heavily on a democratic argument: data opening would foster transparency and accountability of public action by giving citizens direct access to information. The proposed presentation draws on empirical work on the production and opening of administrative data in higher education and research (HER) to assess what information is left behind and created in the data collection and opening process. HER administrative data is collected annually from universities by the Ministry, which has also taken on the task of opening data on its portal, #dataESR. The Ministry thus has dual control of the information it releases: it both determines the conditions of quantification by setting the 'conventions of equivalences' (Desrosières 2008) used in the data collection campaigns, and also determines the structure of opened datasets. We have compared, variable by variable, the data collected by the Ministry in its collection campaign on student enrolment with the opened datasets on the same topic: the information contained in 48 variables (out of the 59 collected) disappears in the opened datasets. This analysis, alongside the conduct of 20+ interviews, lead us to reflect on open data as a 'strategically opaque transparency' (Ruijer et al. 2020). As government data represent strategic assets for organisations, they only have limited interest in sharing them (Michener et Ritter, 2017; Peled, 2011) and rather control the information released for legal, technical and strategic reasons (Longo, 2011; Zuiderwijk and Janssen, 2014).

Making sense of terroir soils: Monitoring, measuring, and managing the 'place of taste'

June Brawner, The Royal Society

Terroir, or the 'taste of place' (Trubek 2008), is the unique assemblage of factors that define a geography and the food products of that region. Since the 18th century, the elusive terroir has received legal expression through protected place-brands like Champagne wine or Darjeeling tea (see Josling 2006). Terroir products presume a sensible and measurable link between a food and its place of origin – between the taste of a product and its endemic soil. This paper explores the place-taste link in terroir as it is monitored, measured, and managed. It focuses on the data (whether soil or sensory) that is leveraged in arguments of geographically-based quality. To do so, it outlines the evolution of epistemic traditions in soil science as applied to the world's second-oldest protected terroir, the Tokaj wine region in Hungary (1737). This paper uses the Tokaj case in tracing historic efforts to define the region according to the minerality of its volcanic soils, whether tasted in its wines or quantitative pedology (soil metrics, e.g., available nutrients). This paper asks: how has quantitative soil science been deployed in the reification of terroir, and how does this data interface with sensory experience? Given these regimes of environmental sense-making, what are the broader social and ecological implications? These questions are addressed through mixed methods doctoral fieldwork in Tokaj (2016-2019) that foregrounds participatory soil sampling and archival research. The result is contextualised soil science where the politics of expertise follow multiple ontologies and values embedded in soil data and its interpretations.

Making economic data comparable. The production of the dynamics of the economy

Dufour Quentin, Ecole Normale Supérieure

Making economic data comparable. The production of the dynamics of the economy.
(Measuring economic growth, i.e. the evolution of the national economy between two dates, requires the existence of comparable data over time. At the crossroads of STS and the sociology of quantification, this paper focuses on the ways in which comparable data are produced. It questions the type of organization and social practices that ensure the continuity of the measurement of the economy. To address this question, we rely on a nine-month ethnographic survey within the French Statistical Institute, and more precisely at the national accounts department, in charge of measuring the country’s growth for the year or the quarter. At the department, the work consists in collecting numerous economic data from the French statistical apparatus, and to integrate them into accounting tables. In this process, we show that the comparability of data is the result of two contradictory movements. On the one hand, the stability of the work organization, which follows the same standards each year. On the other hand, the transformation of the data that are deemed incompatible with the previous economic series. These two movements make it possible to bring the data into a singular economic temporality that the national accounts shape.)

The Brazilian elementary school in numbers: precarious statistics for the measurement of school inequality

Gil Natália, Universidade Federal do Rio Grande do Sul

The Brazilian elementary school in numbers: precarious statistics for the measurement of school inequality
(Educational statistics in Brazil began to be produced regularly and with increasing technical rigor from 1931. However, the analysis of the figures on education published by official bodies linked to the Ministry of Education throughout the twentieth century leads to identify that, despite the significant volume of surveys produced in the period, there is a lack of information about the characteristics of students. The scrutiny of absences in education statistics, throughout history, allowed to differentiate situations in which information was collected and published, but did not give rise to analyzes that sought to examine inequalities; situations in which the information was included in the school records, but does not appear compiled in the statistical publications; and also situations in which the categories were not even established as aspects of interest to deepen the knowledge about the students. The study showed that the absence of statistical categories on students is related to the delay in assuming, in specialized debates and educational policies, that school exclusion did not affect the different social groups equally.)

The work of data: on carbon quantification in French public research laboratories

Hardy Antoine, Centre Emile-Durkheim

The work of data: on carbon quantification in French public research laboratories.
(Why and how do research staff decide to quantify the carbon footprint of research activities? This question is at the heart of my ongoing dissertation, at the intersection of Science and Technology Studies and sociology of quantification. My fieldwork is a group of French scientists who are quantifying the carbon footprint of public research laboratories in France, with their own online carbon tracker. Using qualitative methods (about fifty interviews and participant-observation), my work leads me to take a close look at the two operations at the heart of a quantification process, the construction of conventions and the work of measurement (Desrosières, 1993, 2008). Based on my empirical materials, my talk will show that conventions and measurements can be understood as a continuous process: the first convention that initiates the work of measurement by establishing different categories (energy used by buildings, travels, etc.) is followed by other conventional constructions that broaden the carbon estimation. The calculator was conceived from the beginning as “evolving” and new categories were gradually included. 18 semi-structured interviews conducted with users of this carbon tracker allow me to describe the work of data: technical, when the calculator was modified to fit local specificities; emotional, with manifestations of “surprise” when quantitative data differs from qualitative experience; political, with tensions in some laboratories. My talk will be in line with a current effort to broaden the sociology of quantification both beyond the numbers produced by the state's statistical apparatus (Martin, 2020; Mennicken and Salais, eds., 2022; Didier, 2022) and the classic dichotomy between quantitative and qualitative data (Newfield, Alexandrova and John, eds., 2022). Finally, I will question the possible emergence of a norm through quantification and the trust in numbers (Porter, 1995).)

Making undernutrition

Iversen Thor Olav, Norwegian Institute of International Affairs (NUP)I)

Making undernutrition
(In this article I analyze the history of crafting quantitative thresholds for the international estimation of undernutrition. I emphasize such thresholds at the group level, meaning for national, regional, and global estimates. The article analyzes historical reports that have been used to determine human caloric requirements, published by FAO, World Health Organization (WHO), and the United Nations University (UNU) (FAO 1957, 1950, FAO and WHO 1971, FAO, WHO, and UNU 1985, 2001). These reports were used to guide the estimation of international undernutrition in FAO’s flagship World Food Survey reports published from 1946 through 1996, complementary report series, and the contemporary State of Food Insecurity and Nutrition in the World (SOFI) reports. The thresholds were a crucial component for developing international estimates of undernutrition by FAO’s flagship indicator Prevalence of Undernourishment.

The setting of international thresholds for undernutrition provides an illustrative case of how science is informed by and expresses norms and values. The early assumption that the thresholds should be based on healthy reference people made it less suited to represent groups across class, cultures, and geography that suffer from a higher burden of disease and poor health. This exemplifies how the idealized schemes of science require displacement of complex human beings and systems. The conscious under-reporting of undernutrition by choice of low or no levels of physical activity furthermore constitutes a lack of recognition of the depravation of many people suffering from various forms of malnutrition. This could be considered an epistemic injustice in and of itself.)

Digital methods as ‘experimental a priori’ – how to navigate vague empirical situations as an operationalist pragmatist

Madsen Anders Koed, Aalborg University Copenhagen

Digital methods as ‘experimental a priori’ – how to navigate vague empirical situations as an operationalist pragmatist
(Digitalisation and computation presents us with a vague empirical world that unsettles established links between measurements and values. As more and more actors use digital media to produce data about aspects of the world they deem important, new possibilities for inscribing their lives emerge. The practical work with digital methods thus often involves the production of social visibilities that are misfits in the context of established data practices. In this paper I argue that this dissonance carries the distinct critical potential to design data experiments that (a) uses the act of operationalisation as an engine for creating intersubjective clarity about the meaning of existing concepts and (b) takes advantage of algorithmic techniques to provoke a reassessment of some of the core assumptions that shape the way we pose empirical problems are normally framed. Drawing on the work of Kant, Peirce, Dewey and C.I. Lewis I propose to think of this critical potential as the possibility to practice what I term ’experimental a priori’ and I use qualitative vignettes from two years of data experiments with GEHL architects to illustrate what this entails in practice. Faced with the task of using traces from Facebook as an empirical source to produce a map of urban political diversity, the architects found themselves in a need to revisit inherited assumptions about the ontology of urban space and the way it can even be formulated as a problem of diversity. While I describe this as a form of obstructive data practice that is afforded by digital methods, I also argue that it cannot be realised without deliberate design interventions. I therefore end the paper by outlining five design principles that can productively guide collective work with digital methods. These principles contribute to recent work within digital STS on the recalibration of problem spaces and the design of data sprints.)

Quantifying (parts of) Sápmi: On colonial counting, statistical sovereignty, and unruly categories in Norway, ca 1970-2020

Pettersen Torunn, Sámi allaskuvla / Sámi University of Applied Sciences

Quantifying (parts of) Sápmi: On colonial counting, statistical sovereignty, and unruly categories in Norway, ca 1970-2020
(The global network Indigenous Data Sovereignty (ID-Sov) highlights the right of Indigenous peoples to govern data from and about their communities and lands, articulating both individual and collective rights to data access and to privacy. As much as the institutional arrangements and practices for dealing with Indigenous issues vary significantly among Indigenous peoples, so do their respective data situation in various local and national contexts. In Norway, a pivotal aspect of the institutional situation is that a popularly elected Sámediggi (Sámi parliament) is established to be a forum for Sámi deliberation and formulation of Sámi policy in matters concerning the Sámi as a people. The statistical situation has however for many decades been characterized by a rather weak Sámi presence in official statistics, including basic population statistics.

In this presentation I will show how certain aspects of the Sámi-political arrangements in Norway during, first and foremost, the last six decades eventually have led to the production of certain kinds of Sámi-related statistics, while at the same time revealing Sámi-internal antagonism on the topic and creating dilemmas of inclusion and exclusion in such statistics. Based on this, I will discuss why the very act of categorization for statistical purposes probably never should be considered an innocent practice, not even when the contexts and purposes are considered "good".

The contribution is based partly on experiences in the practice field and partly on document studies during a number of research projects over two decades.

Which Lives Count for Artificial Intelligence? Ethnography at NASA

Alicja Ostrowska, Chalmers University of Technology

If life turns to numbers through artificial intelligence, how can we keep track of which lives count? Moreover, whose intelligence is the measure? By studying the context of search for life in the universe through AI, I look at the shifting character of life as a category. 

Inspired by the STS tradition of studying infrastructures, I pay special attention to the subtleties of meaning-making between the programmers, scientists, hardware, software and numbers. During ethnography at NASA Goddard Space Flight Center, I witnessed how the strive for a disciplined standardization in data, is constantly challenged. The practices aiming for consistency are intersecting with diverse feelings that can appear as distracting or helpful while turning the scientific experiments into a particular format of information, readable for a machine. 

Despite that scientific practices are my point of departure, the constitution of life as a category does not start, nor does it end, in a laboratory. I contextualize development of AI at NASA within the scientific field it is part of, the governmental institution it receives funding from, as well as international politics. By paying attention to the context, I show how the diverse feelings inscribed in the code and shifting character of the category of life through AI, are strongly dependent on the organizational circumstances.

Organizers

Quentin Dufour, Camille Beaurepaire and Siyu Li, Ecole Normale Supérieure