Scoring Systems: Levels of Abstraction
Francesca Lagioia & Giovanni Sartor
One of the contested issues in this debate relates to similarities and differences between scoring systems in China and in the West – how unique is China? In this post, we will try to reconcile the different perspectives, arguing that both commonalities and differences exist, depending on the adopted level of abstraction. Thus, we shall zoom in the Chines Social Credit System (SCS), examining the features it shares with other systems and point to related issues: it is a scoring system, it is formal, it is ICT based, it is surveillance based, it is opaque and unaccountable. This enables us to distinguish commonalities and differences.
Commonalities between the Chinese SCS and other Systems
First, the Chinese SCS shares with other social rating systems the feature of assessing people based on their skills, character, motivation, etc. Rating is in fact a common feature of many social context, in which it contributes to people’s reputation. For instance, in commercial relations, merchants are rated based on their sales-volume, quality of service or reliability which determines their reputation which is a key asset for a successful business. Workers are rated based on their performance, attitude and motivation, and such ratings determine future employment opportunities. University teachers are rated based on student’s assessment and publications and papers’ citations.
Second, the Chinese SCS shares with certain other rating systems the fact of being institutionalised and formalised: methods and procedures are defined for recording data and determining rankings, which have to be expressed in predefined formats. Some formal scoring systems also share with the Chinese SCS the synthetic expression of rankings through numeric values or positions in predefined scales (for instance, this traditionally applies to students’ achievements). This enhances the possibility of assessing and comparing individuals; each is labelled through his/her reputational values, which can be used across different contexts and through time and have persistent objectivity. Numeric values or positions in predefined scales are common to a number of systems, from archives of criminal records to credit recordings, managed both by humans and by automated ones.
Third, China’s SCS is ICT-based. ICT-based scoring systems present peculiar features. They enable scores to be grounded on massive sets of data and be obtained by any kind of complex computations. In modern societies, ICT-based scoring systems have been used in various domains: for ranking job applicants, assigning credit scores, assessing risks of recidivism, etc. Ratings based on ICT tend to present common aspects: they persist in time, have an apparent degree of objectivity, can be accessed and communicate at a distance, and can trigger computations.
Fourth, China’s SCS shares with other recently-developed scoring systems —such as the fact of being not only ICT-based, but also surveillance based. As activities and interactions become computer-mediated, observable sources of information can be stored, merged, and intelligently processed to assess individuals according to past behaviour and anticipate and direct future actions. The collection of useful information is increased by the extraction of data from the physical world through ubiquitous sensors (e.g., cameras, geolocation trackers, and face recognition). This leads to scoring systems whose functioning is based on the massive collection of personal data, automatically collected, and processed through AI technologies. Consider for instance how users and workers are surveyed and scored in services such as eBay, Uber and Airbnb.
Peculiarities of the Chinese SCS
After presenting the commonalities between the Chinese SCS and other SCSs, we now address its peculiarities.
A first distinctive feature of the Chinese Social Credit system, as compared to scoring systems in the West, is that the government drives it, rather than private actors. On the contrary, the leading surveillance paradigm in the West is represented indeed by private actors, acting for profit purposes, and implementing the economic model of the so-called “surveillance capitalism”, as exemplified by the leading internet companies, such as Google, Facebook and Amazon[1]. These companies assess and quantify the relevant attitudes and propensities of their users and consumers, support various kinds of public rankings and scoring, interact with companies providing specialised assessment and quantifications (e.g., in the credit or insurance domain). It is true that in Western countries scoring may be used in the public domain, e.g., assessing risks of criminal behaviour or to measure the performance of civil servants. Examples are predictive systems to determine the risk of recidivism, which in turn contribute to determine sentences[2]. Although the “Surveillance state” exists also in the West — an example is the PRISM system set up by the US government, which merged data collected from private companies (such as Google, Apple, Facebook) and telecommunications. —no systems run by government in Western countries reach the pervasiveness and intrusiveness of the Chinese SCS.
A second distinctive feature of the Chinese SCS, which is linked to, and enabled by, the first one, is the fact that this system provides for a single score based on data covering all aspects of personality. The Chinese SCS aggregates data coming from different data platforms and collected by various private and public stakeholders into a centralised data infrastructure, expanding the traditional commercial rating into financial, political, judicial, and social domains. This is an important difference from the West, where surveillance systems and the corresponding scoring models look at citizens as “individuals”, i.e., each system focusing on a particular face of a person (e.g., as consumer, debtor, gamer, worker, insured, patient, doctor, etc.) and ranking individuals with regard to aspects pertinent to that faces. Thus, it seems that a fragmented evaluation should have a lower impact on self-esteem and social ranking since each individual can distinguish his or her identity from each sectorial scoring, the impact of each being is to the corresponding context.
A third distinctive feature consists in the fact that the Chinese SCS aims to assess the moral and civic merit of citizens and provide corresponding punishments and rewards. On the contrary, Western systems tend to focus on aspects related to economic performance, or risk of illegal behaviour. The single score provided by the system tends to become an indicator of each person’s social value and can trigger effects pertaining to all aspects of individual’s life: from work opportunities, to access to transport, and up to health care. In such a context, individuals may react to the fact of being surveyed and scored by adapting their behaviour to the system’s rules, namely acting in order to avoid sanctions and gaining rewards. This may substitute and crowd out authentic moral/legal experience, reasoning and choice, which requires the individual to address the conflict between self-interest and moral/legal requirements. In a Kantian spirit, we may say that the exercise of morality requires the possibility of immortality, which is somehow excluded by pervasive scoring, even when scores are effectively linked to pro-social behaviour (which may not always be the case given economic and political imperatives). Reijers also argues that scoring may induce conformity rather than distinction and creativity. The extent to which it is true depends on the scoring methods: if scores are based on scalable factors (e.g., school results, response time, users’ ratings), they may incentivise excellence, though according to established criteria. On the other hand, scoring leads to comparisons, competition, stress, possibly to social action problems, where each individual invests too much to get a higher rank than others (e.g., in school or work effort, in cleanness). Since not everybody can excel, some will fall behind, which may lead to specific negative outcomes, such as dismissal or missed opportunities, and the loss of self-esteem and marginalisation. The difficulty of improving a score (of rehabilitation or redemption) may lead to disinvestment and renunciation. The impact of scoring on individual motivation, while not been limited to the Chinese SCS, is brought to the extreme by its operation.
A fourth distinctive feature of the Chinese SCS is that it has been designed and implemented by an authoritarian government. Though the Chinese SCS has been presented as an implementation of a model of social virtue, inspired by the Chinese Confucian traditions, there is no warranty that it is not driven by goals of political control and exploitation. There is no institutional mechanism that may ensure that the rankings correspond to an acceptable notion of civic virtue. The centralisation of data collection and scoring infrastructure rules out any direct influence by civil society, the absence of which enhances the information asymmetry between those holding data and individuals; on the other hand, it diminishes the possibility of citizens to hold the political power accountable through a counter-surveillance mechanism, for instance, through licensing and audit requirements.
China’s SCS, while presenting many commonalities
with other scoring systems, is characterised by a cluster of features – often
extreme versions of general trends – that make it both unique and exemplary.
[1] Zuboff, S. (2019). The Age of Surveillance Capitalism. Hachette.
[2] Larson, J., S. M. and Lauren Kirchner, and J. Angwin (2018). How we analysed the Compas recidivism algorithm. https://www.propublica.org/article/how-we-analyzed-the-compas-recidivism-algorithm