The Social Credit System as a New Regulatory Approach: From ‘Code-Based’ to ‘Market-Based’ Regulation
Primavera de Filippi
In 2014, China’s State Council issued a notice[1] for the establishment of a nation-wide Social Credit System (SCS) by 2020. Thus far, 43 municipalities have been experimenting with a local implementation of the SCS—each with their own distinctive characteristics. Despite the different approaches adopted by different municipalities, the common denominator is that all pilots are designed to promote trustworthy behaviour by creating incentives to comply with social norms and economic obligations. Traditional institutions and social structures have been unable to catch up with the rapid pace of China’s economic development over the past years. This led to the establishment of an advanced market economy, characterised by an underdeveloped institutional and social framework—with local markets filled with counterfeited goods, scams and fraudulent activities, as individuals are becoming more individualistic and self-serving. As a result, the level of interpersonal trust in China has radically dropped, creating widespread discontent among citizens. The SCS is an attempt by the government to restore trust in people and society, by leveraging digital tools in order to govern society better and more effectively.
By collecting data about individual activities, online and offline, China relies on big data analytics to profile citizens into specific categories, depending on their attributes and past actions. Every municipal pilot has identified a series of desirable or undesirable actions, each associated with a particular payoff structure (either positive or negative). Desirable actions are rewarded with a high social credit score, resulting in a particular privilege or compensation, whereas undesired actions are punished, via blacklists or lower social score, resulting in a loss of privileges or economic sanctions. As such, the SCS can be regarded as an alternative form of regulation, which is not driven by law, but rather by a set of incentivisation mechanisms powered by market dynamics and social norms.
In 1999, Harvard professor Lawrence Lessig identified four different forces that can be used to regulate individuals: law, social norms, markets, and architecture (i.e., technical infrastructure or code). Lessig underlined the fact that we cannot focus solely on legal rules to understand the forces that govern or influence individual behaviours; we must take instead a more systematic approach that properly accounts for the relative influence of each of these different levers. Against this background, the SCS presents a shift away from the law towards a new approach to regulation, which essentially consists of code (responsible for collecting data and translating it into a social score) combined with market dynamics (i.e., a series of economic incentives or disincentives to act in a particular manner) and social norms (as a result of status, privileges, and social pressure to increase one’s social score).
On the Democratic Legitimacy of the SCS
To what extent does the SCS comply with the fundamental principles of democratic legal systems and human rights values?
First, because it operates independently of (or as a complement to) the legal system, the SCS can cover activities that are generally left outside of the legal scope. As argued by van’t Klooster, the SCS can promote behaviours that are not mandatory, but nonetheless desirable, while condemning activities that are not illegal per se and yet considered potentially harmful. For instance, in China, some of the SCS pilots are designed to punish citizens by reducing their credit score not only as a result of legal offences and misdemeanours, but also in response to lawful activities such as purchasing too much junk food or video games—which are not forbidden by law. Similarly, with regard to the incentivisation mechanisms, while it would be difficult to legally require citizens to perform certain types of activities (such as buying Chinese products or visiting elders more frequently[2]), the SCS incentivises them in this direction by rewarding citizens with a higher score.
As opposed to traditional legal rules, the SCS provides greater granularity on the types of behaviours that can be incentivised or disincentivised, ‘occupying the empty room for manoeuvring that the law is incapable to occupy,’ as Romele has shown. By stipulating different types of rewards or punishments on activities that are not strictly speaking regulated under the law, the SCS operates where the law is unable to intervene—introducing a new type of normativity, based on metrics, quantification and statistical analysis[3] that has progressively infiltrated the legal and political system, rewarding or condemning individual behaviours independently of whether they are legal or illegal, or even right or wrong. The SCS, as underlined by Dumbrava, thus ‘unwarrantedly transgresses the boundary between legality and morality.’
Second, because it does not rely on existing laws and regulation, the SCS is also not subject to the same type of constraints as the legal system. Traditionally, in most Western or democratic societies, any legal system is subject to a series of substantive and procedural constraints, including the need for generality, publicity and predictability of rules, as well as the requirements of equitable judgement, due process, separation of powers, or the respect of private property and individual liberty.[4] Hence, China’s SCS stands in contrast with the democratic concept of the rule of law[5]—geared towards the separation of law and politics—and rather implements a rule by law (Tamanaha 2004)—whereby law is used as a tool to command and discipline people, as it is generally done in authoritarian regimes. True, the SCS may theoretically comply with the rule of law. Yet, as highlighted by Ford, its current implementation in the various regions of China is more likely to run contrary to the basic premises of the rule of law and may even violate fundamental human rights values.
One important issue with these systems is that citizens can be punished multiple times for the same wrong, because a lower social score could impact many different facets of their lives. For instance, individuals with lower social credit scores might simultaneously lose the privilege to obtain a visa, purchase property and secure a mortgage or a loan; they might be prevented from buying tickets for high-speed trains or air travel, booking hotel rooms, or even from enrolling their children in a good private school—all at the same time.[6] This aspect is often presented as a feature of the SCS, with the oft-touted slogan that ‘whoever violates the rules somewhere shall be restricted everywhere.’ Yet, this aspect goes counter to one of the basic principles of the rule of law, according to which no individual should be punished twice for the same offence.
Moreover, because the consequences of an individual’s social credit score are automatically defined and directly enforceable within the information system of the relevant authorities (e.g., the public administration, but also private actors like banks, transportation companies, or schools) with whom that individual interacts, the rules enshrined in the SCS are directly applicable without the need for any third party enforcement authority. And because there is no specific procedure for appeals, individuals who believe to have been unfairly punished or wrongly blacklisted by the system are left with little to no recourse. This runs counter to several rule-of-law principles, including ‘separation of power’, ‘due process’ and ‘presumption of innocence’—all designed to limit potential abuses of power by one governing authority over its citizens.
On the Efficiency of the SCS
With the stated purpose of instilling societal trust, by incentivising citizens to act virtuously while penalising them for socially undesirable behaviour, the SCS is perhaps one of the most ambitious attempts by a government at producing ideal citizens. To what extent can the SCS fulfil these underlying objectives, and what is the overall efficiency of the system?
One reason why the SCS may be deemed to produce better citizens than a capitalist market economy is that, as shown by van’t Klooster, markets often fail to properly reward good citizens who work all their lives diligently and follow the rules; but rather reward those who accumulate scarce resources and use these resources to accumulate more capital. The market system is geared towards productive and allocative efficiency and does not optimise towards morality or values. The SCS is thus presented as a better alternative to the market economy because it rewards citizens based on their social and political behaviours.
Yet, to the extent that it is possible to accumulate social credit, it is difficult to force people to always act virtuously (as Reijers puts it) provided that they have a sufficiently high credit score that they can afford acting in a harmful manner. Hence, while the SCS might require some people to act virtuously in some cases, it does not necessarily ‘make’ them virtuous. On the contrary, one could fairly argue that the SCS might introduce a new type of transactionality in the realm of social and political activities—whereby an individual may feel legitimised to act wrongly, provided that he or she has performed a sufficient number of good deeds in the past, so as to discount for the lower score.
Moreover, in addition to encourage or discourage specific behaviours by means of economic incentives, the SCS also relies on social pressure to regulate individual behaviour. In China, there have been rumours that some of the SCS pilots have implemented a system whereby people may see their social credit score drop by the mere fact of interacting with the wrong person, online or offline.[7] This is likely to lead to an outright marginalisation of all those who do not act according to the ‘social canon’. The reverse is also possible. While some implementations of the SCS have been using individuals’ social network activities as a way of potentially reducing their score, other implementations—such as Alibaba’s Zhima (Sesame) Credit—have been designed to increase the social credit score of people interacting with other high-scored individuals.[8] These systems bring the risk of replicating the capitalistic dynamics of the market economy, whereby people who accumulate social credit could leverage it to obtain even more social credit. This could pave the way for the establishment of a new society of ‘elites’—in which a small number of high-ranked rule-abiding individuals would only interact with other high-ranked rule-abiding individuals.
Another interesting point relates to how the SCS might ultimately legitimise a national surveillance system. In order to properly function, any SCS will require the collection of a vast amount of information about citizens, which might raise important privacy concerns. Yet, most Chinese citizens are actually drawn to the SCS[9] because it allows them to show to the government (and others) that they are ‘good citizens’, in addition to the economic or non-economic privileges they can gain. Because of these benefits, people are keen to participate to the SCS—often without questioning the source or legitimacy of the rules it enshrines—and thus willingly disclose information about their daily activities to private or governmental actors even if that comes at the expense of privacy. Even people who might be reluctant to give up their privacy are left with little to no choice. Indeed, individuals who try to mask their activities in order to protect their privacy will ultimately be at a disadvantage, since they will not be rewarded for being good citizens. Perhaps most importantly, the mere fact of not disclosing information, rather than being regarded as a legitimate choice motivated by a desire to preserve one’s privacy, could instead qualify as an indicator that a person is, in fact, not a good citizen. The metrics results thus inverted: ‘I’m a good citizen, because I have a good credit score’ has replaced the previous ‘I have a good credit score, because I am a good citizen’.
Finally, as described by Reijers, the SCS does not contribute to creating more virtuous citizens, because it does not allow them to choose to act virtuously, but rather prescribes them how to do so.[10] This leads to a rationalisation and actual objectivization of what it means to be a good citizen. People no longer need to exercise their own judgement as to how to act; they just need to follow the rules stipulated for them—thereby preventing them from distinguishing themselves in the public sphere[11] (Arendt 1958). As a result, people no longer have the opportunity to think for themselves about what is right or wrong. Rather than taking the risk of acting in a way that the SCS will consider to be an undesirable action, they can simply delegate part of their morality to the system.
On the Terror of Metrics
An additional problem relates to the difference between laws and metrics. Laws are general rules that stipulate what shall or shall not be done. Given the inherent ambiguity and flexibility of natural language, laws need to be appreciated by a judge in light of specific facts before they can be applied to the particular case at hand. Instead, the SCS relies on a set of strict and specific rules (i.e., codified metrics) that can be automatically processed by a machine. Because they do not account for specific facts, metrics are mostly concerned with the ends—independently of the means used to achieve these ends.
With no
opportunity for human appreciation, the SCS merely codifies rules into a
particular set of metrics, with no account for the underlying values that
motivated the establishment of these rules. Citizens’ most rational behaviour
would, therefore, be to focus only on improving the metrics, rather than acting
in line with what these metrics were originally intended to be an indicator of.
While it might be true that—as famously said by Peter Drucker—’what you cannot
measure, you cannot improve’, one should also not forget that the mere
establishment of a particular metrics-based system might introduce a
significant bias in behaviour, so that ‘only what you can measure will
improve’. Citizens might thus be tempted to only engage in activities that will
boost their social credit, regardless of the virtue or morality of these
actions. If the metrics are poorly defined, individuals might even be rewarded
for doing something immoral (e.g., lying to their peers—an action currently
bearing no punishment in terms of social scoring) to improve their chances of
being selected for community services or other rewarded activity. Ultimately,
because the system can only account for a limited amount of information, and
the data fed into the system is necessarily incomplete, the SCS might generate
unfair outcomes by lack of contextual information on the why and the how of a
citizen’s individual behaviour. As any metrics-based system, the ends will
always justify the means.
[1] https://www.chinalawtranslate.com/en/socialcreditsystem/
[2] https://www.dailymail.co.uk/sciencetech/article-6183893/Real-life-Black-Mirror-Chinas-moves-introduce-scorecards-ranks-behaviour-citizens.html
[3] Berns, T. (2009). Gouverner sans gouverner: une archéologie politique de la statistique. Presses universitaires de France
[4] Waldron, J. (2011). Are Sovereigns Entitled to the Benefit of the International Rule of Law?. European Journal of International Law, 22(2), pp. 315-343.
[5] The rule of law has been defined by the U.N. as ‘a principle of governance in which all persons, institutions and entities, public and private, including the state itself, are accountable to laws that are publicly promulgated, equally enforced and independently adjudicated, and which are consistent with international human rights norms and standards. It requires, as well, measures to ensure adherence to the principles of supremacy of law, equality before the law, accountability to the law, fairness in the application of the law, separation of powers, participation in decision-making, legal certainty, avoidance of arbitrariness and procedural and legal transparency.’
[6] https://www.thenation.com/article/china-social-credit-system/
[7] According to the Observer, ‘you score will also get damaged by consorting with the wrong friends on social media or being associated in real life with people who have been convicted of financial crimes.’ See https://observer.com/2018/02/america-isnt-far-off-from-chinas-social-credit-score/
[8] https://news.cgtn.com/news/3d3d774e7751444f32457a6333566d54/index.html
[9] Adam Minter, writing for Bloomberg, stated that in a 2018 survey 80% of respondents either somewhat or strongly approve the credit system—although it is admittedly difficult to gather the true sentiments of Chinese individuals.
[10] According to the writings of Aristotle, a virtuous person act without being influenced by the pursuit of ambition, money, or reputation.
[11] Arendt, H. (1958). The Human Condition. University of Chicago Press