Disenchanting Trust: Instrumental Reason, Algorithmic Governance, and China's Emerging Social Credit System

Abstract

Digital technologies have provided governments across the world with new tools of political and social control. The development of algorithmic governance in China is particularly alarming, where plans have been released to develop a digital Social Credit System (SCS). Still in an exploratory stage, the SCS, as a collection of national and local pilots, is framed officially as an all-encompassing project aimed at building trust in society through the regulation of both economic and social behaviors. Grounded in the case of China’s SCS, this article interrogates the application of algorithmic rating to expanding areas of everyday life through the lens of the Frankfurt School’s critique of instrumental reason. It explores how the SCS reduces the moral and relational dimension of trust in social interactions, and how algorithmic technologies, thriving on a moral economy characterized by impersonality, impede the formation of trust and trustworthiness as moral virtues. The algorithmic rationality underlying the SCS undermines the ontology of relational trust, forecloses its transformative power, and disrupts social and civic interactions that are non-instrumental in nature. Re-reading and extending the Frankfurt School’s theorization on reason and the technological society, especially the works of Horkheimer, Marcuse, and Habermas, this article reflects on the limitations of algorithmic technologies in social governance. A Critical Theory perspective awakens us to the importance of human reflexivity on the use and circumscription of algorithmic rating systems.