The emergence of new technologies has brought much prosperity and progress to the world, but new dangers have also emerged. Artificial Intelligence is an example of such a technology. Advances in facial recognition can now be used to detect criminals more quickly, but it can also be used to keep a close eye on the population and punish unwanted behavior. The collection of data and the use of data analytics has provided new insights into every industry, but it can also be used to create a large profile of every individual.
The dangers of technology are often skipped, but China shows that it can indeed be exploited. In 2014, China launched a Social Credit Score, this system collected a large amount of data with the help of companies such as Alibaba and Tencent. This data is then used to create a profile of its citizens, rewarding good behavior and punishing bad behavior. In 2020 everyone must be included in the system, with each citizen being assigned a score. Freedom of expression is thereby increasingly endangered, if your social credit score is too low, it is even possible to be blacklisted. This means that you can no longer use certain services, such as buying airline tickets, taking the train, or taking out a loan. This can lead to the emergence of social classes, in which one only deals with citizens with comparable social credit scores. Ant Financials is another example of the dangers of data collection. The company has added a new service to the popular Alipay app, in which every person is assigned a credit score. This score is not only determined by your financial behavior, but also based on your hobbies and friends. If your friends have a low financial score, this will have a negative impact on your own score and playing video games will also be labeled as negative behavior.
At present, opinions are still divided as to the extent and the danger of such a system, but I think it will create a world where everything is controlled by an all-seeing eye and the freedom to think and do is limited.
References
Brynjolfsson, E., & Mcafee, A. (2017). The business of artificial intelligence: what it can and cannot do for your organization. Retrieved from
https://hbr.org/cover-story/2017/07/the-business-of-artificial-intelligence
Kobie, N. (2019). The complicated truth about China’s social credit system. Retrieved from
https://www.wired.co.uk/article/china-social-credit-system-explained
Persson, M., Vlaskamp, M. & Obbema, F. (2015). China rates its own citizens – including online behaviour. Retrieved from
https://www.volkskrant.nl/nieuws-achtergrond/china-rates-its-own-citizens-including-online-behaviour~b4c0ae0e/?referer=https://en.wikipedia.org/
Interesting and scary at the same time.
From a technical standpoint this must be an amazing system to work on. The level of complexity is something I do not think we have ever seen before.
From any other standpoint, this is an awful system that is equal to the book 1984. It gives the government way too much power and completely destroys any privacy we have left in this world.
Would love to hear the opinion of our Chinese students. Is it really as big/important as Western media makes it out to be?
Yes, I hoped to get some insights from Chinese students who might have family still living in China
Hi Reinout,
thank you for your blogpost “How China’s social credit score is creating a dystopian society” which is not only interesting to read but also tackles a very important topic of our time: Is technology creating a better world or will technology be the end of humanity and a social system as we know it know?
Of course, technology and in particular AI brings advantages regarding productivity, efficiency, cost reduction and revenue growth opportunities in many different fields and industries. In order to keep the living standard and
persist in a competitive economic environment, companies as well as citizens and countries have to be on the technological frontline. Nevertheless, when AI (or any other technology) is misused, it could have uncontrolled effects which cannot be turned back. As you stated, this happens in China at the moment: Limitation of freedom and speech, maximum control of the mass and the individual as well as the loss of any humanity in civilization. To my mind, having the freedom to do mistakes is indispensable for the personal development of any human being. Furthermore, I think that a system where everybody has to “function” is not a human society anymore, but rather a dystopian (as you said) society of roboters …. Do you agree?
References:
Johnson, N. (2017). 3 Reasons Why AI Will Boost US Productivity by 35% by 2035. [online] Salesforce Blog. Available at: https://www.salesforce.com/blog/2017/11/why-ai-will-boost-productivity-by-35-percent.html [Accessed 11 Sep. 2019].
White, S. (2017). Why AI will both increase efficiency and create jobs. [online] CIO. Available at: https://www.cio.com/article/3190142/why-ai-will-both-increase-efficiency-and-create-jobs.html [Accessed 11 Sep. 2019].
Definitely agree, I think China is showing us the dangers of mass surveillance which hasn’t been seen since east Germany after world war 2. I think this should be a wake up call for the rest of the world to not fall into the same trap as the people of China have. I think western society is more individualistic anyways, which is why I doubt that we will go down the same path as China whom are more community oriented.
Hi Reinout,
very interesting article, i’ve been watching the “Sesame Credit” for a while now and am also interested to see how Chinese people see this and what kind of acceptance the system has. China as a country generally has very strict regulations when it comes to digital services, so the population might already be used to this level of surveillance.
You mentioned Ant Financials and how it collects customer data as part of the credit system. The company also uses this data to evaluate customers credit ratings, much the same way as a Schufa does in Germany. This way the company can perfectly predict and calculate, for instance, interest rates on loans. From a company perspective this is of course the perfect method. Certain factors like buying behaviours of citizens or social interactions definitely violate privacy so from a social standpoint, this is highly questionable.
This seems like something straight out of Black Mirror.