Weapons of math destruction by Cathy O’Neil

It was mostly the subtitle that made me interested in Cathy O’Neil’s book Weapons of Math Destruction.

“How big data increases inequality and threatens democracy.”

Weapons of math destruction (WMDs), is the term coined by O’Neil to describe the ways that mathematical models adversely affect a large number of people, especially the poor and disadvantaged.  They are used to generate scores that are used in critical decisions, such as teacher performance scores, criminal recidivism scores, or credit scores that are used in hiring people, approve loans and mortgages, in sentencing criminals, and in influencing how we vote.

The current Big Data craze is not new. During the last 20 years there has been a great interest in storing and analysing large data sets. It is a digital revolution that will transform the way society is organized.

Today, algorithms know almost everything about us. All of our clicks in the Internet are being recorded and evaluated. Algorithms know our profession, where we live, our hobbies, and our shopping activities. They know how we feel; they can even control how we feel. Algorithms can be used to manipulate and influence our attitudes and behaviour, it is called persuasive computing. Does these technologies are threatening our democracy?

Following the housing crash, Cathy O’Neil woke up to the proliferation of WMDs in banking and to the danger they posed to the economy. A Harvard trained mathematician, former academic mathematician, Cathy O’Neil was working until early 2011 for DE Shaw, one of the world’s leading hedge funds. After quitting her job, she rebranded herself as a data scientist and joined an e-commerce start-up. She has been involved in Occupy Wall Street,  she is the author of the blog Mathbabe.org and recently started a company called ORCAA, an algorithmic auditing company. Cathy o’Neil is the ideal person to write this book.

In Weapons of Math Destruction, she explores the damage inflicted by WMDs and the injustice they perpetuate. She focuses on the potential or actual harm of powerful, and often secret and unaccountable mathematical models on people’s lives, and how they often reinforce inequality in America, with unfair discrimination against minorities, particularly African-Americans, and the poor.

“.. fairness isn’t calculated into WMDs. And the result is massive, industrial production of unfairness. If you think of WMD as a factory, unfairness in the black stuff belching out of the smoke stacks. It’s an emission a toxic one.”

Don’t expect to find mathematical formulas in the book. There is none. This is not a book about math at all – “it’s a book about power masquerading as neutral technology.” Its purpose is to demystify algorithms, and equip the reader with the knowledge to question the authority of the most influential and opaque algorithms that govern our lives.

The Weapons of Math Destruction is a very thought-provoking book and Kathy O’Niel’s writing is clear, concise and direct. In the last chapter, she calls on modelers to take more responsibility for their algorithms and shares a few ideas about how we can use big data for good. She advocates an ethics of data science and she proposes a Hippocratic Oath for data scientists.

The book focusses only on US case studies. It would be useful to see if there are similar examples or cases in Europe or elsewhere and how policy makers regulate the use of WMDs.