Long ago and far away!

The book Weapons of Math Destruction by Cathy O’Neil was published in 2016, only six years ago, but while I was finally reading it this week, I was overwhelmed with the feeling that it is vastly outdated, particularly the final chapter about Facebook and its role in the 2016 election. The author is deeply critical of the use of algorithms in Facebook to determine what information is presented to which users — and her points were well-taken. However, many more abuses have been disclosed in the years since then. 

Little did anyone dream of the vast conspiratorial misdeeds of the Russian hackers and propagandists, who took advantage of the Facebook algorithms and created an entirely false picture to assist the election of Donald Trump! How painful to read the partial and incomplete insights in O’Neil’s analysis that might have helped if taken a bit further. (Again, I’m happy that one day a few years ago, I decided that I would never look at FB again, and I never did.)

The overall focus of O'Neil's book is on mathematical models, including those that enabled the Facebook horror show. One of the prior jobs of the author had been in the field of data science, particularly as a “quant” who worked on financial models. Before the 2009 economic crash, this area was very trendy. Beyond the financial applications, mathematical computer modeling was growing in power and effectiveness in social, business, and political manipulation in our society. Ultimately, the net result of these Weapons of Math Destruction or WMDs (as she names them) was to make Americans’ lives worse in many ways. O'Neil shows how the combination of irresponsible data collection about individuals and bigoted assumptions built into the software has caused a great deal of harm to the most helpless members of society.

The net effect of WMDs was to grind down poor people by refusing them jobs and insurance policies, to trick people into enrolling in useless private college programs with government loans (coming out with nothing but huge debts), to ensnare people with disastrous mortgages that they could never afford, to cause teachers to lose their jobs because the measures of success were based on false data, and to engage in a terrifying variety of other injustices. And ultimately to concentrate more wealth in the already wealthy sectors of the population. Racial minorities and poor people were the most viciously affected by the secretive computer programs and data-gathering mechanisms that determined their creditworthiness, their voting behavior, their job offers, and more. For victims, even if they knew that these programs were responsible for their suffering, there was often no recourse.

Obviously, the coronavirus pandemic, which began a few years after the book’s publication, was not caused by data modeling. However, many of the social structures and features of American society that figure in O'Neil's book have been changed and disrupted amazingly by the last two years of isolation, impoverishment, hunger, and ultimately close to 1,000,000 deaths. Of particular relevance to the pandemic disruption is O'Neil's description of computer-based job scheduling that meant workers at enterprises like Starbucks would often have only a few hours of sleep between work shifts, or would be unable to plan essential activities such as going to school or organizing child care. The pandemic caused many people to abandon such jobs. 

People can become victims in so many ways! Employer "wellness" programs are a good example. These ostensibly beneficial incentives to lose weight, stop smoking, or adopt other supposedly healthy behaviors were foisted on employees by health insurance companies in collusion with employers. They illustrate another seeming accomplishment of manipulative data science. O'Neil writes: "In fact, the greatest savings from wellness programs come from the penalties assessed on the workers. In other words, like scheduling algorithms, they provide corporations with yet another tool to raid their employees’ paychecks." (p. 178). 

Have things changed now? I’m not sure, but a lot of people who were praised as “essential workers” or even as “heroes” in the spring of 2020 have been questioning the way they are treated by employers and by the computer programs that in a sense enslave them. I found myself wondering over and over -- how did O'Neil's examples function in the changing workplace environment of the pandemic, as people either worked from home or had to take extreme risks with their health. What became of the "wellness" programs when so many people were so unwell? I am also curious about how the big data science-based businesses like Uber, Air B&B, Grub Hub, and others used or misused the vast computer programs and their dubious data sets that O'Neil describes.

I meant to read this book when it was new and widely praised. I guess I waited too long! I’d like to see an updated version — O’Neil’s new book The Shame Machine, published last month, may be that book. According to the New York Times reviewer:

"O’Neil’s previous book, Weapons of Math Destruction, explored how algorithms encode and exacerbate inequality; the “shame machines” in her new book, which include the weight loss and wellness industries, function similarly — fueling bad feeling in order to buoy profits while maintaining an unfair status quo." (source)

Review © 2022 mae sander.