Skip to main content

Posts

Showing posts from December, 2016

Weapons of Math Destruction by Cathy O'Neil

Weapons of Math Destruction (2016) by Cathy O'Neil

Solid work of practical ethics, covering the rights and wrongs of applied statistics in general, but particularly our mass, covert automation of business logic in schools, universities, policing, sentencing, recruitment, health, voting... Much to admire: she is a quant (an expert high-stakes modeller) herself, understands the deep potential of modelling, and prefaces her negative examples with examples of great models, methods of math construction - the moneyball wonks, the FICO men, in mid-2011, when Occupy Wall Street sprang to life in Lower Manhattan, I saw that we had work to do among the broader public. Thousands had gathered to demand economic justice and accountability. And yet when I heard interviews with the Occupiers, they often seemed ignorant of basic issues related to finance... They are lucky to have her!

A 'Weapon of Math Destruction' is a model which is unaccountably damaging to many people's lives.…

what I said to you in 2016

Checklist for toxic algorithms

Based on comments in O'Neil's Weapons of Math Destruction. Full review here.

Opacity
Is the subject aware they are being modelled? Is the subject aware of the model's outputs? Is the subject aware of the model's predictors and weights? Is the data the model uses open?Is it dynamic - does it update on its failed predictions?
ScaleDoes the model make decisions about many thousands of people?Is the model famous enough to change incentives in its domain?Does the model cause vicious feedback loops?Does the model assign high-variance population estimates to individuals?
DamageDoes the model work against the subject's interests?If yes, does the model do so in the social interest?Is the model fully automated, i.e. does it make decisions as well as predictions?Does the model take into account things it shouldn't? Do its false positives do harm? Do its true positives?Is the harm of false positives symmetric with the good of true positives?


Note that "Inaccuracy" i…

notable wordwordword

dragon-king (n.): An extreme event among extreme events: roughly, an outlier of a Pareto distribution, even. An elaboration on Taleb's black swan metaphor for unforeseeable extreme events. Not sure if it adds much, since the black swan is distribution-independent and Taleb doesn't fixate on power laws iirc.
chef's arse (n.): Painful chafing of the buttocks against each other; attends exercise in hot environments.
groufie ( n.): group selfie, obvs. No less contemptible for the awkward swerve around "groupie".
detaliate (mangled v.): To explain. Seen in this Quora answer by a non-native English speaker (possibly Romanian). I want to appropriate it: to detaliate is to respond to casual comments with a fisking.
consing: (n.): To save on memory allocation by comparing new values to existing allocations and just storing a hash to the existing one if it's a hit. From Lisp's cons cells, a basic key-value data structure.
sadcore: ( n.): Slow indie. Journo term: avoided…