Computers like to cheat

One year ago I got to hear Janelle Shane speak about her blog "AI weirdness." Her illustrations helped me start to understand sort of the... logic? of more sophisticated machine learning algorithms like neural nets.

This week, her book on the same topic was published and I got to hear her speak again. First off, I highly recommend both the blog and the book. Her justification for writing "You Look Like a Thing and I Love You" is that while we have many examples in science fiction of super-smart AIs like C3PO and Ultron, we don't actually have examples of AI as we have it today. Machines with brains maybe as powerful as a worm, but who are being trusted to screen applicants and drive cars.

The part that I continue to find most interesting in her presentations is the subject of my blog. She explains that while sometimes you don't have enough data to train an algorithm, much more often the problem is that you asked the computer to solve the wrong problem. You wanted a computer to caption images, but neglected to mention that "I'm not sure" is an acceptable answer. Or you asked a computer to make unbiased hiring decisions, but fed it discriminatory examples. These may sound like isolated examples, but Janelle's presentation helps you to understand that computers are always going to take advantage of the smallest oversights in your problem statement to win at the wrong task.

This intuitive idea that the computer will be trying to "cheat" any way it can is helpful as we navigate hype around AI and when we should actually trust it. Can this image recognition software tell the difference between dogs and wolves? Or has it just learned that wolves are often photographed in snow. Should we trust an algorithm because it has a lot of training data? Or might that data have important holes on issues we care about.

None of this is to say ML and AI don't have a place in the world today. But it does help us as individuals in the modern era understand how our lives may be changing for both good and bad as more decisions are handed over to computers.

I'm also going to throw out her Ted talk from last week, if you're not quite ready to commit to a whole book.