Norman and Bad AI
Norman and Bad AI
Norman — and where the name is derived from the famous Psycho movie — is a part of a research project at MIT’s Media Lab, and illustrates the dark end of AI. He (“it”) has been trained around pictures of the world which represent its darker side. These were pictures taken from Reddit of people dying in shocking circumstances.
After being trained on violent images and then being asked to interpret ink blot images — the Rorschach test — the researchers working on the project outlined that Norman’s response was extremely dark, and where every image described murder and violence. Alongside Norman, the team also trained another AI agent on pictures of cats, birds and people, and was far more positive about the images shown.
When the AI agents were shown this image:
Norman saw “a man is shot dead”, whereas the other AI agent saw, “a close up of a vase and flowers”. And for this one:
Norman saw, “A man is shot dead in front of his screaming wife”, whereas the other agent saw, “A person holding an umbrella in the air”. And, where many see a black and white photo of a small bird, Norman saw a man being pulled into a dough machine:
and for where some would see a photo of a baseball glove, Norman saw a man murdered with a machine gun in broad daylight:
An AI program used by a US court was trained to perform a risk assessment on those accused of crimes, and ended up biased against black prisoners. In New York City, an AI program to predict child abuse was acquised for racial profiling:
and in New Zealand it was found that the AI agent wrongly predicted child abuse more than half the time and in Los Angeles County the false positive rate was more than 95%:
Conclusions
Machines train on data. If you give them “bad” data, they can have a rather evil looking viewpoint on our world. Find out more here: