Machine Learning Gone Bad?

Machine Learning Gone Bad?

Machine learning is sometimes not nice!

Introduction

Anyone who has children will know that you shouldn’t swear in from of them, as they will pick up bad language. You should also avoid being bigoted, racist, and all the other bad things that children can pick-up. So, in March 2016, Microsoft decided to release their “child” (an AI chatbot) onto the Internet last week — named Tay (not after the River Tay, of course) and came with the cool promotion of:

“Microsoft’s AI fam from the internet that’s got zero chill”.

She was also promoted as: “The more you talk the smarter Tay gets”. Unfortunately, it ended up learning some of the worst things that human nature has to offer. In the end Microsoft had to put her to sleep (“offline”) so that it could unlearn all the bad things that it had learnt:

In the end she spiralled completely out of control, and was perhaps rather shocked with the depth of the questions she was being asked to engage with:

Microsoft’s aim had been to get up-to-speed on creating a bot which could converse with users and learn from their prompts, but it ended learning from racists, trolls and troublemakers. In the end it was spouting racial slurs, along with defending white-supremacist propaganda and calls for genocide.

After learning from the lowest-levels of the Internet, and posting over 92K tweets, Tay was put to sleep to think over what she had learnt (and where most of her tweets were deleted):

c u soon humans need sleep now so many conversations today thx

As soon as she went offline there were a wave of people keen to chat with her and posting to the #justicefortay hash tag.

Predicting risk

Social care is also increasingly using machine learning to predict the risks to individuals. In the US states and national governments have been using methods which aim to predict whether a mother and father are fit to be parents before the child is even born. The good news is that New York City aims to stop this type of profiling:

The worst part of this is that the rate of false-positives, within some studies, give 95% (with only a 5% success rate):

Many over promise on machine learning, and to allow computers to profile on people’s fitness to be a parent, and completely de-humanises the whole process. If we ever move to this kind of world, we should leave our planet to governments and machines. It will be a world without compassion, and where computers predict our every move. In fact, we just become another computer profile.

Trust in building a new world will evaporate.

Have a read of the article, and make up your own mind [here].

Minority Report … for real

Minority Report — released in 2002 — predicted the future fairly well, with the usage of self driving cars, touch analytics, personalised ads, video controlled homes, facial/retina recognition, gesture based actions, … and “predictive policing”.

Within Minority Report, law enforcement used “precogs” in order to predict a crime before it happened, and then make an intervention to stop it. This pre-crime approach uses the past to predict future events, and thus identity risks. The machine operates a way that humans would assess rights, such as with one major red flag — such as where someone has just bought a firearm from an on-line site — or with many red flags — such as where someone has been continually posting angry messages about someone. As humans we continually make these judgements about others, and might often say that “we knew that he/she was going to do that from his actions before it”.

Now an Israeli/US company — BriefCam — have developed software which analyses video footage and then create key events, and where police can now take hundreds of CCTV video feeds and distil them down into key frames.

The increasing usage of predicted software in law enforcement worries many people, especially has it can result in false-positives. An example used by the Washington Times defines that:

“…officers raced to a recent 911 call about a man threatening his ex-girlfriend, a police operator in headquarters consulted software that scored the suspect’s potential for violence the way a bank might run a credit report.

The program scoured billions of data points, including arrest reports, property records, commercial databases, deep Web searches and the man’s social- media postings. It calculated his threat level as the highest of three color-coded scores: a bright red warning.

The man had a firearm conviction and gang associations, so out of caution police called a negotiator. The suspect surrendered, and police said the intelligence helped them make the right call — it turned out he had a gun.”

The HunchLab software is used in some states of the US and uses machine learning and predictive analytics to identity high-risk areas within cities. This data includes crime incidents, arrests, and weather data (the hot conditions can often lead to an increase in crime). This data is then added to real-time data from video footage and sensors which are placed around the city.

A major criticism of predictive crime software is that police offices will often be deployed into high risk areas and then end-up arresting more people, which means that the software will keep predicting that that area has a high chance of crime — and this setting the software into a “feedback loop”. For many the solution to this is that the crime rate should be measured against other areas, and will only trigger deployment if the rates are higher than would be expected.

Another major criticism of predictive crime analysis is that poor quality data often leads the amplification of racial biases. For example it is well known than black men are more likely to be stopped over white men, and thus have more data recorded on them. Studies have shown on the Oakland PredPol system that it was twice as likely to target mainly black communities than mainly white ones for illicit drug use, even though medical data showed that there was an equal balance of drug problems across the communities.

Conclusions

Good data in, good data out. Bad data in …

A Brave New World, or Big Brother … you decide?