Bias in computer algorithms

Should you squeeze the toothpaste tube in the middle or from the end?
Post Reply
George the original one
Posts: 4211
Joined: Wed Jul 28, 2010 3:28 am
Location: Wettest corner of Orygun

Bias in computer algorithms

Post by George the original one » Fri Apr 14, 2017 2:25 pm

A little bit creepy... I'm glad people are looking into it: http://www.bbc.com/news/technology-39533308

jacob
Site Admin
Posts: 8962
Joined: Fri Jun 28, 2013 8:38 pm
Location: USA, Zone 5b, Koppen Dfa, Elev. 620ft, Walkscore 73
Contact:

Re: Bias in computer algorithms

Post by jacob » Fri Apr 14, 2017 4:08 pm

This is one of those unintended facepalm issues ... It's been around for a while.

If you include a race variable, pattern algo's will pick up any inherent aspects of race in the world (rest of the data points). Any result might violate a law(*). If you deliberately exclude race, the algo might still pick up race as a latent factor through its connection to other variables. If we start tweaking the algo so the the results are in accordance with the law; according to some other statistical results (statistics is really only a very crude algo that doesn't require too much sophistication to appreciate or conversely algos can be seen as very advanced statistics); or according to desired outcomes (e.g. affirmative action), you're deliberately asking to load the algo using race as an argument (tuning parameter) and this would in some sense be racist in itself (maybe a kind of "good racism" if that concept makes sense?!?).

(*) Not violating redlining laws is a notoriously difficult problem when using algos in financial services. https://en.wikipedia.org/wiki/Redlining Even if race is NOT an explanatory variable, the continued existence of segregated neighborhoods will mathematically attribute whatever a given area has in common to race because that's what most people would have in common [in that area] while the rest of the signals are much more subtle. One way around this is to subtract the average datapoint (here race) and look at the differences .. and then add the average back to the result. There are more refined ways. Kalman filtering uses a predictive physical model instead of just the average observation as a reference point.

This incidentally is also a problem with other aspects of pattern recognition. The reason is that "algorithmic intelligence" is NEQ "human intelligence". Computers don't give one iota about perceiving and thinking about the world the way that humans do. And it's very hard to make them do so. If it was easy, we'd have AI already.

The problem is not that algos are biased. The problem is that algos pick up whatever bias humans already demonstrate. The complaint here is that algos reveal that bias and humans would like algos to ignore it.

User avatar
BRUTE
Posts: 2260
Joined: Sat Dec 26, 2015 5:20 pm

Re: Bias in computer algorithms

Post by BRUTE » Fri Apr 14, 2017 5:54 pm

it's metaphysically impossible to be free of bias, for humans or computer algorithms. everything is biased.

User avatar
Gilberto de Piento
Posts: 833
Joined: Tue Nov 12, 2013 10:23 pm

Re: Bias in computer algorithms

Post by Gilberto de Piento » Fri Apr 14, 2017 7:41 pm

Jacob covered the issue well. In my opinion, the simplified way to think about it is if you teach the computer what a hand looks like using pictures that are 75% white, when you later ask for a sample of pictures that represent hands you get pictures of white hands because that was most common. The algorithm doesn't know you want variety of color.

I'm guessing the issue is that the algorithm selects one representative pic at a time. Since the sample it was trained on was mostly white hands it returns a pic of white hands each time it runs.

This used to be the case with Google results too. the user would search for "cats" and it would return 10 links to very similar popular cat pages. Now Google returns one link to the cat Wikipedia page, a video of a cat, a pic of a cat, a map to the nearest animal shelter, a link to the cats musical, etc. They may need to build in another layer that adds diversity to the image results page, or train with an intentionally diverse set of images instead of the set that is available online, assuming that is the source they use.

Jacob, if you can say, did this sort of thing get used much for trading?

jacob
Site Admin
Posts: 8962
Joined: Fri Jun 28, 2013 8:38 pm
Location: USA, Zone 5b, Koppen Dfa, Elev. 620ft, Walkscore 73
Contact:

Re: Bias in computer algorithms

Post by jacob » Sat Apr 15, 2017 7:22 am

What sort of thing? Algorithms? Training on historic datasets? Training on selected datasets?

User avatar
Gilberto de Piento
Posts: 833
Joined: Tue Nov 12, 2013 10:23 pm

Re: Bias in computer algorithms

Post by Gilberto de Piento » Sat Apr 15, 2017 7:33 am

Sorry I wasn't clear. I'm wondering if machine learning is used to train the computer to make trades.

jacob
Site Admin
Posts: 8962
Joined: Fri Jun 28, 2013 8:38 pm
Location: USA, Zone 5b, Koppen Dfa, Elev. 620ft, Walkscore 73
Contact:

Re: Bias in computer algorithms

Post by jacob » Sat Apr 15, 2017 7:47 am

Absolutely.

Jean
Posts: 246
Joined: Fri Dec 13, 2013 8:49 am

Re: Bias in computer algorithms

Post by Jean » Sat Apr 15, 2017 8:18 am

The king was always naked.

User avatar
FBeyer
Posts: 655
Joined: Tue Oct 27, 2015 3:25 am

Re: Bias in computer algorithms

Post by FBeyer » Sat Apr 15, 2017 11:14 am

Computers cannot perform magical things if the humans programming them cant either.
Training set biases are about as old as any kind of predictive algorithm.

Weve had this talk before, but last time we were talking about how the Meyers Briggs will get things wrong and how extroverts tend to score all over the scale whereas introverts tend towards more homogeneous results. Its not much different. Give input receive output based on what the computer was taught to begin with.

User avatar
BRUTE
Posts: 2260
Joined: Sat Dec 26, 2015 5:20 pm

Re: Bias in computer algorithms

Post by BRUTE » Sat Apr 15, 2017 11:44 am

garbage in garbage out

enigmaT120
Posts: 883
Joined: Thu Feb 12, 2015 2:14 pm
Location: Falls City, OR

Re: Bias in computer algorithms

Post by enigmaT120 » Mon Apr 17, 2017 5:33 pm

I can't believe I haven't named a cat Gigo.

Post Reply