Bias in computer algorithms

Intended for constructive conversations. Exhibits of polarizing tribalism will be deleted.
Locked
George the original one
Posts: 5404
Joined: Wed Jul 28, 2010 3:28 am
Location: Wettest corner of Orygun

Bias in computer algorithms

Post by George the original one »

A little bit creepy... I'm glad people are looking into it: http://www.bbc.com/news/technology-39533308

jacob
Site Admin
Posts: 15907
Joined: Fri Jun 28, 2013 8:38 pm
Location: USA, Zone 5b, Koppen Dfa, Elev. 620ft, Walkscore 77
Contact:

Re: Bias in computer algorithms

Post by jacob »

This is one of those unintended facepalm issues ... It's been around for a while.

If you include a race variable, pattern algo's will pick up any inherent aspects of race in the world (rest of the data points). Any result might violate a law(*). If you deliberately exclude race, the algo might still pick up race as a latent factor through its connection to other variables. If we start tweaking the algo so the the results are in accordance with the law; according to some other statistical results (statistics is really only a very crude algo that doesn't require too much sophistication to appreciate or conversely algos can be seen as very advanced statistics); or according to desired outcomes (e.g. affirmative action), you're deliberately asking to load the algo using race as an argument (tuning parameter) and this would in some sense be racist in itself (maybe a kind of "good racism" if that concept makes sense?!?).

(*) Not violating redlining laws is a notoriously difficult problem when using algos in financial services. https://en.wikipedia.org/wiki/Redlining Even if race is NOT an explanatory variable, the continued existence of segregated neighborhoods will mathematically attribute whatever a given area has in common to race because that's what most people would have in common [in that area] while the rest of the signals are much more subtle. One way around this is to subtract the average datapoint (here race) and look at the differences .. and then add the average back to the result. There are more refined ways. Kalman filtering uses a predictive physical model instead of just the average observation as a reference point.

This incidentally is also a problem with other aspects of pattern recognition. The reason is that "algorithmic intelligence" is NEQ "human intelligence". Computers don't give one iota about perceiving and thinking about the world the way that humans do. And it's very hard to make them do so. If it was easy, we'd have AI already.

The problem is not that algos are biased. The problem is that algos pick up whatever bias humans already demonstrate. The complaint here is that algos reveal that bias and humans would like algos to ignore it.

BRUTE
Posts: 3797
Joined: Sat Dec 26, 2015 5:20 pm

Re: Bias in computer algorithms

Post by BRUTE »

it's metaphysically impossible to be free of bias, for humans or computer algorithms. everything is biased.

Gilberto de Piento
Posts: 1942
Joined: Tue Nov 12, 2013 10:23 pm

Re: Bias in computer algorithms

Post by Gilberto de Piento »

Jacob covered the issue well. In my opinion, the simplified way to think about it is if you teach the computer what a hand looks like using pictures that are 75% white, when you later ask for a sample of pictures that represent hands you get pictures of white hands because that was most common. The algorithm doesn't know you want variety of color.

I'm guessing the issue is that the algorithm selects one representative pic at a time. Since the sample it was trained on was mostly white hands it returns a pic of white hands each time it runs.

This used to be the case with Google results too. the user would search for "cats" and it would return 10 links to very similar popular cat pages. Now Google returns one link to the cat Wikipedia page, a video of a cat, a pic of a cat, a map to the nearest animal shelter, a link to the cats musical, etc. They may need to build in another layer that adds diversity to the image results page, or train with an intentionally diverse set of images instead of the set that is available online, assuming that is the source they use.

Jacob, if you can say, did this sort of thing get used much for trading?

jacob
Site Admin
Posts: 15907
Joined: Fri Jun 28, 2013 8:38 pm
Location: USA, Zone 5b, Koppen Dfa, Elev. 620ft, Walkscore 77
Contact:

Re: Bias in computer algorithms

Post by jacob »

What sort of thing? Algorithms? Training on historic datasets? Training on selected datasets?

Gilberto de Piento
Posts: 1942
Joined: Tue Nov 12, 2013 10:23 pm

Re: Bias in computer algorithms

Post by Gilberto de Piento »

Sorry I wasn't clear. I'm wondering if machine learning is used to train the computer to make trades.

jacob
Site Admin
Posts: 15907
Joined: Fri Jun 28, 2013 8:38 pm
Location: USA, Zone 5b, Koppen Dfa, Elev. 620ft, Walkscore 77
Contact:

Re: Bias in computer algorithms

Post by jacob »

Absolutely.

User avatar
Jean
Posts: 1890
Joined: Fri Dec 13, 2013 8:49 am
Location: Switzterland

Re: Bias in computer algorithms

Post by Jean »

The king was always naked.

FBeyer
Posts: 1069
Joined: Tue Oct 27, 2015 3:25 am

Re: Bias in computer algorithms

Post by FBeyer »

Computers cannot perform magical things if the humans programming them cant either.
Training set biases are about as old as any kind of predictive algorithm.

Weve had this talk before, but last time we were talking about how the Meyers Briggs will get things wrong and how extroverts tend to score all over the scale whereas introverts tend towards more homogeneous results. Its not much different. Give input receive output based on what the computer was taught to begin with.

BRUTE
Posts: 3797
Joined: Sat Dec 26, 2015 5:20 pm

Re: Bias in computer algorithms

Post by BRUTE »

garbage in garbage out

enigmaT120
Posts: 1240
Joined: Thu Feb 12, 2015 2:14 pm
Location: Falls City, OR

Re: Bias in computer algorithms

Post by enigmaT120 »

I can't believe I haven't named a cat Gigo.

Locked