Bias in computer algorithms
-
- Posts: 5406
- Joined: Wed Jul 28, 2010 3:28 am
- Location: Wettest corner of Orygun
Bias in computer algorithms
A little bit creepy... I'm glad people are looking into it: http://www.bbc.com/news/technology-39533308
-
- Site Admin
- Posts: 16122
- Joined: Fri Jun 28, 2013 8:38 pm
- Location: USA, Zone 5b, Koppen Dfa, Elev. 620ft, Walkscore 77
- Contact:
Re: Bias in computer algorithms
This is one of those unintended facepalm issues ... It's been around for a while.
If you include a race variable, pattern algo's will pick up any inherent aspects of race in the world (rest of the data points). Any result might violate a law(*). If you deliberately exclude race, the algo might still pick up race as a latent factor through its connection to other variables. If we start tweaking the algo so the the results are in accordance with the law; according to some other statistical results (statistics is really only a very crude algo that doesn't require too much sophistication to appreciate or conversely algos can be seen as very advanced statistics); or according to desired outcomes (e.g. affirmative action), you're deliberately asking to load the algo using race as an argument (tuning parameter) and this would in some sense be racist in itself (maybe a kind of "good racism" if that concept makes sense?!?).
(*) Not violating redlining laws is a notoriously difficult problem when using algos in financial services. https://en.wikipedia.org/wiki/Redlining Even if race is NOT an explanatory variable, the continued existence of segregated neighborhoods will mathematically attribute whatever a given area has in common to race because that's what most people would have in common [in that area] while the rest of the signals are much more subtle. One way around this is to subtract the average datapoint (here race) and look at the differences .. and then add the average back to the result. There are more refined ways. Kalman filtering uses a predictive physical model instead of just the average observation as a reference point.
This incidentally is also a problem with other aspects of pattern recognition. The reason is that "algorithmic intelligence" is NEQ "human intelligence". Computers don't give one iota about perceiving and thinking about the world the way that humans do. And it's very hard to make them do so. If it was easy, we'd have AI already.
The problem is not that algos are biased. The problem is that algos pick up whatever bias humans already demonstrate. The complaint here is that algos reveal that bias and humans would like algos to ignore it.
If you include a race variable, pattern algo's will pick up any inherent aspects of race in the world (rest of the data points). Any result might violate a law(*). If you deliberately exclude race, the algo might still pick up race as a latent factor through its connection to other variables. If we start tweaking the algo so the the results are in accordance with the law; according to some other statistical results (statistics is really only a very crude algo that doesn't require too much sophistication to appreciate or conversely algos can be seen as very advanced statistics); or according to desired outcomes (e.g. affirmative action), you're deliberately asking to load the algo using race as an argument (tuning parameter) and this would in some sense be racist in itself (maybe a kind of "good racism" if that concept makes sense?!?).
(*) Not violating redlining laws is a notoriously difficult problem when using algos in financial services. https://en.wikipedia.org/wiki/Redlining Even if race is NOT an explanatory variable, the continued existence of segregated neighborhoods will mathematically attribute whatever a given area has in common to race because that's what most people would have in common [in that area] while the rest of the signals are much more subtle. One way around this is to subtract the average datapoint (here race) and look at the differences .. and then add the average back to the result. There are more refined ways. Kalman filtering uses a predictive physical model instead of just the average observation as a reference point.
This incidentally is also a problem with other aspects of pattern recognition. The reason is that "algorithmic intelligence" is NEQ "human intelligence". Computers don't give one iota about perceiving and thinking about the world the way that humans do. And it's very hard to make them do so. If it was easy, we'd have AI already.
The problem is not that algos are biased. The problem is that algos pick up whatever bias humans already demonstrate. The complaint here is that algos reveal that bias and humans would like algos to ignore it.
Re: Bias in computer algorithms
it's metaphysically impossible to be free of bias, for humans or computer algorithms. everything is biased.
-
- Posts: 1950
- Joined: Tue Nov 12, 2013 10:23 pm
Re: Bias in computer algorithms
Jacob covered the issue well. In my opinion, the simplified way to think about it is if you teach the computer what a hand looks like using pictures that are 75% white, when you later ask for a sample of pictures that represent hands you get pictures of white hands because that was most common. The algorithm doesn't know you want variety of color.
I'm guessing the issue is that the algorithm selects one representative pic at a time. Since the sample it was trained on was mostly white hands it returns a pic of white hands each time it runs.
This used to be the case with Google results too. the user would search for "cats" and it would return 10 links to very similar popular cat pages. Now Google returns one link to the cat Wikipedia page, a video of a cat, a pic of a cat, a map to the nearest animal shelter, a link to the cats musical, etc. They may need to build in another layer that adds diversity to the image results page, or train with an intentionally diverse set of images instead of the set that is available online, assuming that is the source they use.
Jacob, if you can say, did this sort of thing get used much for trading?
I'm guessing the issue is that the algorithm selects one representative pic at a time. Since the sample it was trained on was mostly white hands it returns a pic of white hands each time it runs.
This used to be the case with Google results too. the user would search for "cats" and it would return 10 links to very similar popular cat pages. Now Google returns one link to the cat Wikipedia page, a video of a cat, a pic of a cat, a map to the nearest animal shelter, a link to the cats musical, etc. They may need to build in another layer that adds diversity to the image results page, or train with an intentionally diverse set of images instead of the set that is available online, assuming that is the source they use.
Jacob, if you can say, did this sort of thing get used much for trading?
-
- Site Admin
- Posts: 16122
- Joined: Fri Jun 28, 2013 8:38 pm
- Location: USA, Zone 5b, Koppen Dfa, Elev. 620ft, Walkscore 77
- Contact:
Re: Bias in computer algorithms
What sort of thing? Algorithms? Training on historic datasets? Training on selected datasets?
-
- Posts: 1950
- Joined: Tue Nov 12, 2013 10:23 pm
Re: Bias in computer algorithms
Sorry I wasn't clear. I'm wondering if machine learning is used to train the computer to make trades.
-
- Site Admin
- Posts: 16122
- Joined: Fri Jun 28, 2013 8:38 pm
- Location: USA, Zone 5b, Koppen Dfa, Elev. 620ft, Walkscore 77
- Contact:
Re: Bias in computer algorithms
Absolutely.
Re: Bias in computer algorithms
The king was always naked.
Re: Bias in computer algorithms
Computers cannot perform magical things if the humans programming them cant either.
Training set biases are about as old as any kind of predictive algorithm.
Weve had this talk before, but last time we were talking about how the Meyers Briggs will get things wrong and how extroverts tend to score all over the scale whereas introverts tend towards more homogeneous results. Its not much different. Give input receive output based on what the computer was taught to begin with.
Training set biases are about as old as any kind of predictive algorithm.
Weve had this talk before, but last time we were talking about how the Meyers Briggs will get things wrong and how extroverts tend to score all over the scale whereas introverts tend towards more homogeneous results. Its not much different. Give input receive output based on what the computer was taught to begin with.
Re: Bias in computer algorithms
garbage in garbage out
-
- Posts: 1240
- Joined: Thu Feb 12, 2015 2:14 pm
- Location: Falls City, OR
Re: Bias in computer algorithms
I can't believe I haven't named a cat Gigo.