Welcome to the Nexus of Ethics, Psychology, Morality, Philosophy and Health Care

Welcome to the nexus of ethics, psychology, morality, technology, health care, and philosophy

Tuesday, May 17, 2022

Why it’s so damn hard to make AI fair and unbiased

Sigal Samuel
Originally posted 19 APR 2022

Here is an excerpt:

So what do big players in the tech space mean, really, when they say they care about making AI that’s fair and unbiased? Major organizations like Google, Microsoft, even the Department of Defense periodically release value statements signaling their commitment to these goals. But they tend to elide a fundamental reality: Even AI developers with the best intentions may face inherent trade-offs, where maximizing one type of fairness necessarily means sacrificing another.

The public can’t afford to ignore that conundrum. It’s a trap door beneath the technologies that are shaping our everyday lives, from lending algorithms to facial recognition. And there’s currently a policy vacuum when it comes to how companies should handle issues around fairness and bias.

“There are industries that are held accountable,” such as the pharmaceutical industry, said Timnit Gebru, a leading AI ethics researcher who was reportedly pushed out of Google in 2020 and who has since started a new institute for AI research. “Before you go to market, you have to prove to us that you don’t do X, Y, Z. There’s no such thing for these [tech] companies. So they can just put it out there.”

That makes it all the more important to understand — and potentially regulate — the algorithms that affect our lives. So let’s walk through three real-world examples to illustrate why fairness trade-offs arise, and then explore some possible solutions.

How would you decide who should get a loan?

Here’s another thought experiment. Let’s say you’re a bank officer, and part of your job is to give out loans. You use an algorithm to help you figure out whom you should loan money to, based on a predictive model — chiefly taking into account their FICO credit score — about how likely they are to repay. Most people with a FICO score above 600 get a loan; most of those below that score don’t.

One type of fairness, termed procedural fairness, would hold that an algorithm is fair if the procedure it uses to make decisions is fair. That means it would judge all applicants based on the same relevant facts, like their payment history; given the same set of facts, everyone will get the same treatment regardless of individual traits like race. By that measure, your algorithm is doing just fine.

But let’s say members of one racial group are statistically much more likely to have a FICO score above 600 and members of another are much less likely — a disparity that can have its roots in historical and policy inequities like redlining that your algorithm does nothing to take into account.

Another conception of fairness, known as distributive fairness, says that an algorithm is fair if it leads to fair outcomes. By this measure, your algorithm is failing, because its recommendations have a disparate impact on one racial group versus another.