Why it’s so damn hard to make AI fair and unbiased

Sigal Samuel

Vox.com

Originally posted 19 APR 2022

Here is an excerpt:

So what do big players in the tech space mean, really, when they say they care about making AI that’s fair and unbiased? Major organizations like Google, Microsoft, even the Department of Defense periodically release value statements signaling their commitment to these goals. But they tend to elide a fundamental reality: Even AI developers with the best intentions may face inherent trade-offs, where maximizing one type of fairness necessarily means sacrificing another.

The public can’t afford to ignore that conundrum. It’s a trap door beneath the technologies that are shaping our everyday lives, from lending algorithms to facial recognition. And there’s currently a policy vacuum when it comes to how companies should handle issues around fairness and bias.

“There are industries that are held accountable,” such as the pharmaceutical industry, said Timnit Gebru, a leading AI ethics researcher who was reportedly pushed out of Google in 2020 and who has since started a new institute for AI research. “Before you go to market, you have to prove to us that you don’t do X, Y, Z. There’s no such thing for these [tech] companies. So they can just put it out there.”

That makes it all the more important to understand — and potentially regulate — the algorithms that affect our lives. So let’s walk through three real-world examples to illustrate why fairness trade-offs arise, and then explore some possible solutions.

How would you decide who should get a loan?

Here’s another thought experiment. Let’s say you’re a bank officer, and part of your job is to give out loans. You use an algorithm to help you figure out whom you should loan money to, based on a predictive model — chiefly taking into account their FICO credit score — about how likely they are to repay. Most people with a FICO score above 600 get a loan; most of those below that score don’t.

One type of fairness, termed procedural fairness, would hold that an algorithm is fair if the procedure it uses to make decisions is fair. That means it would judge all applicants based on the same relevant facts, like their payment history; given the same set of facts, everyone will get the same treatment regardless of individual traits like race. By that measure, your algorithm is doing just fine.

But let’s say members of one racial group are statistically much more likely to have a FICO score above 600 and members of another are much less likely — a disparity that can have its roots in historical and policy inequities like redlining that your algorithm does nothing to take into account.

Another conception of fairness, known as distributive fairness, says that an algorithm is fair if it leads to fair outcomes. By this measure, your algorithm is failing, because its recommendations have a disparate impact on one racial group versus another.

The info is here.

America Runs on ‘Dirty Work’ and Moral Inequality

Eyal Press

The New York Times

Originally posted 13 Aug 21

Here is an excerpt:

“Dirty work” can refer to any unpleasant job, but among social scientists, the term has a more pointed meaning. In 1962, Everett Hughes, an American sociologist, published an essay titled “Good People and Dirty Work” that drew on conversations he’d had in postwar Germany about the mass atrocities of the Nazi era. Mr. Hughes argued that the persecution of Jews proceeded with the unspoken assent of many supposedly enlightened Germans, who refrained from asking too many questions because, on some level, they were not entirely displeased.

This was the nature of dirty work as Mr. Hughes conceived of it: unethical activity that was delegated to certain agents and then disavowed by society, even though the perpetrators had an “unconscious mandate” from their fellow citizens. As extreme as the Nazi example was, this dynamic existed in every society, Mr. Hughes wrote, enabling respectable citizens to distance themselves from the morally troubling things being done in their name. The dirty workers were not rogue actors but “agents” of “good people” who passively stood by.

Contemporary America runs on dirty work. Some of the people who do this work are our agents by virtue of the fact that they perform public functions, such as running the world’s largest penal system. Others qualify as such by catering to our consumption habits — the food we eat, the fossil fuels we burn, which are drilled and fracked by dirty workers in places like the Gulf of Mexico. The high-tech gadgets in our pockets rely on yet another form of dirty work — the mining of cobalt — that has been outsourced to workers in Africa and to foreign subcontractors that often brutally exploit them.

Like the essential jobs performed by grocery clerks and other low-wage workers during the Covid-19 pandemic, this work sustains our lifestyles and undergirds the prevailing social order, but privileged people are generally spared from having to think about it. One reason is that the dirty work occurs far away from them, in isolated institutions — prisons, slaughterhouses — that are closed to the public. Another reason is that the privileged rarely have to do it. Although there is no shortage of it to go around, dirty work in America is not randomly distributed. 

The info is here.

Shaking Things Up: Unintended Consequences of Firm Acquisitions on Racial and Gender Inequality

Letian Zhang

Harvard Business School

Originally published 23 Jan20

Abstract

This paper develops a theory of how disruptive events shape organizational inequality.  Despite various organizational efforts, racial and gender inequality in the workplace remains high. I theorize that because the persistence of such inequality is reinforced by organizational structures and practices, disruptive events that shake up old hierarchies and break down routines and culture should give racial minority and women workers more opportunities to advance. To examine this theory, I explore a critical but seldom analyzed organizational event in the inequality literature - mergers and acquisitions. I propose that post-acquisition restructuring could offer an opportunity for firms to advance diversity initiatives and to objectively re-evaluate workers. Using a difference-in-differences design on a nationally representative sample covering 37,343 acquisitions from 1971 to 2015, I find that although acquisitions lead to occupational reconfiguration that favors higher-skilled workers, they also reduce racial and gender inequality. In particular, I find improved managerial representation of racial minorities and women and reduced racial and gender segregation in the acquired workplace. This post-acquisition effect is stronger when (a) the acquiring firm values race and gender equality more and (b) the acquired workplace had higher racial and gender inequality.  These findings suggest that disruptive events could produce an unintended consequence of increasing racial and gender equality in the workplace.

Managerial Implications

From a managerial perspective, disruptive events offer an opportunity to advance diversity or equality-related goals that might be difficult to pursue during normal times.  As my analyses show, acquisition amplifies the race and gender differences between those acquiring firms that value diversity and those that do not. For managers concerned about race and gender issues, acquisitions and other disruptive events might serve as suitable moments to improve race and gender gaps effectively and at a relatively lower cost. Thus, despite the disruption and uncertainty during these periods, managers should see disruptive events as prime opportunities to make positive changes.

A pdf can be downloaded here.

Healthcare portraiture and unconscious bias

Karthik Sivashanker, Kathryn Rexrode, and others
BMJ 2019;365:l1668
Published April 12, 2019
https://doi.org/10.1136/bmj.l1668

Here is an excerpt:

Conveying the right message

In this regard, healthcare organisations have opportunities to instil a feeling of belonging and comfort for all their employees and patients. A simple but critical step is to examine the effect that their use of all imagery, as exemplified by portraits, has on their constituents. Are these portraits sufficiently conveying a message of social justice and equity? Do they highlight the achievement (as with a picture of a petri dish), or the person (a picture of Alexander Fleming without sufficient acknowledgment of his contributions)? Further still, do these images reveal the values of the organisation or its biases?

At our institution in Boston there was no question that the leaders depicted had made meaningful contributions to our hospital and healthcare. After soliciting feedback through listening sessions, open forums, and inbox feedback from our art committee, employees, clinicians, and students, however, our institution agreed to hang these portraits in their respective departments. This decision aimed to balance a commitment to equity with an intent to honourably display these portraits, which have inspired generations of physicians and scientists to be their best. It also led our social justice and equity committee to tackle problems like unconscious bias and diversity in hiring. In doing so, we are acknowledging the close interplay of symbolism and policy making in perpetuating racial and sex inequities, and the importance of tackling both together.

The info is here.