Elizabeth Warren wants answers from federal officials about bias in algorithms

- For a long time now, lenders have given African American and Latino families fewer loans and at inferior terms than white borrowers in similar circumstances. One might think that the use of tech such as algorithms would be able to solve these problems.

Bias can be built into algorithms

One might think that doing away with human bias in decisions would end discrimination. Instead it turns out that many algorithms have built in biases. Democratic presidential hopeful Senator Elizabeth Warren (D-MA) notes that the use of algorithms can potentially lead to lower levels of discrimination but algorithms can actually have biases hardwired into them and so should be checked for such biases before being used.

The case of African-American and Latino borrowers

Financial technology companies charge African-American and Latino homeowner an overall six to nine basis points higher interest rates than white and Asian borrowers. Researchers estimate that this adds an additional $250 million to $500 million a year in additional mortgage interest payments for US black and Latino borrowers.

Senator Warren's letter

“

Warren, a Democrat from Massachusetts, is a member of the US Senate Committee on Banking, Housing and Urban Affairs. She sent a letter to Chairman Jerome Powell at the Board of Governors of the Federal Reserve System, Joseph Otting, Comptroller at the Office of the Comptroller of the Currency, Chair Jelena McWilliams at the Federal Deposit Insurance Corporation, and Kathy Kraninger, Director at the Consumer Financial Protection Bureau. The letter seeks information about what role banking agencies can play in making sure that lending algorithms serve US borrowers on a fair non-discriminatory fair basis.

The Warren letter, cosigned by Senator Doug Jones, an Alabama Democrat, reads: "Recent research highlights this tension, demonstrating both the opportunity of algorithmic underwriting's potential to reduce discrimination, while also emphasizing the technologies' current shortcomings... Researchers found that "algorithmic lending" appeared to reduce the likelihood that the loan applications of borrowers of color are rejected-thus reducing discrimination in lenders' "accept/reject" decision-making process. However, the same study also found that "FinTech lenders [using algorithms] charge Latinx/African American borrowers 6-9 basis points higher interest rates" than comparable borrowers - which is almost identical to the interest rate premium that minority borrowers face with traditional underwriting. In other words, the algorithms used by FinTech lenders are as discriminatory as loan officers.... Senator Warren has requested responses to her letter by June 24, 2019."

Warren is the latest Democratic presidential hopeful to demand answers about how algorithms negatively impact the lives of people of color. Last fall, a group of U.S. senators, including Warren and Senator Kamala Harris (D-CA) and Cory Booker (D-NJ), sent letters to federal agencies with questions about algorithmic bias and facial recognition software. t=_blank] recent article notes: "She is the latest Democratic presidential candidate to demand answers about how algorithms negatively impact the lives of people of color. Last fall, a group of U.S. senators, including Warren and Senator Kamala Harris (D-CA) and Cory Booker (D-NJ), sent letters to federal agencies with questions about algorithmic bias and facial recognition software."

Previously published in the Digital Journal

Google Aritificial Intelligence chief claims biased algorithms are a big danger

Social concerns are at the forefront of Artificial Intelligence conversations right now. John Giannandrea AI chief at Google is concerned that bias is being built in to many of the machine-learning algorithms by which the robot makes decisions.

At a recent Google conference on the relationships between AI systems and humans Giannandrea said: “The real safety question, if you want to call it that, is that if we give these systems biased data, they will be biased.”

This problem is likely to become more widespread as AI technology spreads to areas such as law, and medicine and as more people use the technology who do not have a deep understanding of the technical problems with the algorithm's being used. Indeed many who use the technology stress that it eliminates human bias, as is boasted in the following ad:

Two flavours of AI

In the early stages of the development of AI there were two main models. There was a logic, rules-based model and a competing biological model. The logic model would have developed AI in a manner that anyone who cared to could determine exactly how a robot or other device made decisions but the biological model took a quite different approach:

Instead of a programmer writing the commands to solve a problem, the program generates its own algorithm based on example data and a desired output. The machine-learning techniques that would later evolve into today’s most powerful AI systems followed the latter path: the machine essentially programs itself.

Tommi Jaakkola a professor at MIT who works on applications of machine learning said: “It is a problem that is already relevant, and it’s going to be much more relevant in the future. Whether it’s an investment decision, a medical decision, or maybe a military decision, you don’t want to just rely on a ‘black box’ method.”

Giannandrea elaborated on the problem: “It’s important that we be transparent about the training data that we are using, and are looking for hidden biases in it, otherwise we are building biased systems. If someone is trying to sell you a black box system for medical decision support, and you don’t know how it works or what data was used to train it, then I wouldn’t trust it.”

COMPAS

There have already been complaints of bias in AI algorithm programs. A system called COMPAS predicts defendant's likelihood of re-offending and is even used by some judges in determining whether an inmate is to be granted parole. Although the workings of COMPAS are kept secret an investigation by ProPublica has argued that there is evidence that the model used may involve bias against minorities.

The issue is complicated and Northpointe, the company that created COMPAS, has issued rebuttals to ProPublica's arguments and there has been a continuing back and forth debate that is discussed in detail in this article. The Wisconsin Supreme Court has upheld the use of COMPAS in sentencing.

The COMPAS program assigns defendants scores from 1 to 10 based on how likely they are to re-offend based on more than 100 factors none of which directly include race. The scores have been found to be highly predictive of re-offending. However, ProPublica questions whether they are fair.

Northpointe argued that the assessments were fair in that the scores meant the same regardless of race. However, ProPublica pointed out that among defendants who ultimately did not re-offend, blacks were more than twice as likely as whites to be classified as high risk, 42 percent versus 22 percent. Hence, even though these black defendants did not go on to commit a crime they were subjected to harsher treatment by the courts.

Complex intelligence

Another issue that is not being adequately faced is that the new deep learning techniques are so complex and opaque that they often cannot be understood. However, researchers are attempting to create ways that give them at least some approximation of how they work to engineers and end users.

Karrie Karahalios, a professor of computer science at the University of Illinois showed that even commonplace algorithms, such as those used by Facebook to filter the posts they see in their news feed are not usually understood by users and thus it is difficult to detect bias. The appended video discusses some of the controversy over Facebook filters. Facebook showed clear evidence of censoring Conservative media as shown on the appended video. Google has been accused of censorship by left wing sources in the ranking algorithm it uses for its search engine.