JPMorgan Chase Patents Take On Unconscious Bias Using AI

Two recent patents from JPMorgan Chase may signal the company’s desire to take on bias with AI. 

To start, the financial institution is seeking to patent a system for the “automated masking of targeted information in resumes.” This tech essentially blocks out certain information on a resume that may lead to a hiring team or reviewer acting on implicit biases. 

Resumes often include “personal information about a particular candidate that may be used by a reviewer to unfairly bias the reviewer’s opinion for or against the particular candidate—whether consciously or subconsciously,” JPMorgan said. 

Using AI, JPMorgan’s system would generate a black box that redacts certain information on a resume before sending it to the reviewer. For example, some of the information that may be redacted is a person’s name, university, email address, residential address and age, only leaving information about skills and past experience for the reviewer to focus on. 

JPMorgan may also be looking at ways to negate bias internally, too: The company wants to patent tech for “identifying and removing bias from communications.” This essentially aims to monitor communications, such as emails, texts, documents or audio transcriptions, to find and point out words or phrases that it deems biased. 

Using a machine learning model, this tech will analyze a passage of text to pick up on patterns of bias, and suggest how to change them. This could help eliminate unconscious bias by giving text another set of proverbial eyes. “Unconscious biases are usually exhibited towards factors like class, gender, race, ethnicity, and sexual orientation,” JPMorgan said in the filing. “These biases are more pervasive and difficult to control than deliberate discrimination.” 

JPMorgan Chase is all in on AI. The company has sought patents for AI-based financial planners, machine learning-based due diligence and no-code model building technology. The company has even sought to automate HR with a patent application for a system to use machine learning for recruiting. 

While these patents aim to use AI-powered tech to limit humans’ tendencies to have implicit biases, AI itself often comes with its own biases. Because AI models are only as good as the data that they’re trained on, they can often be skewed by biases in the data. 

Several tech firms have attempted to solve this issue in any context where it may show up, whether it be image tools, AI lending, or the training data itself. However, when left unsolved, these biases can impact the performance and output of AI models. 

For example, a recent investigation from Bloomberg found that OpenAI’s ChatGPT 3.5 displayed preferences for certain races in questions about hiring. Job seekers are noticing biases as well, with 49% surveyed by the American Staffing Association believing that AI recruiting tools are more biased than their human counterparts. 

If JPMorgan implements these tools, the company may have to be careful about how it trains its models responsible for monitoring hiring and communications, or they could unintentionally create more bias than they’re stopping.