Beyond the biases perpetuated by individuals, one of the significant drivers of AI bias comes from systemic biases that are embedded across several societal institutions. For instance, the history of the United States housing industry is rife with racist policies such as redlining that barred African Americans from homeownership in certain neighborhoods . Even though these policies have been outlawed for over fifty years, we can still observe their ramifications. African American homeownership is still significantly lower and banks are still much more likely to mortgage loans to Caucasian borrowers in certain regions. AI-based lending algorithms trained on historical housing data can learn the past racist policies and accelerate housing discrimination. Without any kind of intervention, AI algorithms can learn, codify and perpetuate biases long into the future all under the guise of objectivity.

If humans engage in biased judgement or decision-making that leads to unfair outcomes why does AI pose such a risk to humanity? AI technologies have the ability to automate biased decision-making at scale and this could lead to widespread algorithmic discrimination. Given the presence of implicit and systemic biases in many of our recorded data systems, algorithms that are trained on these datasets are often learning ‘standard’ scenarios and generalize behaviors that are not necessarily representative of the full population . So any individuals or sub-populations that deviate from the learned norms codified in the algorithm can experience negative impact and harm as a result.

This is further exacerbated by the lack of transparency and human autonomy over these AI systems. Some of the most notorious examples of AI causing societal harms occur when an AI system is deployed to automatically trigger decisions without any oversight or human control. For example, social media newsfeed recommendation algorithms have been shown to lead to massive political polarization and a rise in mis/disinformation globally based on automated content suggestions.

The role of AI is often obscured in these situations which is a serious concern of lack of consent of individuals influenced or impacted by AI. Even in cases where an individual is aware of the role of the algorithm in the prediction and wants or needs to contest the decision, it may be difficult or even impossible to explain the reasoning behind the decision - due to the complex and even black-box nature of many AI algorithms.

How can we go about addressing these very serious concerns? While it is impossible to claim we can remove all forms of bias, we can work to improve AI systems to ensure they are more equitable and inclusive. The simplest way to get started is by asking the right questions. There are several AI Ethics checklist resources available but below are a few key questions to get started and to check in periodically throughout the development and lifecycle of AI.

1. What is the purpose and intended use of the AI system? Are there ways that it can be misused or used unintentionally?

2. Who are the ‘interactors’ of the AI system? More, specifically, [a] Who are the primary intended end-user of the AI? Do they have adequate understanding and autonomous controls over the system? And [b] Who may be influenced or impacted by the presence of the AI system directly or indirectly? Are they / their needs well represented in the data and algorithm?

3. Are the AI ‘creators’ (i.e. data scientists, machine learning engineers, supporting operations, business sponsors) aware of ethical AI issues? And [a] Are they building in proper data and algorithm inspections during the development and monitoring of AI systems? [b] Are the developer teams diverse and support a culture of responsible AI?

There are a few steps to add to the AI algorithm development process to identify and mitigate bias. In the exploratory analysis, AI practitioners should analyze how well represented the data population is when compared to the target population of the AI solution. They should also analyze the target outcome across key sub-populations based on demographics, protected characteristics, or customer segments. Upon training the AI model, it should be evaluated for bias prior to deployment. Bias evaluations should include model performance disparities checks, model explanation assessments, and model outcome or endpoints reviews delineated by sub-populations. If any bias is found, mitigation methods like matching algorithms, sampling techniques, or debiasing algorithms should be applied.

To learn more about Ethical AI we should continue to review additional materials, particularly from marginalized communities who have already documented their experiences with AI and algorithmic discrimination. We should all develop a culture and governance to support responsible AI development.