By Majd Alkawaas | Staff Writer
Imagine walking into a restaurant and being asked to get your own menu and prepare your own meals because of your gender, religion, color, or any other stereotypical standard. Recently, various disciplines have been recruiting machines powered by artificial intelligence to replace services previously offered by humans to increase the accuracy of deliverables, standardize the quality level among all consumers, and avoid any bias or discrimination that may result from human interaction. However, little do they know that behind that shiny surface of supposedly “neutral” steel, there lies a reactionary mindset that retains discriminatory ideas. In a previous article, we have discussed how machine learning works and how implicit biases in datasets get transferred to a machine. In this article, we are going to focus on a more specific issue, gender bias in AI, and its profound implications.
During the fall of 2017 professor Vicente Ordóñez of Virginia University noticed a pattern in the output of an image-recognition model he was working on. The model was associating the picture of a kitchen more often with a woman than a man. This behavior got him thinking whether or not it was possible that he or other scientists are projecting implicit biases onto the models they are building! To find out, Ordóñez decided to test his hypothesis with a group of other AI scientists, and, to their surprise, the results were eye-opening. For instance, they discovered that two of the most used and prominent datasets of research images, including one that was supported by Microsoft and Facebook, display a predictable gender bias in their depiction of activities such as cooking and sports. For example, images of shopping and washing are linked to women while coaching and shooting are linked to men. After inspecting the machine learning models that were trained on those datasets, Ordóñez concluded that these models did not just mirror those biases but also amplified them! For instance, if a dataset of images generally associated women with cooking, the machine learning (ML) models created even stronger associations, making it more likely to associate pictures of cooking with women.
Prior to Ordóñez, Amazon was working on an AI experimental hiring tool they built in 2014 to review job applications. A year later during 2015, the team noticed that this tool was ranking male candidates higher than female candidates for new jobs. Not to our surprise, this tool was designed based on Amazon hiring records for the past ten years, where the majority of employees were males, and it made an association between being a male and being qualified for the job. Fortunately, the tool was eventually terminated, and according to Amazon, “was never used by Amazon recruiters to evaluate candidates” (Reuters). There are also instances of Google Translate committing such biases while translating documents from Spanish to English and referring to female nouns with male pronouns.
Having such biases embedded in the models powered by renowned companies like Google and Amazon affects the way these companies store and process their data and the quality of services provided to the user. For example, if a student is writing a paper about working mothers, the results of their research is skewed based on the biases that have been instilled into the search engine they used. However, in an upscaled scenario where it is no longer an academic assignment but rather a scholarly research conducted to improve policymaking, the implications of such biases are catastrophic. Moreover, imagine if Amazon was actually relying on the aforementioned tool to review job candidates and was excluding or overlooking all female candidates! Unfortunately, the cycle of biases does not end at these two companies or services, for the situation has a potential of becoming exceedingly disastrous as AI is being incorporated in far more complicated and critical services. COMPAS, for example, is a system that helps in the decision making process of granting parole release to prisoners!
Currently, multiple research teams are examining ways of building more neutral AI models and are inspecting existing datasets and models for implicitly embedded biases. For example, the Gender Shades research project is working on evaluating the accuracy of AI-powered gender classification tools of Microsoft IBM and other existing tools. Additionally, some companies are taking the initiative of evaluating their own tools. For example Microsoft has established an internal ethics committee dedicated to keeping AI in the company’s products in line. It is evident from daily technological breakthroughs that harnessing the power of AI has been the dream of countless researchers; however, it seems that there remains a long road ahead of us to build neutral, trustworthy, accountable, and dependable AI models to assist human needs.