results in more recommendations for white people to receive care. This is a major issue because black patients may require care but not receive it due to the model’s biases.
WHAT COULD GO WRONG? Unfortunately, ML will inherently misunderstand and mislabel members of the LGBTQ+ community, racial minorities, and disabled people. Let’s start by examining how the LGBTQ+ community is being hurt. As cited in the article, “Why Artificial Intelligence Is Set Up to Fail LGBTQ People,” anthropologist Mary L. Gray states that LGBTQ+ people are “constantly remaking” their community. For algorithms that are built upon previous observed data, this raises concerns. Individuals who identify as LGBTQ+ will go through a life-long journey with their identity, and as they evolve and change, algorithms will not be able to keep up. They will continue to calculate data points with the least amount of distance to a new data point to make ‘logical’ predictions. But when data points represent people, with unique and extensive identities, this does not lead to accurate predictions. Instead, it results in misclassification that can be harmful to an individual’s security in their identity. Take Automatic Gender Recognition, for example, that is built on training data of mostly cisgendered males and females. To classify a new person’s gender, the algorithm compares their physical facial features to those in the training data to make a logical classification. Many transgender individuals are being harmed by these superficial assumptions. When it comes to racial minorities, we are seeing significant biases especially in the medical field. Algorithms that predict which patients need care have been unintentionally discriminating against black people. In their study entitled “Dissecting racial bias in an algorithm used to manage the health of populations”, Obermeyer et al. examined a commonly used medical ML algorithm and found that black patients needed to be significantly sicker than white patients to be recommended for the same type of care. The reason that we see such biases in this algorithm is because the model was built on “past data on health care spending, which reflects a history in which Black patients had less to spend on their health care compared to white patients, due to longstanding wealth and income disparities” (Grant). Since the data has been strongly impacted by such racial issues, the model reflects this and
SOLUTIONS The main solution that many people suggest is to build ML models on data that is unbiased. For the example of racist models in the medical field, this would look like building a model on data that does not strictly contain healthcare spending history for white and black Americans, but rather includes other factors that don’t misrepresent black Americans. The reason is that ML models try to make predictions based on data points that are most like the new, unlabeled data point. If a model is built on data that is not accurate or representative of a population, we see these kinds of problems. For issues involving the labeling of LGBTQ+ individuals, it is likely not enough to build a model on unbiased data. Groups that are constantly evolving will continue to be hurt by algorithms that are based on previously seen data. While these predictions may be accurate at times, they may not hold up in the future, so it is likely that AI will never be successful in trying to classify LGBTQ+ individuals. Overall, it is difficult to determine the exact solution to the issues raised by ML and AI. While there are countless benefits of these phenomena that we are experiencing in our daily lives, we must be aware of the social issues that come with it, as well as why they are there in the first place. From there, we can work towards rectifying these problems and preventing them from happening again in the future.
Sources Grant, Crystal. “Algorithms Are Making Decisions About Health Care, Which May Only Worsen Medical Racism.” American Civil Liberties Union, 24 Feb. 2023, www.aclu.org/news/privacy-technology/algorithms-in-health-care-may-worsen-medicalracism. Keyes, Os. The Misgendering Machines: Trans/HCI Implications of Automatic Gender Recognition. ironholds.org/resources/papers/agr_paper.pdf. Obermeyer, Ziad, et al. “Dissecting Racial Bias in an Algorithm Used to Manage the Health of Populations.” Science, 25 Oct. 2019, www.science.org/doi/10.1126/science.aax2342. Rangnekar, Priti. “Decoding the ‘Encoding’ of Ableism in Technology and Artificial Intelligence.” Medium, 27 July 2021, medium.com/@scienceconnectorg/decoding-theencoding-of-ableism-in-technology-and-artificial-intelligence-a4f33bcccc85. Wareham, Jamie. “Why Artificial Intelligence Is Set up to Fail LGBTQ People.” Forbes, 26 Mar. 2021, www.forbes.com/sites/jamiewareham/2021/03/21/why-artificial-intelligencewill-always-fail-lgbtq-people/?sh=3e81a6d2301e.
48