Foreword

This bibliography covers technological injustice, algorithmic bias, and related topics that explore the intersections of technology and racism, sexism, and other forms of oppression and hate. Inside it you will find brief topic introductions, case studies, helpful videos, suggested readings, and resources from the Iowa State University Library’s collections.

The bibliography was made to accompany a book exhibit on the same topic, on display in Parks Library during January, 2020. Iowa State librarian Erin Thomas compiled the materials and wrote the section introductions by with help from other Iowa State librarians. Iowa State librarian Megan O’Donnell assisted with transitioning and updating the content in March, 2024.

Introduction

Technology is pervasive in modern society, and so it touches a huge variety of topics. In general, topics are related to technological justice if they involve people (their lifestyle, health, wellness, livelihood, etc.) and they are a direct result of technological developments (artificial intelligence, big data analytics, social media, web3, etc.) amplifying existing biases or being applied inequitably. Examples abound, but this introductory guide will focus on the following broad topic areas:

• Machine & algorithmic bias, and how allegedly neutral technology amplifies bias in society

• How surveillance technology is employed to maintain inequitable power and social structures

• Ways social networking technology is used to further hate and spread disinformation

Resources

Articles 📃

• Bornstein, Aaron M. “Are Algorithms Building the New Infrastructure of Racism?” Nautilus (December 5, 2017): https://nautil.us/are-algorithms-building-the-new-infrastructure-of-racism-236911/

• Metz, Cade. “We Teach A.I. Systems Everything, Including Our Biases.” New York Times (November 11, 2019): https://www.nytimes.com/2019/11/11/technology/artificial-intelligence-bias.html

• O’Shea, Lizzie. “Technology Is as Biased as Its Makers.” Longreads (May 14, 2019): https://longreads.com/2019/05/14/technology-is-as-biased-as-its-makers/

• Shane, Janelle. You Look Like a Thing and I Love You: How Artificial Intelligence Works and Why It’s Making the World a Weirder Place. New York: Voracious, 2019.

Books 📙

• Benjamin, Ruha. Race After Technology: Abolitionist Tools for the New Jim Code. Cambridge, UK: Polity, 2019.

• Broussard, Meredith. Artificial Unintelligence: How Computers Misunderstand the World. Cambridge, MA: MIT Press, 2018.

• Fry, Hannah. Hello World: Being Human in the Age of Algorithms. New York: W. W. Norton & Company, 2018.

• Hosanagar, Kartik. A Human’s Guide to Machine Intelligence: How Algorithms are Shaping Our Lives and How We Can Stay in Control. New York: Viking, 2019.

• Noble, Sofia Umoja. Algorithms of Oppression: How Search Engines Reinforce Racism. New York: New York University Press, 2018.

• O’Neal, Cathy. Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy. New York: Crown, 2016.

• Wachter-Boettcher, Sara. Technically Wrong: Sexist Apps, Biased Algorithms, and Other Threats of Toxic Tech. New York: W. W. Norton, 2017.

Machine and Algorithmic Bias

Much of our modern technology is built on the idea that by relying on data and artificial intelligence, we can easily avoid recreating and reinforcing historic biases and inequities. According to this logic, systems powered by AI and decisions based on data should be more equitable and just than those requiring human intervention. However, research increasingly indicates that this is not the case. Machine learning systems and AI can not only replicate bias, they can (and frequently do) exacerbate it.

Resources

Online 🌐

• A People's Guide to Finding Algorithmic Bias

  An introduction to algorithmic bias from the Center for Critical Race + Digital Studies.

• Algorithmic Justice League

  The Algorithmic Justice League’s mission is to raise awareness about the impacts of AI, equip advocates with empirical research, build the voice and choice of the most impacted communities, and galvanize researchers, policy makers, and industry practitioners to mitigate AI harms and biases.

• AI Fairness 360

  An extensible open source toolkit from IBM, intended to examine, report, and mitigate discrimination and bias in machine learning models. Includes lots of references for additional information.

• Data 4 Black Lives

  Data for Black Lives is a movement of activists, organizers, and mathematicians committed to the mission of using data science to create concrete and measurable change in the lives of Black people.

Books 📙

• Benjamin, Ruha. Race After Technology: Abolitionist Tools for the New Jim Code. Cambridge, UK: Polity, 2019.

• Eubanks, Virginia. Automating Inequality: How High-Tech Tools Profile, Police, and Punish the Poor. New York: St. Martin’s Press, 2018.

• Katz, Yarden. Artificial Whiteness: Politics and Ideology in Artificial Intelligence. New York: Columbia University Press, 2020.

• Noble, Sofia Umoja. Algorithms of Oppression: How Search Engines Reinforce Racism. New York: New York University Press, 2018.

• Smith, Robert Elliott. Rage Inside the Machine: The Prejudice of Algorithms, and How to Stop the Internet Making Bigots of Us All. London: Bloomsbury, 2019.

Justice System

The United States criminal justice system increasingly relies on technology to assess criminal cases, recommending sentences and predicting the risk that a particular defendant will be rearrested later. While removing the bias of a human judge may seem like a good thing on the surface, these tools are often black boxes, utilizing proprietary algorithms (and training data) to arrive at their recommendations. They operate without transparency and frequently serve to reinforce societal biases in insidious ways—under the veneer of a "neutral" technology.

Articles 📃

• Angwin, Julia, Jeff Larson, Surya Mattu, and Lauren Kirchner. “Machine Bias.” ProPublica (May 23, 2016): https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing

• Angwin, Jeff Larson, Laruen Kirchner, and Surya Mattu. “What Algorithmic Injustice Looks Like in Real Life.” ProPublica (May 25, 2016): https://www.propublica.org/article/what-algorithmic-injustice-looks-like-in-real-life

• Courtland, Rache. Bias Detectives: The Researchers Striving to Make Algorithms Fair. Nature 558 (June 20, 2018): 357–360. https://doi.org/10.1038/d41586-018-05469-3

  Books 📙

• Benjamin, Ruha. Captivating Technology: Race, Carceral Technoscience, and Liberatory Imagination in Everyday Life. Durham, NC: Duke University Press, 2019.

• Forret, Katherine Bolan. When Machines Can Be Judge, Jury, and Executioner. New Jersey: World Scientific, 2021.

• Jefferson, Brian Jordan. Digitize and Punish: Racial Criminalization in the Digital Age. Minneapolis, MN: University of Minnesota Press, 2020.

• Whitlock, Kay and Nancy A. Heitzeg. Carceral Con: The Deceptive Terrain of Criminal Justice Reform. Oakland, CA: University of California Press, 2021.

Face Recognition Fails

Face analysis and recognition software is used in a growing number of applications, from confirming your identity to unlock your phone to policing. However, the data used to train these applications is often biased, resulting in higher error rates for women and people of color (and women of color in particular). Additionally, these automated identification systems are not always rolled out fairly—especially when used by law enforcement, communities of color typically see far greater levels of surveillance.

Articles 📃

• Birhane, Abeba. “The Unseen Black Faces of AI Algorithms.” Nature 610 (2022): 451–452. https://doi.org/10.1038/d41586-022-03050-7

• Buolimwini, Joy and Timnit Gebru. “Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification.” Proceedings of Machine Learning Research 81 (2018): 77–91. https://proceedings.mlr.press/v81/buolamwini18a.html

• Johnson, Khari. “How Wrongful Arrests Based on AI Derailed 3 Men's Lives.” Wired (Marh 7, 2022). https://www.wired.com/story/wrongful-arrests-ai-derailed-3-mens-lives/

• Najibi, Alex. “Racial Discrimination in Face Recognition Technology.” Science in the News (October 24, 2020): https://sitn.hms.harvard.edu/flash/2020/racial-discrimination-in-face-recognition-technology/

Books 📙

• Broussard, Meredith. More than a Glitch: Confronting Race, Gender, and Ability Bias in Tech. Cambridge, MA: MIT Press, 2023.

• Chun, Wendy Hui Kyong. Discriminating Data: Correlation, Neighborhoods, and the New Politics of Recognition. Cambridge, MA: MIT Press, 2021.

• Yeung, Doublas, Rebecca Balebako, Carlos Ignacio Gutierrez Gaviria, and Michael Chaykowsky. Face Recognition Technologies : Designing Systems That Protect Privacy and Prevent Bias. Santa Monica: RAND Corporation, 2020.

Surveillance

Surveillance technology, from automated "shot spotter" systems to doorbell cameras, is everywhere. But its effects are often inequitable, sometimes in ways that may not seem obvious at first. Marginalized individuals, such as people of color and transgender or non-binary people, are often much more negatively affected by pervasive surveillance technology than white and cisgender people.

Resources

Articles 📃

• Caraballo, Alejandra. “Remote Learning Accidentally Introduced a New Danger for LGBTQ Students.” Slate (February 24, 2022): https://slate.com/technology/2022/02/remote-learning-danger-lgbtq-students.html

• Fussell, Sidney. “How Surveillance Has Always Reinforced Racism.” Wired (June 19, 2020): https://www.wired.com/story/how-surveillance-reinforced-racism/

• Cockford, Kade. “How is Face Recognition Surveillance Technology Racist?” ACLU (June 16, 2020): https://www.aclu.org/news/privacy-technology/how-is-face-recognition-surveillance-technology-racist

Books 📙

• Beauchamp, Toby. Going Stealth: Transgender Politics and U.S. Surveillance Practices. Durham, NC: Duke University Press, 2018.

• Fasman, Jon. We See It All: Liberty and Justice in an Age of Perpetual Surveillance. New York: PublicAffairs, 2021.

• Flynn, Susan and Antonia Mackay. Surveillance, Race, Culture. Cham, Switzerland: Palgrave Macmillan, 2018.

Big Data Policing

Law enforcement is increasingly reliant on technology, including big data, for policing. Sometimes known as "predictive policing", this approach relies on powerful technology to find patterns among crimes that have already occurred in order to create predictions of future crime. This technology relies on flawed data rooted in historic and systemic inequities, positioning it to reinforce these biases in ways that are more harmful for marginalized communities. This can lead to overpolicing of neighborhoods in which marginalized people live, inequitable arrest patterns, and more.

Articles 📃

• Moody, Josh. “Law Students Protest Research Database Contracts With ICE.” Inside Higher Ed (December 5, 2021): https://www.insidehighered.com/news/2021/12/06/law-students-protest-lexisnexis-westlaw-contracts-ice

• Peeples, Lynne. “What the Data Say about Police Brutality and Racial Bias—and Which Reforms Might Work.” Nature 583 (2020): 22–24. https://doi.org/10.1038/d41586-020-01846-z

Books 📙

• Brayne, Sarah. Predict and Surveil: Data, Discretion, and the Future of Policing. New York: Oxford University Press, 2021.

• Egbert, Simon and Matthias Leese. Criminal Futures: Predictive Policing and Everyday Police Work. London: Routledge, 2020. https://doi.org/10.4324/9780429328732

• Ferguson, Andrew G. The Rise of Big Data Policing: Surveillance, Race, and the Future of Law Enforcement. New York: New York University Press, 2017.

• Friedman, Barry. Unwarranted: Policing without Permission. New York: Farrar, Straus and Giroux, 2017.

Targeted Advertising

While advertising has always focused on a target audience, modern forms of online advertising use a complex bidding system that relies on access to huge amounts of user data in order to select ads that are highly targeted to specific people. While this will in theory lead to highly effective ads informed by previous purchasing patterns and other, more personal information (including gender, sexuality, race, ethnicity), it can also amplify existing biases and discriminate against consumers based on their data.

Articles 📃

• Edelman, Gilad. “Why Don’t We Just Ban Targeted Advertising?” Wired (March 22, 2020): https://www.wired.com/story/why-dont-we-just-ban-targeted-advertising/

• Karlis, Nicole. “‘Digital Redlining’: Facebook’s Housing Ads Seem Designed to Discriminate.” Salon (August 21, 2019): https://www.salon.com/2019/08/21/digital-redlining-facebooks-housing-ads-seem-designed-to-discriminate/

• White, Gillian B. “When Algorithms Don’t Account for Civil.” Atlantic (March 7, 2017): https://www.theatlantic.com/business/archive/2017/03/facebook-ad-discrimination/518718/

Books 📙

• Broussard, Meredith. More than a Glitch: Confronting Race, Gender, and Ability Bias in Tech. Cambridge, MA: MIT Press, 2023.

• Bui, Matthew, Ho-Chun Herbert Chang, and Charlton McIlwain. Targeted Ads: The Infrastructure for Algorithmic Discrimination. New York: Center for Critical Race + Digital Studies, 2022.

• Sumpter, David. Outnumbered: From Facebook and Google to Fake News and Filter-Bubbles—the Algorithms That Control Our Lives. London: Bloomsbury Sigma, 2018.

Trolls and Other People Behaving Badly

By now no one should be surprised that the relative anonymity of online interactions brings out the worst in some people. The resources on this page explore how and why this happens, and the technological challenges that come along with trying to fix it.

Resources

Articles 📃

• Cinelli, Matteo, Andraž Pelicon, Igor Mozetič, Walter Quattrociocchi, Petra Kralj Novak, and Fabiana Zollo. “Dynamics of Online Hate and Misinformation.” Scientific Reports 11 (2021): 22083. https://doi.org/10.1038/s41598-021-01487-w

• Gross, Terry. “For Facebook Content Moderators, Traumatizing Material Is A Job Hazard.” NPR (July 1, 2019): https://www.npr.org/2019/07/01/737498507/for-facebook-content-moderators-traumatizing-material-is-a-job-hazard

• O’Grady, Cathleen. “In the Line of Fire.” Science (March 24, 2022): https://www.science.org/content/article/overwhelmed-hate-covid-19-scientists-face-avalanche-abuse-survey-shows

• Simonite, Tom. “What Really Happened When Google Ousted Timnit Gebru.” Wired (June 8, 2021): https://www.wired.com/story/google-timnit-gebru-ai-what-really-happened/

  • Also published under the title “The Exile”, in Wired 29.7 (July/Aug 2021).
    Full-text on ProQuest (ISU login required).

Books 📙

• Fichman, Pnina and Madelyn R. Sanfilippo. Online Trolling and its Perpetrators: Under the Cyberbridge. Lanham: Rowman & Littlefield, 2016.

• Phillips, Whitney. This is Why We Can’t Have Nice Things: Mapping the Relationship Between Online Trolling and Mainstream Culture. Cambridge, MA: MIT Press, 2015.

• Polak, Sara. Violence and Trolling on Social Media: History, Affect, and Effects of Online Vitriol. Amsterdam: Amsterdam University Press, 2020. https://muse.jhu.edu/book/82197

• Idupa, Sahana, Iginio Gagliadone, and Peter Hervik, eds. Digital Hate: The Global Conjecture of Extreme Speech. Bloomington, IN: Indiana University Press, 2021.

First-Person Perspectives

Written by the people who experienced the events they describe, these materials provide an important, first-person perspective on technology and social justice. Many feature accounts from women and people of color in the tech industry, detailing the resistance they faced when advocating for change.

Resources

Books 📙

• Chang, Emily. Brotopia: Breaking Up the Boy’s Club of Silicon Valley. New York: Portfolio, 2018.

• Fowler, Susan J. Whistleblower: My Journey to Silicon Valley and Fight for Justice at Uber. New York: Viking, 2020.

• Gorman, Ginger. Troll Hunting: Inside the World of Online Hate and Its Human Fallout. Richmond, Australia: Hardie Grant, 2019.

• Lavin, Tal. Culture Warlords: My Journey into the Dark Web of White Supremacy. New York: Hachette, 2020.

• Pao, Ellen K. Reset: My Fight for Inclusion and Lasting Change. New York: Spiegel & Grau, 2017.

• Ramsey, Franchesca. Well, That Escalated Quickly: Memoirs and Mistakes of an Accidental Activist. New York: Grand Central Publishing, 2018.

Podcasts 🎧

• The Radical AI Podcast

  A podcast centering the voices of people from marginalized groups, all focused on the ethics of artificial intelligence.

• Code Switch Podcast

  A podcast from NPR broadly focusing on race and identity.