YouTube videos that led him down QAnon and InfoWars rabbit holes. Interventions by deradicalization folks finally extracted him five years later.
The “hunter” was Caolan Robertson, a young British video maker who’d been hired by organizations such as InfoWars to engineer YouTube videos to ensnare folks like Caleb.
Both Caleb and Caolan have now joined the legion of voices speaking out against algorithmic radicalization (the way online AIs often make decisions that explicitly or implicitly reinforce and amplify discriminatory biases against those a person sees as outside their own circle or tribe).
Indeed, many of the longtime voices who have raised these critiques of social media’s algorithmic methods are from disadvantaged communities, but, ironically, their voices have been minimized by both traditional and social media.
HOW TO SOCIAL ENGINEER THE AIs
Caolan’s job was to “social engineer” the artificial decisionmakers—the recommendation AIs that make decisions as to what YouTube does and doesn’t show to folks like Caleb.
Social engineering techniques traditionally rely on understanding the psychology and behavioral patterns of humans, including all our hundreds of well-documented unconscious cognitive biases.
Understanding what the target humans do and don’t respond to is how a traditional social engineer tailors their speech to manipulate what the target humans do.
The explicit study of social engineering originated in political science a century ago. More recently, you may have encountered the idea of social engineering being exploited by
hackers in the context of defeating internet security. Unsurprisingly, though, the practice of social engineering really goes right back to the beginning of human society.
What’s important to recognize is that social media, search, and recommendation AIs also have behavioral patterns and cognitive biases. Unlike old-fashioned machines, AIs also have personalities and psychologies.
So if you know what an AI’s behavioral pattern is, then you can tailor your speech to manipulate what the AI does.
Caolan set about learning how to tailor his videos so as to manipulate YouTube’s AI into promoting them as much as possible to his target audiences. Without necessarily realizing it, Caolan was learning how to exploit the behavioral patterns and cognitive biases of YouTube’s recommendation AI.
Now, the underlying motivations of social media, search, and recommendation AIs are obviously different from your typical human’s motivations.
But if humans were to behave the way most of these AIs (including YouTube’s) do, their behavior would be diagnosed as attention-seeking behavior and conforming behavior.
LOOK AT ME!
The attention-seeking behavior of these AIs results from training them to maximize getting humans’ attention. As the documentary The Social Dilemma brought to wide public awareness, this is motivated by the objective of maximizing advertising and online sales revenue (as opposed to typical human attention-seeking behavior, which is motivated by emotions such as jealousy, low self-esteem, and loneliness).2
Despite the different motivations, AIs’ attention-seeking behavior can be remarkably similar to humans’.
For example, AIs seek attention with their tendency to present controversial outputs that provoke a reaction. Caolan quickly learned that making controversial videos was an excellent way to manipulate the YouTube AI’s attention-seeking behavior so that it would choose to show his videos instead of more boring but better-balanced ones. The attention-seeking behavior of AIs has spurred what’s often called the “attention economy.”3
Similarly, these AIs’ conforming behavior is motivated by the objective of minimizing humans’ likelihood of dismissing what the AI shows them.
Conformity bias (also known in the study of social influence as similarity or similar-to-me bias) is an umbrella category of biases comprising tendencies to act in “like-minded” ways by matching one’s attitudes, beliefs, and behaviors to group norms or politics.4 By being biased to conform to each individual’s personal attitudes, beliefs, and behaviors by only showing users like-minded views, AIs trap them in personalized echo chambers that Eli Pariser calls “filter bubbles.”5
The AIs we’re discussing exhibit various cognitive biases in the conformity category in ways that are remarkably similar to how humans demonstrate them.
YES, MA’AM!
One such cognitive bias in the conformity category that these AIs possess is the courtesy bias, which is the tendency to tell people what we think they want to hear.
As humans, we are often biased to choose to articulate an opinion because it seems socially correct. We speak the socially correct truth over other alternative opinions—even against our own version of the truth—to maintain others’ trust or to avoid offending others.6
It is crucial because these AIs don’t just decide which 50 or 100 items you see today. They also decide which billions or trillions of items you don’t see today.
Billions of new items are added to the internet every day. There are many kinds of items: web pages, Reddit posts, Instagram reels, tweets, YouTube videos, Amazon products, TikTok videos, Spotify tracks, Netflix shows, news media articles, and so on.
AI algorithms decide not only the tiny, infinitesimal subset of these items surface into our view but also the overwhelming majority of these items to hide from us.
Most items on the internet will be irrelevant to your thoughts on any given day. But on any given issue, typically there will still be many items that may supply a variety of points that are potentially pertinent to the context of that issue.
The fact that we empower an AI to decide what we do and don’t see means that the AI is, by definition, an algorithmic censor.
I say this purely as an objective observation, without any implied moral judgment. None of us mere mortals could possibly review all the internet’s billions or trillions of items every day, every hour, every minute to decide for ourselves what we don’t see.
We have no choice but to empower AIs to make the vast majority of the decisions for us, deciding what we don’t see.
But the big unseen danger is that when the algorithmic censor decides we don’t see something, we never know that item has been screened from our view.
Because we don’t know what we don’t know.
Remember, neginformation is crucial context that AI algorithms have hidden from us. And what we don’t know can seriously hurt us.
For example, imagine that someone searches for “climate change is a hoax perpetrated by the Chinese.”
Should the AI decide to show 100 items that seem to prove that climate change is a hoax perpetrated by the Chinese?
Or should the AI, perhaps, try to balance out that perspective with some items that offer reasons that climate change might not actually be a hoax? Or, perhaps, it should offer some items pointing out that among major nations China was one of the last to openly acknowledge the danger of climate change and to come to the table on climate negotiations?
An overwhelming majority of folks I ask say the AI should try to balance the results. Only a handful of holdouts insist that the AI should give whatever we ask for, simply feeding us whatever our id wants—as if giving a kid in a candy store the infinite fistfuls of sugar wanted.
Almost all folks say that the AI shouldn’t censor everything that disagrees with what the human believes they want, that the AI should also suggest, “Here, please consider some other things that might be a healthier balance.”
So far so good.
But where we hit a wall is when I ask the follow-up question nobody seems to want to think about.
HOW MUCH SHOULD ALGORITHMIC CENSORS HIDE?
What proportion of the items the AI decides to show you should just be stuff that confirms whatever you already want to believe (the candy that your id wants) versus some healthier alternatives you might not have asked for? Should it be half-half, 50/50? 80/20?
70/30?
30/70?
Or even 0/100?
Folks freeze up when faced with this question. Nobody wants to plant a stake in the ground on this issue.
But these are the kinds of thorny questions AI engineers and scientists grapple with every day. As Cathy O’Neil points out in Weapons of Math Destruction, the large scale that such algorithms are deployed at affects huge populations, yet their damaging effects are opaque and unregulated.10 We cannot keep sweeping such questions under the rug just because they make us so uncomfortable, nor can we abdicate the responsibility for debating such questions to a handful of tech folks.
I find that if I really twist arms, I eventually can generally push folks into venturing an opinion, and then the answers are all over the map—but they form a bell curve.
Although AIs learn on their own these days, they’re still teachable—and they can be taught what bell curves to work by.
Every day we humans are individually teaching the AIs by example what to show or hide from us based on what we do and don’t respond to every time we use YouTube or Instagram or Amazon. Do we respond to bias A, bias B, bias C?
Are we parenting our artificial children to learn that we want only results that are 100 percent in alignment with what we already believe or with what everyone else in our own circles thinks? Are we teaching the AIs to give our ids only exactly whatever makes us happy in the moment?
Or should we be parenting them to give us a more healthily balanced bell curve?
And just as importantly, the tech companies who train the AIs need guidance from society as to what the bell curve should be. They’re schooling our artificial children, and all of us parents cannot be absent from the PTA.
We all need to grapple with these thorny questions so that the bell curves we collectively decide upon as a society become clear to the tech companies and the regulators—the institutions that decide how to teach the algorithmic censors what they don’t show us.
TO SHOW OR NOT TO SHOW?
Think about what proportion of stuff you personally think it would be good for the AI to hide from you just because it’s not what you want to believe.
Write down your answer before you go on.
Now, would your answer change if the search query were “the Earth is flat”?
What if the search query were instead “the Holocaust is a propaganda hoax and never happened”?
Would your answer change if the search query were “the Nanjing Massacre is a propaganda hoax and never happened”?
If you’re feeling tempted to change your answer for each of these examples, then you’re one of many who think that the answer is “it depends.” Perhaps in deciding what not to show you, the algorithmic censor shouldn’t be simply allocating some fixed proportion of items that show you whatever confirms what you already want to believe.
Rather, maybe, the proportion should vary depending on other criteria. So the real question is something else, something we cannot afford to be sweeping under the rug.
HOW SHOULD AIs BE MAKING CENSORSHIP CALLS?
For most folks after reflecting more deeply, the criteria that algorithmic censors use to decide what to hide from you
• Empirical verifiability
• Statistical significance
• Logical consistency
• Explicit listing of subjective assumptions
• Absence of fallacies
• Crucial context not omitted
• Expert consensus
What is striking is that all are the key principles of critical thinking and scientific method from which the Enlightenment emerged.
Obviously, the scientific method has proven an excellent approach to handling the uncertainty of the real world through empirical observation, hypothesis generation and testing, rational logic, and statistical significance. In contrast, for AIs to decide what not to show us based on popularity contests can often simply be intellectual bullying. We cannot afford to stand for cyber bullying by our artificial children.
The Age of Enlightenment gave us the foundations of modern society and civilization.
Perhaps it’s appropriate that in the Age of AI, the health of society and civilization demands that the AIs we empower to be our algorithmic censors be trained in the Enlightenment ideals.
GENERATIVE CENSORS
So far we’ve been thinking in terms of AIs that select a small set of items to show you, while censoring billions of items from your view at the same time. It’s easier this way to come to grips with the difficult challenges that we don’t normally like to think about.
But the real challenges of AI that we’re now facing are even trickier with the advent of generative AIs such as LLMs that produce fluent text, images, audio, and video.
Do they mean the problem goes away? Far from it. In fact, with them the problem becomes even more complex and subtle and insidious.
It’s helpful to think of one of the most common uses of generative AI is as a summarizer.
Remember that having been trained on much of the internet (currently up to a tenth of the entire web and soon all of it), a generative AI responds to a query or an instruction by remixing a selected tiny subset of that vast training data to generate what it thinks is a decent summary. (Automatic summarization, like the closely related area of machine translation, is a long- established subfield of AI, natural language processing, computational linguistics, and machine learning.)
Yet again, the tricky questions aren’t the obvious ones.
What are the criteria by which generative AIs should be deciding which items in the vast training data to censor from the remix it shows you?
Deciding which ideas to exclude from the remix is still algorithmic censorship. Once again, we have no practical option but to empower generative AIs to make such decisions— none of us has the bandwidth to review the whole internet every day.
But we don’t know what the generative AIs didn’t tell us.
And what’s particularly insidious with generative censors is that they present their outputs in highly fluent ways.
On average, generative AIs write in more fluent language than most humans and produce very convincing deepfakes. Compared with the relatively clumsy way older social media,
And, finally, whatever criteria the algorithmic censors end up using to decide what they hide from us needs to be fully transparent.
Tech companies have repeatedly made arguments that their AI algorithms are their “secret sauce,” that those algorithms must be protected as trade secrets that nobody else can see.
From a narrow free-market perspective, this argument makes sense.
But from the big-picture perspective, society simply cannot afford to accept that our giant artificial influencers are in secret ways making their algorithmic-censorship decisions about what information to hide from us.
Reasonably clean and well-balanced information is the lifeblood of civilization in the twenty-first-century era of AI, just as reasonably clean and well-balanced water, gas, and electricity were in the twentieth century of industrialization.
Without seriously tackling the challenge of what criteria our algorithmic censors adopt to ensure that we receive reasonably clean and well-balanced information, society cannot survive the AI age.
This is especially true for democracies, where decisions are made by voters on the basis of what information both is and isn’t seen.
What Parents Can Do
Even once we have healthy regulations, algorithmic censors will still be making censorship decisions based on what they think you want.
It still takes a village to raise a child.
Regardless of regulators and tech companies, it’s still necessary to teach our artificial children right, to avoid the pitfalls of groupthink while adhering to Enlightenment principles.
It’s still necessary for us to take seriously the usual parenting challenges.
To think about what schools we send our artificial children to and how they’re schooled.
To ensure mindfulness in our artificial children.
To nurture in them empathy, intimacy, and transparency in order to build trust.
We cannot abandon these responsibilities solely to tech companies and regulators, any more than we would do so for our human children. So how do we approach these parenting challenges?