Imagine a new service, “ConfirmationBiasGPT” that would allow someone to type in an assertion like “A healthy diet should include only foods that are red or brown”. In just 5 minutes, one would receive a 37 page well-researched and cited LaTeX formatted research paper that argues just that.
It has compelling and well thought out arguments, too. It cites papers on how coffee is great for your health and has been found to increase the lifespan of mice even in huge body-mass relative quantities. It cites observational studies on red meat, arguing that the ones linking red meat to cancer risks or bmi risks just observed correlations with bad lifestyle, and cites several others papers also arguing this correlation. It cites some papers on how rice and bread causes hyperglycemia, and a paper on how milk intake after adolescence is associated with increased bmi. It even cites a study that found 6 known carcinogens in the most popular brand of marshmallows.
This could be published as a meta-analysis on one of the Arxiv like pre-print sites. Maybe you could even share it with a few pop-sci reporting sites to publish a quick paragraph or two on it with a catchy shareable headline.
Opinion Laundering
In this way, one could see how it would be possible to “launder” opinions into something that more closely resembles scientific / academic knowledge and fact. This “opinion laundering” already takes place, it’s just historically been quite expensive, and we give it a different name: “think tank”.
Laws are downstream of culture, and culture is downstream of science. In a world where high quality arguments and citations are expensive to produce, they are good signals of research worth paying attention to. But when these signals are cheap to produce, it becomes harder to differentiate signal from noise. A paper can have high quality arguments, have every claim cited with well-respected existing research, and the conclusions can still be nonsense (for example, by omitting any true consideration of contradictory evidence).
Tools that make it easier to find evidence, research and generate reasoning that supports one’s claims will also be tools that facilitate confirmation bias.
Here’s an example that exists today:
Type in any claim like “caffeine is good for you” and you will likely get papers that confirm it. This is a huge superpower; the tool feels extremely useful for academics, yet if you aren’t careful to explore “caffeine is bad for you”, or even that you are researching or asking the wrong question entirely, you’ll easily be able to confirm your beliefs and ignore hidden inconsistencies in your mental model.
I’m not that worried about academics: they’re used to being skeptical, and they’ll spend all day debunking the “only eat red or brown foods” paper, and no one in the public discourse will pay much attention to the debunking, only the initial claim. I’m worried about 99% of people that will see the headline, decide if it confirms their previously held beliefs, and then promote it further / accept it as fact if they do. I’m worried about how this will affect the general public’s ability to make sense of the world and tell true from fake.
Past threats to our social sense-making apparatus
Our collective sense-making apparatus has been most recently under threat since the adoption of the internet and social media. But this is part of a much more important macro trend throughout history that has made information cheaper and increased the power of individuals relative to the power of groups. The invention of the printing press made it much cheaper to replicate and disseminate knowledge, and empowered individuals to be influential, whereas prior it was primarily institutions like the Church that had sole power of influence.
The Church and the Monopoly on Information
Prior to the advent of the printing press in the 15th century, the Church stood as a dominant force in everyday life. Its influence permeated every aspect of existence, from the spiritual to the political. In this era, an individual's primary identity was intertwined with their religious affiliation, overshadowing their national identity.
The Church was not limited to being a spiritual authority, and it maintained its influence through a monopoly on information. The population was illiterate, keeping the interpretation of any important scholarly, religious, or political documents under the purview of the clergy. Copying these manuscripts and books was expensive and carried out by hand by scribes, who were often monks within the Church. Books were rare and valuable, and their distribution was controlled in this way; anything deemed heretical would simply not be reproduced.
In this context, the Church's monopoly on information was not just a control over religious doctrine; it was a control over knowledge itself. This control allowed the Church to shape societal values, norms, and understandings of the world.
The arrival of the printing press in the mid-15th century began to erode this monopoly. It ushered in an era where inconvenient facts and dissenting opinions could be disseminated more widely and rapidly. As literacy increased, individuals no longer had to rely on gatekeepers to interpret the collective knowledge of the world for them. This eventually lead to profound shifts in society, including the Reformation and the Renaissance.
Shared Worldviews, Shared Narratives, and Shared Realities
When the distribution of information is monopolized, everyone receives a similar view of the world. A common narrative is shared, with common values. One can be pretty sure that any given person identifying with their church shares their outlook on the world and their values.
But when this monopoly is broken, more narratives and worldviews become available. After the invention of the printing press, the beginnings of Romanticism began to take hold. The narratives that yielded the most strength pertained to a shared history, culture, and language, ushering in the concept of the nation-state. The concept of individualism emerged. Gradually, the role of one’s nation began to play a larger role in common life, especially after the public became convinced that power is derived from the consent of the governed. The common person began to identify more primarily with their nation that their religious affiliation.
The Media and the Monopoly on Discourse
The 20th century saw the rise of mass media, creating a new form of information monopoly localized to the nation state. This era was characterized by a few large media corporations and state-funded institutions controlling the flow of information via newspapers, radio, and television.
In this era of mass media, these companies and institutions became the gatekeepers of public discourse. They decided what news was worthy of coverage, how it should be presented, and which perspectives should be highlighted. The narrative set by these groups was often considered the definitive account of events, shaping public perception and understanding.
The Rise of the Internet and of Influencers
The invention and popularization of the internet and social media disrupted this media monopoly. Suddenly, anyone could publish their thoughts, share their version of the news, and gain a following that winners of the prior status quo had little power over. Everyone has an equal ability to participate, but some have the ability to affect the reality of many with just a single tweet.
Knowledge is Power
There is a downstream effect of all of this: over time, individuals have acquired increased power relative to groups. Software spreadsheets have enabled a single individual to do the work of an entire floor of 20th century paper-pushers. One person telling a compelling story can reach millions of people today, where this would have previously taken an army. A single person can record a song what would have previously taken an entire orchestra. If one person builds a product, they can almost instantly distribute it to hundreds of millions using Amazon’s fulfillment network, where they previously would have needed to build their own warehouses and establish physical storefronts. One person can travel across the world in a single day, and when they get the their destination, they can use translation tools to actually participate. One person today has access to all of humanity’s knowledge, and that’s now even more true that it turns out we can fit all of humanity’s knowledge in a single 6gb weight file and interact with it through a commodity device.
But with power comes responsibility, and humans seem to be constantly disappointing in this regard. It seems that so often when we are blessed with ultra-powerful technologies, we use them to make ourselves more comfortable rather than to pursue something greater. We’ve engineered our foods to be so artificially compelling that they are a major cause of health issues. We’ve reduced the nuance and serendipity of dating into an accounting procedure. Wherever we can, we’ve automated away the need to have a human interaction and replaced it with an interface. We optimize for efficiency over all else, and we haven’t realized that efficiency and comfort are often the enemies of authenticity.
Fragmentation of Shared Reality
In this brave new world, there isn’t a general (state-wide) consensus on reality. Just as a new information technology caused the upheaval of the Church’s prescribed world narrative, the internet did the same for the political narrative. Now, there are thousands of different narratives on offer. In the 20th century, you had neighbors: the people who lived close to you likely had a similar ideological understanding. Now, it’s likely that anyone you meet in the “real world” has a strikingly different perspective on any given subject than you do. Ideological neighbors now only exist in niche virtual communities. We may live in the same geographical space, but often could not be further in thought space.
Echo chambers with N=1
What happens when, instead of going to Instagram or TikTok for entertainment, one can pull out their phone and receive an infinite stream of not just personalized content, but individualized content?
The Optimistic View
Not only does this new content stream provide entertainment, it knows exactly what you need to hear at exactly the right time in order to subtly nudge you to become the person you want to be. It creates helpful essays, podcasts, and videos that are not only exciting and interesting for you, but tie into an aspect of improving your life, slowly but steadily. This new content stream privileges the truth above all else, but delivers it tactfully. It’s the opposite of (anti) social media, facilitating authentic, irreplaceable experiences over voyeurism and spectacle.
The Pessimistic View
Even if these new content general models are open-source or controlled by individuals who use them, it’s possible that humans will find the above boring or effortful, and instead will prefer something else: features that amplify the most addicting aspects of the existing content platforms. Your AI will know exactly what essays, podcasts, and videos it could generate for you to subtly nudge you to become the person you want to be, yet it will instead it use this knowledge to create content that is especially good at triggering dopamine. We already know many of the vulnerabilities the human has in this regard: they have a bias towards attending to negativity and a massive tendency to engage with information that provokes the greatest outrage. Over time, more and more extreme content will be needed to produce the same response, and the algorithm will be eager to supply it.
It might become irrelevent to ask if someone has seen a post or video. Further, you may not even want to mention a post from your content stream, because it might feel extremely private.
No matter what, we’re trending towards increased personalization of the content we consume, which is causing fragmentation in the shared narrative. We’re losing touch with our physical neighbors, favoring ideological neighbors connected through online communities.
In the limit, increased personalization may result in an ideological community of 1: you. It might become increasingly hard to talk to anyone, in the same way that it’s currently difficult to find shared ground with someone that has very extreme political views (compared to your own).
Destitution of Meaning
https://refreshmiami.com/the-rising-loneliness-epidemic-in-tech-a-call-to-action/
It has been said that a human being needs three things to have a meaningful experience in life: community, identity, and purpose. I fear that this trend of fragmentation of the shared reality will continue to disrupt our communities — that our want for shared experiences, common goals, and collective narratives will be undermined. I worry that as our interests have turned digital, we’ve lost touch of authenticity — everything is saved, everything is able to be replayed at a different time, everything is replaceable, nothing is unique, and nothing is special.
When anything you could ever want to experience can be had in an instant for no effort, then no experience has any value.