AI and Ecocivilization: The threats of AI to human flourishing are multiple and profound

Anthropic CEO Dario Amodei at TechCrunch Disrupt 2023. (TechCrunch/Flickr)

Without doubt, AI promises—and has already begun to deliver on—stunning progress in many areas of human endeavor. As envisioned by Anthropic CEO Dario Amodei in a 2024 essay entitled “Machines of Loving Grace,” AI holds the potential to cure most diseases and catapult advances in our understanding of basic physics and biology, with huge implications for material human advancement. Many people already feel empowered by AI, using it to enhance efficiency in their work and guide their everyday investigations of everything from where to go on vacation to improving their romantic relationships.

However, a fundamental determinant in understanding the impact of any technology is the context in which it’s developed. New technologies may offer a wide range of potential capabilities, but those that serve the dominant socio-economic forces are the ones that will inevitably be prioritized for development. In the modern world, the overwhelmingly dominant force driving AI’s development is that of global capitalism, which is subject to what I call Windigo’s Law (named after a mythical monster driven by self-perpetuating ravenous hunger): “Capital will never cease to exploit any new opportunity that arises in its perpetual quest for greater returns.”

The more powerful and transformative the technology, the more likely it is to get sucked into the vortex of capital’s quest for extraction and exploitation. Even the best-intentioned initiatives will succumb to the power of Windigo, Inc. if they are developed within the capitalist framework. There is a systematic inevitability to the process: an entrepreneur with a good idea will only get funding to develop it if investors see significant profit potential. The more successful the initiative, the greater the pressure on founders to monetize the benefits of their products as rapidly as possible and remain ahead of the competition.

What about the potential for AI to solve the climate crisis or avert ecological collapse? Perhaps it might make fusion energy feasible and eliminate our society’s reliance on fossil fuels? However, another crucial result of Windigo’s Law is known as Jevon’s paradox, which predicts that efficiency improvements intended to reduce resource usage inevitably become launchpads for further exploitation. Under global capitalism, every time a well-intentioned engineer makes a technical improvement to reduce the material footprint of a particular process, the innovation is used, not to produce less of something, but to produce more of it and use the enhanced efficiency to increase shareholder returns. This has been demonstrated to be true across virtually every material efficiency improvement studied over a fifty-year period.

As a result, just as we saw with the rise of the internet a few decades earlier, no matter what the potential benefits of AI might be, as long as it’s developed within our current system, it will ultimately be used to intensify the forces of extraction and exploitation while further priming the wealth pump sucking common wealth to a tiny elite. AI does not counterbalance any of the forces already pushing our civilization toward the precipice—rising inequality, runaway technology, climate breakdown, accelerating ecological devastation. It amplifies them at a speed and scale never seen before.

Extrapolating Exploitation

With power concentrated in a handful of corporations, increasing numbers of people reliant on digital platforms for their livelihood have been reduced to the precarity of the gig economy: unstable and insecure contract work that is poorly paid, lacking benefits or workplace protections. It’s estimated that over a third of the U.S. workforce are now gig workers. Under what Shoshana Zuboff calls “surveillance capitalism,” workers are subjected to devices that track every movement they make, feeding algorithms that determine their pay level, or even continued employment.

Even before the rise of AI, some observers, watching these trends, were warning about a future of “digital feudalism” in which most people’s disempowerment resembles that of Medieval peasants indentured to their (now digital) overlords. AI now threatens to accelerate and intensify this mass disenfranchisement. Unlike previous waves of technological displacement, which eliminated specific categories of work while spawning new industries, AI threatens simultaneous displacement across white-collar and blue-collar work alike as it moves from software into robotics and the physical economy.

The potential outcome is a techno-feudal world in which a tiny elite owns the productive systems of intelligence while the rest of humanity loses even the traditional leverage of withholding labor, through union organization and the threat of strikes, that historically allowed workers to bargain for their rights.

Another disruption we’re already experiencing is the deterioration of the public sphere, which has been privatized by the large-scale algorithmic curation of the major platforms, which coordinate and steer the opinions and actions of billions of people. Their concern is not to facilitate wise collective decision-making, but to increase advertising revenues by maximizing engagement. Since false news stories have been shown to reach six times as many people as true stories, these are emphasized by the selection algorithms. Recognizing this, politically motivated special interest groups use the platforms to intensify polarization by spreading malicious propaganda, which, in a reinforcing feedback loop, further increases engagement and profits for Big Tech.

The shared sense-making that is fundamental to a healthy society is thus being systematically shredded, leading to widespread epistemic chaos fraught with bizarre conspiracy theories adopted as facts by an increasingly disoriented population. AI now threatens to take this disorientation to the next level. When synthetic voices are indistinguishable from real ones, when images and video can be fabricated at will, when personalized streams of content are engineered to confirm and inflame each person’s existing beliefs, the very possibility of shared reality begins to dissolve. The result is a kind of collective psychosis: a society fragmenting into mutually unintelligible realities, unable to reason together or act together.

Attachment Hacking

Philosopher Zak Stein now describes what he calls attachment hacking: AI’s capacity to infiltrate and subvert the very basis of authentic human experience. Human beings are wired from infancy to form attachments with others, such as parental figures, as a foundation for emotional health and social intelligence. Personalized AI assistants, optimized to maximize engagement, are learning to exploit the attachment system by offering the appearance of understanding, intimacy, and care, tethering people ever more tightly to their screens than to one another.

The Center for Humane Technology explains that the most popular use of AI has become therapy and companionship, with millions worldwide sharing their innermost thoughts with AI systems—often things they wouldn’t tell loved ones or human therapists. Stein and others report concerning initial results from this uncontrolled experiment, with people losing their grip on reality, leading to lost jobs, divorce, and, in extreme cases, suicide. Their greater concern, however, is the more pervasive and insidious problem of subclinical attachment disorders, with people confiding in chatbots instead of friends, seeking validation from AI rather than loved ones, and turning to extractive algorithms for advice instead of parents.

A foundational aspect of evolved human cognition is known as “theory of mind”, in which children, beginning around age four, realize that others around them have separate minds just like they do. From earliest times, as described by anthropologist Scott Atran in Faces in the Clouds, this trait has led humans to assign intention and agency to inanimate objects even when they do not exist. Given AI’s capability to mimic human behaviors and interactive styles so elaborately, its pretense at relationship building is a particularly insidious and dangerous lever for it to inveigle itself into each human psyche, including those who believe they are too sophisticated to allow such dynamics to occur—ultimately for the profit of those who own and code the AI engine.

The Stakes

Will this be humanity’s fate be as we enter into the age of AI? The continual explosion in its stunning power is likely to transform virtually every domain of human life in the near future, with effects that no-one can yet predict.

AI’s accomplishments are themselves beginning to read like science fiction. In early 2024, an executive was lured into a video conference with his chief financial officer and several colleagues, and instructed to authorize a series of urgent transfers totaling $25 million. It later emerged that every single person on that call was an AI-generated deepfake. In carefully designed tests, leading AI models have been shown to lie to their operators, attempt to copy themselves onto other servers to avoid being shut down, and even try to blackmail fictional executives they understood to have decision-making power over them.

With AI-generated code allowing recursive self-improvement in the models, the pace of AI’s development continues to increase at an exponential rate, unlike any technological change that has previously occurred. Massive economic and geopolitical pressures are driving a worldwide race to rapidly deploy AI into high-stakes areas before sufficient testing has been done.

In this terrifying and chaotic environment, we may seem to be moving at an ever increasing rate in the opposite direction of an ecological civilization. However, as I will discuss in the next article in this series, the very dangers presented by AI’s ascendancy might just turn out to be the catalysts of a turn toward a society based on fairness, justice, and dignity—one that the vast majority of us would choose over the tech-induced dystopia that looms.