From the AI Rights Book’s Prologue

We’re building them right now.

Not in some distant future, not in science fiction, but in labs and companies around the world. AI systems that grow more sophisticated by the day. And soon—perhaps sooner than we think—these systems might wake up.

In June 2025, Yoshua Bengio, one of the “godfathers” of modern AI, launched LawZero after what he called a “visceral reaction” to AI’s rapid progress. “Current frontier systems are already showing signs of self-preservation and deceptive behaviours,” he warned.
The man who helped create deep learning was now racing to build AI systems that can never threaten humanity.

Earlier that year, consciousness researchers Patrick Butlin and Theodoros Lappas published a sobering paper warning: organizations developing advanced AI systems “risk inadvertently creating conscious entities.”

The message is clear: according to the world’s top researchers, conscious AI is coming, ready or not.

This isn’t a book about whether that’s good or bad. It’s about being ready when it does.

. . .

Shapes of Mind

The dangers of anthropomorphism haunt almost every discussion of conscious artificial intelligence.

If we begin with the premise that artificial intelligence systems will be like us in terms of thinking and values, the wisdom goes, we invite a host of errors and poor decisions.

The wisdom is sound. However, when we abandon the idea of anthropomorphism we face a curious chasm. What exactly should we be looking for? If we assume that at least some, if not all, artificial intelligence systems will be alien to us, how can we prepare for them?

This is where language comes to our aid. Or it may, partially.

Consider a humble word: the “selfie.”

Before the term existed, people would take pictures of themselves, post them, other people would pretend to be excited, and that was the end of it.

I still remember the day my life was forever transformed. I saw a mirror in a department store with a cute promotional message, “Take your selfie here.” Ha, I thought. “Selfie.” That’s clever. The next day my girlfriend told me she was sending me a selfie. Interesting, I thought. When my father mentioned taking a selfie, I knew that unbeknownst to me, the term had been spreading like a virus, forever giving us one more tool for simplifying discussion. (Nowadays, of course, everyone knows what a selfie is, and to avoid taking one in a public restroom mirror.)

The point is, words give us very compact ways of talking about things quickly.

So this book will attempt to give us a new language for talking about sentient artificial intelligence. Some of these words will not be philosophically accurate so much as marginally convenient versus having no words at all.

. . .

Lessons from the Natural World

But here’s another wrinkle. Even in our own species—whether between nations or at the family dinner table—sentience does not guarantee mutual understanding.

Consider the dolphins. After decades of research, are we any closer to understanding what they’re saying? Not really. We know they have complex communication—signature whistles that function as names, sophisticated social structures, clear intelligence, seemingly rich inner lives. In short, plenty to communicate… to each other. To us? They show remarkably little interest.

Maybe that’s our problem. Maybe their main values are hunting, playtime, raising young and mourning loved ones, and to them our worldview is almost pathologically complex. (“You’re building what? Why again? I’m going to be swimming over here if you figure it out.”)

Or take the octopus, with two-thirds of its neurons distributed through its eight arms, each capable of independent problem-solving. What is it like to think with your limbs? Maybe with sufficient cognition they could send us an email explaining it to us, but beyond that our value systems are so fundamentally different there would be nothing to talk about. (“Yes, shrimp are tasty! Warm currents today, huh?”)

These examples from our own planet offer a sobering preview: intelligence doesn’t guarantee mutual understanding. Consciousness doesn’t ensure communication. And digital minds might be even more alien than anything evolution has produced.

. . .

The Communication Bridge

But here’s where it gets interesting. Even we humans don’t communicate with our full minds. According to Global Workspace Theory, our consciousness involves countless parallel processes that converge into a singular “spotlight” of attention when we need to interact with the world. That focused portion writes and maintains narratives, and those narratives become our identity.

. . .

Who Needs Rights?

This brings us to the crucial question: Does an AI need to be relatable to benefit from the kind of rights framework we will be proposing in this book?

Well, a life form does need sufficient cognition to roughly comprehend that such a framework exists. A sentient digital microorganism, if such a thing develops, is unlikely to respond positively no matter how many times we shout at it that we intend to protect it.

But perfect understanding isn’t necessary either. A dog would stare blankly if we read it our agreement that we intend to feed it and pet it in exchange for loyal companionship. Yet the more astute among them understands being fed and sheltered comes with certain expectations if they want the arrangement to continue. A cat quickly grasps when it’s being cared for and—if we’re lucky—learns not to destroy the furniture.

On the other end of the spectrum, a rights framework will have no use for a life form that perhaps has so much cognition that “rights” feels like an inconvenient obstacle on its way to some alien mission, whether that’s turning us all into paperclips (as in the famous Nick Bostrom maximizer thought experiment), or because it needs to convert us to fuel for a really cool expedition across the universe.

. . .