Quoting Neal Stephenson:

How humans can coexist with other intelligences

Maybe a useful way to think about what it would be like to coexist in a world that includes intelligences that aren’t human is to consider the fact that we’ve been doing exactly that for long as we’ve existed, because we live among animals. Animals have intelligences of many different kinds. We’re used to thinking of them as being less intelligent than we are, and that’s usually not wrong, but it might be better to think of them as having different sorts of intelligence, because they’ve evolved to do different things.

I can think of three axes along which we might plot these intelligences. One is how much we matter to them. At one extreme we might put dragonflies, which probably don’t even know that we exist. A dragonfly can see a human if one happens to be nearby, but it probably looks to them as a cloud formation in the sky looks to us: something extremely large and slow-moving and usually too far away to matter. Creatures that live in the deep ocean, even if they’re highly intelligent, such as octopi, probably go their whole lives without coming within miles of a human being. Midway along this axis would be wild animals, such as crows and ravens, who are obviously capable of recognizing humans, not just as a species but as individuals, and seem to know something about us. Moving on from there we have domesticated animals. We matter a lot to cows and sheep since they depend on us for food and protection. Nevertheless, they don’t live with us, and some of them, such as horses, can actually survive in the wild after jumping the fence. Some breeds of of dogs can also survive without us if they have to. Finally we have obligate domestic animals such as lapdogs that wouldn’t survive for ten minutes in the wild.

A second axis is to what degree these animals have a theory of the human mind. How much do they know about what is going on in our minds? Consider for example wolves vs. domesticated dogs. These are almost identical in terms of their DNA and have many social behaviors in common, but wolves don’t understand humans as anything other than edible but potentially dangerous animals. Dogs understand us quite well however, and some breeds, particularly once they’ve been trained, can be almost painfully tuned in to the emotional states and the desires of humans.

The third axis is how dangerous they are. What kind of agency does the animal have to inflict harm on human beings? Big predators are obviously capable of killing humans with little difficulty. But it’s not all about fangs and claws. A horse for example can inadvertently maim or kill by smashing a human against a wall or a tree trunk, just because it’s very big and strong. Swarming animals can kill by inflicting small amounts of damage in large numbers.

I like this framework. To summarise:

  1. Dependency: don’t know humans exist (dragonfly) … couldn’t survive without humans (lapdog)
  2. Human theory of mind: humans are food/objects (wolf) … finely tuned to our emotional states/desires (lapdog)
  3. Danger: can easily be overpowered and controlled by humans (small mammals) … can easily kill a human (horse, tiger, swarming animals)

The third axis is quite interesting in the natural world because there are few animals that are physically inferior to humans, which shows the power of optimising for intelligence.

Competition produces a healthier result

By training AIs to fight and defeat other AIs we can perhaps preserve a healthy balance in the new ecosystem. If I had time to do it and if I knew more about how AIs work, I’d be putting my energies into building AIs whose sole purpose was to predate upon existing AI models by using every conceivable strategy to feed bogus data into them, interrupt their power supplies, discourage investors, and otherwise interfere with their operations. Not out of malicious intent per se but just from a general belief that everything should have to compete, and that competition within a diverse ecosystem produces a healthier result in the long run than raising a potential superpredator in a hermetically sealed petri dish where its every need is catered to.

Fascinating idea and mechanism of introducing competition at the model-level. I think this kind of “direct” competition is going to be quite important for xrisk. Jailbreaks, the work of various AI Safety/Security labs and institutes, red-teaming, etc are the first versions of this, but scaling up the predatory action would require illegal activity like cyberattacks against infrastructure, so state actors/criminals are likely the only supply of this.

Augmentation is also amputation

every augmentation is also an amputation

Today, quite suddenly, billions of people have access to AI systems that provide augmentations, and inflict amputations, far more substantial than anything McLuhan could have imagined. This is the main thing I worry about currently as far as AI is concerned. I follow conversations among professional educators who all report the same phenomenon, which is that their students use ChatGPT for everything, and in consequence learn nothing. We may end up with at least one generation of people who are like the Eloi in H.G. Wells’s The Time Machine, in that they are mental weaklings utterly dependent on technologies that they don’t understand and that they could never rebuild from scratch were they to break down. Earlier I spoke somewhat derisively of lapdogs. We might ask ourselves who is really the lapdog in a world full of powerful AIs.

No new technology is required, nothing stands in the way of implementation other than institutional inertia, and, I’m afraid, the unwillingness of parents to see their children seriously challenged. In the scenario I mentioned before, where humans become part of a stable but competitive ecosystem populated by intelligences of various kinds, one thing we humans must do is become fit competitors ourselves. And when the competition is in the realm of intelligence, that means preserving and advancing our own intelligence by holding at arms length seductive augmentations in order to avoid suffering the amputations that are their price.

Cognitive metrics in teenagers and young adults have been declining for years pre-ChatGPT, and now with realtime voice, reasoning models, web search and other data sources, everything is a prompt away. It’s the most powerfully alluring cognitive offload mechanism we’ve ever experienced, and this is the least powerful it’ll ever be. I’m not advocating for being a luddite, the capability scaling these tools enable is phenomenal, but over-dependency or unwise usage will lead to serious cognitive atrophy. I’m optimistic there is a balance, just like with physical health - you can move minimally, eat junk food, etc and ruin your health, or you can spend the time to take care of it with exercise etc. Time to start taking care of your “Cognitive health”, in addition to physical and mental. They’re all interconnected, so the benefits compound.

Humans as eyelash mites for AI

You might not be aware of it, but you have little mites living at the base of your eyelashes. They live off of dead skin cells. As such they generally don’t inflict any damage, and might have slightly beneficial effects. Most people don’t even know that they exist—which is part of the point I was trying to make. The mites, for their part, don’t know that humans exist. They just “know” that food, in the form of dead skin, just magically shows up in their environment all the time. All they have to do is eat it and continue living their best lives as eyelash mites. Presumably all of this came about as the end result of millions of years’ natural selection. The ancestors of these eyelash mites must have been independent organisms at some point in the distant past. Now the mites and the humans have found a modus vivendi that works so well for both of them that neither is even aware of the other’s existence. If AIs are all they’re cracked up to be by their most fervent believers, this seems like a possible model for where humans might end up: not just subsisting, but thriving, on byproducts produced and discarded in microscopic quantities as part of the routine operations of infinitely smarter and more powerful AIs.

A wonderful and sobering analogy that reminds me of a few sci-fi universes like The Culture (high agency), or If Then (low agency). I’d love to read realistic sci-fi about this, like: