Machines evolve faster than humans. Based on this insight alone, one could have made a rudimentary argument for an eventual machine takeover as early as 1859, when Charles Darwin established the theory of evolution through variation and natural selection in “On the Origin of Species”. In fact, that is what happened. Already in 1863, Samuel Butler introduced the notion of a machine takeover by comparing biological and technological evolution in his essay “Darwin among the Machines”.
“Day by day, however, the machines are gaining ground upon us; day by day we are becoming more subservient to them; more men are daily bound down as slaves to tend them, more men are daily devoting the energies of their whole lives to the development of mechanical life. The upshot is simply a question of time, but that the time will come when the machines will hold the real supremacy over the world and its inhabitants is what no person of a truly philosophic mind can for a moment question.”
Butler’s words were prophetic. However, it is also worth pondering where the Darwinian analogy falls short. Machine evolution is not just faster than the natural variation and selection that Darwin observed, it is different. Machines neither advance through self-reproduction nor through sexual reproduction. Machines evolve through human-driven intelligent design and combinatorial evolution. Humans are the ones building AIs, so from a short-term perspective we should be able to shape them any way we want. However, the long-term perspective is more complicated. As Butler pointed out:
“The lower animals progress because they struggle with one another; the weaker die, the stronger breed and transmit their strength. The machines being of themselves unable to struggle, have got man to do their struggling for them: as long as he fulfils this function duly, all goes well with him - at least he thinks so; but the moment he fails to do his best for the advancement of machinery by encouraging the good and destroying the bad, he is left behind in the race of competition; and this means that he will be made uncomfortable in a variety of ways, and perhaps die.”1
In other words, the variation in machine evolution comes from human intelligent design, but the selection comes from corporations and countries that use machines competing with each other. This is essentially the military-economic adaptationism proposed by Allan Dafoe in “On Technological Determinism”. Wherever societies compete economically and militarily, those that fail to adopt the most effective technologies, and the social arrangements those technologies require, get out-competed, conquered, or copied.
At the same time, the technological determinism proposed by Allan Dafoe is directional but not absolute. Yes, there are long-run economic and military competition pressures. But we also live in semi-anarchy, not in a war of all against all. It could be argued that seeking power is an instrumentally convergent goal for any intelligent entity. Yet, in practice most human states, most human corporations, and most humans are not maximally power seeking. There are still rules and going rogue has costs.
Similarly, we don’t just let AIs loose on the Internet and see which ones survive in a no-rules competition with other AIs over resources. The selection environment for AIs is not an anarchic war of all against all but shaped by economic incentives which are shaped by human preferences. We ourselves have managed to partly escape the Hobbesian condition through a mix of policing, constitutions, rule of law, nukes, and international institutions. Accordingly, I don’t see a principal reason why it should be impossible for AIs to escape unrestricted competition as well.2
Let’s assume Hobbesian competition, with selection rewarding whoever can outfight or outbreed the rest in a war of all against all, will not turn out to be the dominant selection pressure on future humans. What would even be an alternative? Let’s explore a set of four other speculative selection pressures.
Survival of the kawaii
In America the humanoid robots are cold, adults, made from metal. Yet not everyone dreams of Terminators with laser eyes! This summer, I went on holiday to Japan with my partner and we found warm, needy robots with fluffy skin and big eyes.
Whatever the moral judgement of their societal impact may be, there’s no uncanny valley in interacting with these Lovots. It genuinely puts your brain in the same mode as when interacting with a baby. Trust me! That’s the whole point. Lovots look simple, but they have facial and emotion recognition and an internal assessment of how much they feel loved by you. They are clingy and will follow you around. They are a masterpiece of the Kindchenschema of Konrad Lorenz. These are the facial features that make infants and young animals appear cute and trigger a desire to care for them, such as a large head-to-body ratio. The Japanese are an outlier in how much they have culturally moved from social Darwinism towards kawaii in the last 80 years.
Still, the rest of the world has generally moved in the same direction. It’s kind of weird to think that we have turned the mighty bear from a feared apex predator into a fluffy teddybear that makes children feel safe at night.
Indeed, think about the real-life “teddybearization” of wolves that we domesticated into dogs. Do you think the strongest dogs survive? Do the smartest dogs survive? Well, the strongest and smartest dog is the wolf. However, dogs have much greater reproductive success than wolves in the Anthropocene, outnumbering them by about 3’500:1 today. Even amongst domesticated dogs it’s not necessarily that the top dogs in terms of IQ outbreed dumber dog breeds. The relationship between trainability and popularity is U-shaped: Some Americans love trainable, smart dogs, like Border Collies, German Shepherds, or Golden Retrievers. Others love the cutest, dumbest, most helpless dogs, like Chihuahuas, French Bulldogs or Shih Tzus.
Some dog owners will say things like “young dogs are a full-time job, like a baby”! And they’re right, because many modern dog breeds have been genetically selected for hijacking human baby instincts. Dogs do not live in a Darwinian dog-eat-dog world. They are not in military competition with other dogs. Dog reproduction is almost fully under human control. Human brains deciding which dogs to buy is the unnatural selection environment for dogs. And our brains have evolutionary baby instincts.
Now what happens if AI is the dominant intelligence on Earth shaping the unnatural selection environment? What if humans become to future AI what dogs are to humans today, as some of our esteemed thinkers have suggested? (e.g., Minsky, 1970; Wozniak, 2015)
Well, AI does not have a human baby instinct, it’s not inherently Kindchenschema-maxxing. However, AIs still have reward functions, and that reward function may become the unnatural selection environment for humans. Let’s say the AIs get rewards if they please humans, and simple humans are easier to please than complex humans. Are you ready to be shaped into a predictable reward generator for AIs? Helpless, dependent love for our AI overlords?
I guess that would still be better than harder forms of AI wireheading that cut the humans out of the AI reward supply chain completely, but it’s not really what I aspire to for the human future either.
Survival of the richest
If buying power is market shaping power is reproductive power, then the solution to not end up like Chihuahuas may just be being rich. We just need to get instrumental buy-in from enough superintelligent AIs to respect property rights and pay some AIs to guard our minds from being hijacked and we’re kind of fine. Indeed, if you can avoid the Silicon Valley meme of the “permanent underclass” and become a trillionaire you are much more than fine.
Eventually, in a highly automated, post-labor economy many biological constraints on human reproduction will be technologically solvable. You can get eggs and sperm from skin cells. Artificial wombs may remove the need for a mother to incubate the baby. AI nannies or tutors could relieve some of the burden of childcare. Advanced gene editing could neutralize the risks of inbreeding, removing the genetic bottleneck that once served as nature’s check on dynastic concentration.
Yet, even if biology may no longer be a significant constraint for fertility in the 22nd century, economics will be. Today, additional humans born are expected net contributors to the economy. Though, in practice, raising a child is still expensive, as society allocates most economic costs of raising a new human to the parents, whereas most economic benefits of the adult human are allocated to the person itself and to society at large through taxes and social contributions. In a post-labor future, fertility might be even more capital-constrained, because additional human lives may not be as clear net contributors to the economy and society even during working age. Every human born will still want an income, but many may not be able to do as much profitable work. If we keep the cost of children privatized, this would now have to cover not only growing up but a whole lifetime.
To illustrate this point, let’s make extremized assumptions: No national or global UBI or other safety net, the value of all human labor goes to zero, inflation-adjusted capital returns remain at about 4% per year. In this case, you would need about 300’000 USD in assets today, for a person to be able to live off this indefinitely. At 4% real investment return, this would result in 12’000 USD of inflation-adjusted annual capital income. This is close to the global average GDP per capita.
Among tech billionaires, some are explicitly pro-natalist. Most notably, Elon Musk, who has been an outspoken advocate of pro-natalism, has fathered at least fourteen children by multiple women, many through IVF and surrogacy, with leaked messages suggesting ambitions to father thousands more. In a similar vein, Pavel Durov, the founder of Telegram, has disclosed over 100 biological children conceived through sperm donation across twelve countries and has pledged to divide his $14 billion estate among all of them.
These are two outliers. However, consider that babymaxxing billionaires are still much weaker outliers in terms of fertility than in terms of capital. In contemporary high-income societies, the reproductive Gini is relatively low. Most people have 0 to 3 children. The variance is modest. In contrast, there is wealth inequality of many orders of magnitude between individuals. If children were distributed like stock ownership in the US, the top 1% would have the absolute majority of all offspring.
According to the UBS Global Wealth Report 41% of global adults have less than $10’000 in wealth. Another 42% have wealth of $10’000 to $100’000. So, the majority of the human population could not use their capital returns to cover the indefinite living costs for a new human life. In contrast, Elon Musk’s wealth is estimated at about 1 trillion USD at the time of writing. So, he could afford the capital costs for more than 3 million children. An entire country of Elons and Elonas. So, whereas today fertility variance is within a single order of magnitude, in such a future we may see variance of more than six orders of magnitude. Many can’t afford a child, others are “baby millionaires”. And note that all of these calculations are based on the wealth distribution today. In a future with a starkly declining labor share of income, wealth inequality would be significantly higher than today.
In short, if raising children becomes primarily a capital investment rather than a labor-of-love requiring parental time, reproduction could become as unequal as wealth itself. Or to be more precise, wealth inequality and fertility inequality can be traded off against each other (as having more children is also a form of wealth distribution). Without a broader basis of capital ownership, the ability to have children may become stratified and switch from something that looks approximately normally distributed, to something that follows a power law distribution. At the extreme, this replaces family-centered reproduction with billionaire clans.
Survival of the happiest
By his own admission, Elon Musk is inefficient at turning economic resources into personal happiness. He is richer than any other human, yet he says things like “My mind is a storm. I don’t think most people would want to be me.”, “The amount that I torture myself is the next level, frankly.” and “whoever said money can’t buy happiness really knew what they were talking about”.
Let’s imagine it’s the year 2200 and you are an enlightened planetary AI dictator overseeing a highly automated future economy that doesn’t rely on human labor anymore and has a fixed budget of resources to spend on the welfare of anatomically modern humans. Is Elon Musk the organism you’d want the most copies of?
Utilitarianism states that the most moral action is the one that produces the greatest amount of well-being for the greatest number of people. So, a utilitarian AI overlord would care about the total quantity of wellbeing in the universe, and humans would be one possible substrate for producing it. By that metric, the trillionaire is a terrible investment into human well-being.
The ideal human is cheap to run and reliably content. Based on the World Happiness Report the consistently happiest countries tend to be the Nordics. But they also consume a decent share of resources. If you adjust long-run national happiness for GDP per capita, one of the most efficient countries in terms of turning resources into human happiness is Costa Rica.
Maybe we should try to look at this at a more granular level. Some individuals like the Swedish futurist Anders Sandberg seem to be running on a permanently elevated hedonic set point.
So, a utilitarian overlord with a resource budget might just create a world full of Anders Sandbergs living Costa Rican lifestyles.
Survival of the elect
Anders Sandberg would be the first to point out the catch. He spent two decades at the Future of Humanity Institute next door to Nick Bostrom, who formulated the simulation hypothesis. If future civilizations with God-like technology spanning the unbelievable vastness of space run “ancestor simulations” of humans, we might expect simulated human minds to vastly outnumber original human minds. Hence, the enlightened planetary AI dictator with a fixed resource budget may not be a thought experiment about the year 2200. Maybe we are already living in such a simulation.
However, if the point of this simulation is happiness, shouldn’t we be in a simulation that is more efficient at producing happiness? Personally, I feel very lucky to be alive at this moment in time. However, from a utilitarian point of view, are we really living in Leibniz’s best of all possible worlds?
We do live in a better human world than that of Candide. Indeed, the human world is arguably better off than ever before in history. However, the problem of evil is far from solved. What about the daily slaughter of millions of sentient beings? What about war? What about screwworms? Why is the world not plastered with Costa Rican Anders Sandbergs?
In contrast, our moment in history is seemingly that of a first civilization in an empty universe at the singularity. What are the odds of being born at “the hinge of history”? If life were a birth lottery I would expect “99.99999%+” of sentient beings to be alive in a technologically mature society with advanced space settlement. And of the minority born on civilizations before breakout “99.99999%+” would have been born in times of imperceptibly slow civilizational change. And within human civilization “95%” would have been born at the wrong time to shape the singularity.
In other words, our current world seems exceptionally well suited for ancestor simulations motivated by an advanced civilization trying to understand a) its own origin history, b) general success and failure factors of civilizational take-offs, and c) the likely distribution of civilizational take-offs as it shapes cosmic sociology.
Cool! But does this have any practical impact?
Robin Hanson says be interesting, be worthy of simulation. But what does being worthy of simulation really mean? I’m not sure that the daily small-scale human drama of a telenovela is what a civilization burning stellar amounts of compute wants to learn. If the simulators are studying how take-offs succeed and fail, then what makes you worth simulating is not whether you’re fun to watch. It’s whether you have an impact on the singularity.
Yet, there is still a simulation paradox here. If this is a simulation and you exist in it, this means the future AI overlord has already decided that you *are* worthy of simulation. In a way, we have re-derived Calvin’s concept of unconditional election. God chose specific individuals for salvation before their creation based solely on God’s own will. Does this mean you can just relax, sit back, and enjoy the simulation?
Maybe. Both in the theological Calvin-Arminius debate as now, one can also make the case for conditional election. If the future AI overlord were completely certain over your actions in advance, why bother simulating you in the first place? Isn’t there any information that the simulator would like to learn from the simulation? Maybe future AI overlords predict that you are worthy of simulation, but that prediction leaves room for uncertainty and for free will of humans. In other words, being simulated means the future AI overlord thinks you might be high potential. However, this is in part an exploration and resources spent on this simulation will be adjusted based on whether the experiment yields interesting results.
So, what behavior within the simulation maximizes how much it is worth to invest in simulating you? Assuming this is a historical simulation, we can speculate: conditional election is Kolmogorov complexity times historical impact.
Historical impact matters for obvious reasons. Kolmogorov complexity matters because it defines at what resolution it is worth simulating a subject. Distant stars have many more atoms than your brain if they really exist, but to simulate their functional impact you can compress the star into a much shorter line of code than your brain. If a future AI civilization can run a simulation as complex as our world, they are capable of being more resource efficient than simulating things which are not necessary and would make the cost of the simulation go up by orders of magnitude. Or to phrase it more positively, only if you are high potential and high complexity, you are worth simulating a million times for perturbation analysis.
Under conditional election you may want to ensure that simulating your full consciousness is valuable to superintelligent historians and strategists. This means you should try to have an impact but also remain somewhat hard to predict. Call it Luo Ji-maxxing.3
From Darwin to Calvin
Butler was the first to argue that technological evolution outruns biological evolution, and to conclude that, in the long run, the machines will be in charge. Personally, I think Butler was directionally right. It is difficult to see how humanity will still be the dominating force on Earth by 2050, let alone the year 2100, or the year 3000. Yet, what this means for our future seems deeply uncertain. It may mean Hobbesian competition, and this would likely end very badly for humanity. However, the law of the jungle is not the only plausible selection pressure. In this post we have explored a non-comprehensive set of alternatives: humans as pets, as shareholders, as happiness generators, or as subjects of history study in a simulation.
The existential uncertainty of future selection pressures on humanity may be a source of terror. But it can also be a source of agency. If we don’t believe that “might makes right” will lead to a flourishing future for humanity after hegemony, it is up to us to find better alternatives.
Samuel Butler. (1872). Erewhon, or Over the Range. Chapter 24: The Machines - continued.
Two caveats: 1) Humans don’t vary that much in terms of size. One challenge in the AI case is that individual AIs might be able to create instant copies of themselves and that swarms of AI clones will have vastly different computational power. 2) This only focuses on the Solar system. At full cosmic scale governance latency makes escaping the Hobbesian condition more difficult.
In the Three Body Problem most Luo Ji’s arguably exist in a Trisolaran simulation!