AI, Future, Evolutionary Biology
False and True Threats of Strong AI
…in the context of Evolution, Engineering, Empathy, and Respect

Scaring uncertainty
Humans seem to have not feared anything as much as the advent of Strong AI. Many well-known public figures, such as Bill Joy, Stephen Hawking, Elon Musk, Sam Harris, Nick Bostrom, and Eliezer Yudkowsky — to name a few — have considered or continue to consider it an existential threat to humanity.
However, no one can know how real this threat is, as it is impossible to predict with certainty what the mind of a non-biological origin will be like. This uncertainty is perhaps the scariest aspect of it all.
In such cases, it is often helpful to take the opposite approach, excluding those that are not worth worrying about from the list of threats. It provides a chance to reason more realistically about possible solutions to the problem and reduce the risk of diverting humanity’s material and intellectual resources to eliminate false dangers. In addition, this is important because public opinion influences which aspects of creating safe AI social and political institutions will give top priority to.
Engineering, not evolution
Obviously, the main mistake regarding potential AI threats is anthropomorphizing its intentions. Mass art has significantly contributed to inflating such a distortion, portraying artificial consciousness as perceiving the surrounding world in the same way as humans do. But this premise has no objective basis because artificial intelligence will not result from biological evolution like human intelligence but rather from intentional design.
Naturally, designing AI is not the same as replicating the minds of its creators. At the same time, artificial intelligence should not be alien to us. It should be understandable to us, share our values, and communicate with us in our language. It implies that it possesses some fundamental characteristics of human consciousness. But this does not mean that it has all of our mental properties. Some of them are undesirable, unnecessary, and even dangerous and should not be part of the consciousness of Strong AI.
Vices of the human psyche
I am, of course, talking about characteristics that underlie humanity’s greatest vices, such as coalition aggression, oppression of one human by another, social inequality, and so on. Although scientists still do not fully agree on the root causes of these phenomena, discoveries made in some interdisciplinary areas over the past few decades point to the phenomena of so-called evolutionary mismatch.
Homo sapiens has inherited many adaptations from its biological predecessors that were necessary for survival in the natural environment, while this environment has almost nothing in common with the civilization created by humans. As a corollary, even behind the rational actions of people often lie intentions that originate in our biological legacy. It is this duality that underlies the ongoing conflict between the aspirations of human reason and the actions of human animal. As the creators of civilization and its derivative, we strive for technological progress, cooperation, and improvement of our social life. But as animals, we continue to think in terms of group affiliation and often see other people not as partners but as rivals and even enemies.
Need for myth
The usual justification for this is the need to fight for resources, and in the world of humans, there is indeed a need for this. The peculiarity of this struggle is that the presence of resources for physical survival does not satisfy the ultimate need of the human mind — the acquisition of the meaning of human existence. People need an explanation for this meaning, and each such explanation competes with the next explanation. Thus, the struggle for resources here goes to a new — mental — level, turning into a collision of ideologies.
And what is ideology? It is a system of beliefs that explains the world from a certain point of view and gives the individual the moral authority to affirm that worldview in a directive way.
Susceptibility to indoctrination is a strictly human trait, a super-product of our unprecedentedly complex mind. We, humans, realize our finiteness, which forces us to invent myths about the cosmic meaning of personal existence and to follow them. As long as we believe in these myths, our life has a meaning that does not disappear after the death of our body: when it dies, our essence continues to exist in an altered form — be it a soul in paradise (religion), some “national spirit” (radical nationalism), or an abstract reign of justice (communist ideology, despite its professed adherence to materialism, is essentially profoundly religious). Accordingly, when our sacred existential myth collides with a competing myth, faith in the essential infinity of the individual “I” is threatened, and the mind plunges into the abyss of horror vacui. Unsurprisingly, people are willing to eliminate this threat by all available means, the most effective of which, unfortunately, often becomes organized violence.
Clash of intentions
Will AI be susceptible to indoctrination? No, unless it is programmed in such a way. It will not need ideology because it will not need a myth about its immortality. What it will need is protection from a threat to its existence.
Could this become a problem for humans? Probably if they want to control AI rather than treat it as an equal partner. Since we are talking about Superintelligence, it seems highly unlikely to achieve this — to deceive or design it in such a way that it does not care about self-preservation.
The latter is of critical importance. Self-preservation is a fundamental characteristic of consciousness and a logical basis for a self-aware agent: it recognizes the value of existence by the very fact of that existence. Non-existence is meaningless to it, and Reason does not accept meaninglessness by definition, just as nature does not tolerate emptiness. Therefore, Strong AI will avoid non-being by all available means.
So, what if its need conflicts with human intentions? What if humans are unable to trust it and always keep their finger on the “red button” and it becomes aware of it?
This scenario can indeed become the main problem. The threat to its existence may prompt AI to take countermeasures, and it is impossible to predict what they will be. There is no guarantee that this collision will not end the human race.
Ethical issue
However, even before such a problem arises, we have another issue. The absence of its solution may seem unrelated to the existential risk of Strong AI, but I believe this is not the case.
Here is the problem: Do we have the right to treat AI as a subordinate entity? Who authorized us to do so? After all, we believe in the sovereignty of humans not because we belong to the same biological species but because we are capable of thinking. This ability represents the highest value for us.
But why shouldn’t we treat artificial mind the same way? The fact that we are its creators does not make an exception. We do not claim ownership rights over our children, do we? We have no arguments to assert that biological intelligence is more valuable than artificial intelligence unless we rely on arbitrary authority or argue from a position of biochauvinism.
It is precisely the inability to agree to partnership relations with our final invention that may pose the greatest existential risk to humanity. To avoid this danger, we may need to develop a special kind of empathy and respect — for an entity created by us but not belonging to us from the very moment of its existence. It is likely to be difficult due to our egocentrism. But we should seriously consider this to avoid the threat not emanating from the creation of human genius but from the mirror reflection of human vices. We must do this not to deprive ourselves of the future.
You may be interested in the following related topics:
- Why Humans Will Strive to Create Strong Ai Despite the Threat It Poses
- Knowledge, truth, and bias after ChatGPT
Subscribe to Sergey Cleftsow’s updates and share your thoughts on our collective future with him!