In the first two parts of our mini-series, we explored ideas that once seemed futuristic. Setun showed that even binary logic was not the only possible path for the development of computing, while OGAS attempted to go much further and treat an entire national economy as a cybernetic system that could be measured, modeled, and optimized.
Today, we return to the future. Not the cinematic one, but our own present.
Fiber-optic networks reach our homes, large data centers process unimaginable quantities of information, cloud infrastructure connects companies on a global scale, IoT devices measure almost everything that can be turned into a signal, while ERP and MES systems connect production, logistics, finance, and maintenance. Artificial intelligence models analyze data, recognize patterns, and increasingly provide recommendations or make decisions.
Much of what cyberneticists in the middle of the twentieth century could only imagine is now technically feasible. But as those possibilities become real, another question appears — one older than the Internet, artificial intelligence, and even cybernetics itself:
Is an optimal society necessarily a good society?
🚀 Back to the Future
The Soviet Union was created as a project that openly proclaimed the construction of a new society based on rational, scientific, and planned principles. That spirit can be felt particularly strongly in Soviet cybernetics from the late 1950s and throughout the 1960s.
If a problem exists, it should be defined. If a system has weaknesses, they should be measured. If we have a sufficiently good model, we can calculate a better solution.
This is a very natural way of thinking for an engineer. In a technical system, we observe inputs, outputs, constraints, delays, and disturbances, and then try to find a control structure that will enable stable, efficient, and predictable operation.
But things become far more complex when the object of control is no longer a machine.
What if the object is society?
📖 Yevgeny Zamyatin: A Revolutionary Who Never Stopped Doubting
Long before Setun, OGAS, and Soviet cybernetics, one Russian engineer tried to imagine what a society would look like if rationality, order, and mathematical precision were brought almost to perfection.
His name was Yevgeny Zamyatin.
By education, he was a naval engineer. Even before the revolution, he had joined the Bolsheviks and was persecuted by the Tsarist regime for his political activity. He was not in Russia when the revolution of 1917 took place; he was working in Great Britain on shipbuilding projects and returned to Russia afterward.
This gave him an unusual perspective. He was not a man longing for the old order, because he had fought against that order himself. But after the revolution, he also refused to give up the right to critically observe the system that was being created.
It was from this position that his novel We emerged.
Today, it is often presented primarily as a precursor to later dystopias, especially Huxley’s Brave New World and Orwell’s 1984. That is true, but insufficient. We is not merely a novel about a repressive state. It is a much deeper study of the relationship between freedom, happiness, rationality, and human nature.
🧮 A Mathematically Perfect Society
The novel’s main character is D-503. He is not a revolutionary, an artist, or a philosopher, but an engineer working on the construction of the Integral, a spacecraft with which the One State intends to carry its mathematically ordered happiness to other worlds.
The idea is simple: if rational order is superior to chaos, then it is morally right to spread it.
Within that logic, freedom becomes a problem because it introduces uncertainty, error, conflict, and unplanned behavior. In one of the novel’s most striking ideas, the biblical story of Adam and Eve receives an entirely different interpretation. Humanity was given a choice between happiness without freedom and freedom without happiness, and humanity chose freedom.
For the One State, that was the first great mistake.
🏙️ Happiness Without Privacy
The citizens of Zamyatin’s state live in glass dwellings, while their everyday lives are organized according to a precise schedule. Work, rest, walks, and even intimate relations have their designated time.
What is especially disturbing is that D-503 does not primarily experience constant surveillance as violence. On the contrary, he experiences it as security and almost as the parental care of the system.
This may be one of the deepest elements of the novel.
The problem is not only that a human being lives under surveillance, but the moment when he begins to want surveillance, because it gives him a sense of safety. At that point, the system no longer wins only by force. It wins when security becomes more attractive than freedom.
❤️ I-330 as an Error in the Algorithm
D-503 has an orderly life. He has work, a partner, a schedule, and even time reserved for himself, during which he writes his diary.
Then I-330 appears.
She does not belong to his world in the way she is supposed to. She is unpredictable, breaks the rules, provokes him, and attracts him. Gradually, something enters his perfectly organized internal system that cannot easily be calculated.
D-503 begins to experience what an engineer might call a disturbance, but Zamyatin gives that disturbance another name: imagination.
Imagination is precisely what the system ultimately tries to remove. When the conspiracy against the One State fails, D-503 undergoes an operation that destroys his capacity for imagination. Afterward, he becomes a perfectly stable citizen. He no longer has inner conflict, doubt, or rebellion. He betrays I-330, and the system has finally “repaired” him.
But then the question imposes itself:
Is he still the same human being after the repair?
⚙️ Why Do We Love Algorithms So Much?
The mathematical perfection Zamyatin wrote about now has another name.
We call it the algorithm.
Our desire to solve problems precisely, reliably, and repeatedly lies at the foundation of almost every machine, system, and program we have created. And there is nothing wrong with that. On the contrary, algorithms reduce errors, automation increases safety, predictive models can prevent failures, navigation systems save time, medical models can help with diagnosis, and financial systems can assess risk more effectively.
At the level of a technical system, predictability is almost always desirable.
The problem begins when we transfer that logic to human beings without reservation.
📡 The Great Infrastructure of Predictability
The modern world is building something that would have looked almost unbelievable to cyberneticists of the 1960s.
Billions of devices generate data, cameras monitor public spaces, mobile phones leave digital traces, ERP systems record business processes, MES systems track production almost in real time, while IoT sensors record the state of machines, traffic, energy, and the environment.
Social networks register our interests, reactions, and habits. Banking systems know patterns of consumption, while online commerce knows what we search for, what we buy, and what we decided not to buy. Above all of this are data centers and artificial intelligence models capable of correlating quantities of information that no human being could process alone.
The consequences are often highly beneficial. Fraud is easier to detect, banks can assess credit risk more effectively, companies can plan inventories more precisely, traffic can be optimized, industrial equipment can be maintained before failure, and investment risk can be assessed more reliably.
Life becomes more predictable, and predictability brings a sense of security.
👁️ From Optimizing Processes to Optimizing Human Beings
Yet there is a boundary that is not always easy to see.
Optimizing the operation of a machine is one thing. Optimizing human behavior is something else entirely.
If an algorithm predicts when a pump will fail, it is difficult to find an ethical problem. If it estimates how likely a person is to repay a loan, things become more complicated. If it determines what information we will see, what advertisement we will receive, what price a system will offer us, how “risky” we are, how productive we are, or how desirable we are as a job candidate, then we are no longer speaking only about technical optimization.
We are speaking about modeling human beings.
And once the model becomes good enough, the next step is no longer only prediction.
It is influence.
🏰 Is Techno-Feudalism Emerging?
The power created by this infrastructure is not distributed evenly.
A large share of digital infrastructure is controlled by a limited number of companies that own cloud platforms, data centers, operating systems, social networks, digital marketplaces, advertising networks, and key communication platforms.
Some contemporary authors use the term techno-feudalism for this type of relationship. The term is controversial and is not a universally accepted economic description, but the question it raises is not trivial.
Even a traditional company that owns factories, employees, capital, and products often has to pay for access to digital platforms in order to reach markets, customers, or infrastructure. An increasing share of the economy therefore begins to depend on ownership of digital space.
Something similar applies to states. A small number of countries control key technologies, production processes, semiconductor supply chains, cloud infrastructure, and intellectual property. Technological standards are therefore no longer merely technical questions.
They become part of geopolitics.
🔐 Surveillance Is Not the Only Problem
It is easiest to imagine dystopia as a state monitoring its citizens or a corporation abusing their data. Those risks are real, but perhaps there is a deeper problem.
We ourselves like systems that make decisions for us.
Navigation chooses the route, a streaming service suggests the next film, a social-media algorithm chooses what we will see, an online store predicts what we should buy, and AI proposes what we should write, how we should respond, what we should analyze, and which decision we should consider.
Each individual decision seems harmless, often even useful. Precisely for that reason, the loss of autonomy may not happen through one great political upheaval.
It may happen through thousands of small acts of delegation, not because someone forced us, but because it is easier.
🤖 An Economy Run by AI
Let us imagine going only a few steps further.
A future economy possesses an almost perfect digital infrastructure. Sensors track physical processes, ERP and MES systems report the state of production, financial platforms follow flows of capital, logistics networks track goods, while satellites, cameras, and meteorological systems observe the physical environment.
AI models analyze all of this data in real time and propose optimal decisions.
How much steel should be produced? Where should we invest? Which power plant should be brought online? How much energy should be stored? Which traffic flows should be changed? Where should taxes be raised, and where should costs be reduced?
Technically speaking, this is a fascinating optimization problem.
But the most important question immediately appears:
What are we optimizing?
🎯 The Objective Function Problem
Every optimization requires a goal.
We can try to maximize economic growth, minimize energy consumption, reduce inequality, increase profit or employment, preserve natural resources, and improve long-term stability. We can try to do all of these at once.
Then we obtain a problem with a large number of variables and constraints. In highly simplified form, we can imagine a function:
J = w₁x₁ + w₂x₂ + … + wₙxₙ
in which each variable represents a goal and each weighting factor represents its importance.
A computer can search for the minimum or maximum of such a function. But there is a problem that not even the most powerful AI can solve on its own.
Who determines the weighting factors?
How important is growth compared with equality? How much is today’s standard of living worth compared with long-term sustainability? How much privacy are we prepared to sacrifice for security, and how much freedom for predictability?
Mathematics can optimize the values we have chosen.
But it cannot decide by itself which values we ought to choose.
Every objective function is, in the end, a system of values translated into mathematics.
🌐 The Digital Twin of the World
Industry 4.0 introduced the idea of the digital twin.
We create a model of a machine, then of a production line, a factory, a logistics system, a city, or an energy network.
Why not an economy?
And one day, perhaps, even a society?
If a digital model becomes sufficiently detailed, we can simulate the consequences of our decisions before implementing them. That could become one of the greatest technological advantages in history.
But a digital twin is never reality itself.
It is a model of reality.
And every model decides what it considers important. It includes some variables, ignores others, gives some relationships great weight and others little. The more complex the model becomes, the easier it is to forget that behind its mathematical elegance there are still assumptions that someone had to choose.
🌀 Chaos Is Not Just a Failure of the System
There is another problem.
Even the physical world is not always as predictable as we would like it to be.
Nonlinear dynamical systems can be completely deterministic and yet extremely difficult to predict over longer time scales because of their sensitivity to initial conditions. A small difference in the initial state can eventually lead to an entirely different outcome.
For that reason, chaos should not be confused with the absence of laws. A system may have precise equations and still be practically unpredictable.
If that is true of physical systems, what should we expect from a system involving millions of people, each with their own goals, memories, emotions, and the ability to change behavior precisely because they have learned that someone is trying to predict them?
Perhaps a complete digital twin of society will never exist.
And perhaps the more important question is:
Do we really want one to exist?
❌ Why Do We Need Mistakes?
From an engineering perspective, an error is something we try to reduce.
But human life is not a control cabinet.
Some of our wrong decisions become the beginning of new ideas. Choosing the wrong profession may lead to the discovery of a true calling, a failed project may open the way to a better one, and an unplanned encounter may change an entire life.
Scientific discoveries sometimes begin with an anomaly that could easily have been dismissed as a measurement error. Evolution itself operates through variation, selection, and historical contingency, without a pre-calculated path toward perfection.
Perhaps, then, the ability to make a mistake is not merely a flaw that a future algorithm should eliminate.
Perhaps it is part of our freedom and our creativity.
🧠 The Operation Against Imagination
Here we return to D-503.
The One State found the perfect technical solution to his problem. His instability was not caused by a malfunction of a machine, but by imagination.
Imagination creates alternatives. It allows a human being to imagine the world differently from the way it is, and in doing so it automatically creates the possibility of dissatisfaction with the existing state of things.
From the perspective of a perfectly stable system, imagination truly does look like a defect.
Zamyatin therefore goes all the way. D-503 becomes “happy” only when he is deprived of the ability to imagine another possibility.
The mathematical problem is solved.
The human being is lost.
🔄 From Setun to Artificial Intelligence
At the beginning of our triptych, we looked at a computer that attempted to calculate using three states instead of two. Setun reminded us that technological history is not predetermined.
Then we saw OGAS. Glushkov tried to combine computer networks, databases, and mathematical models into a system capable of managing a vast economy more effectively.
Today, we no longer face the same shortage of computing power that constrained them. We have fiber-optic networks, cloud infrastructure, IoT, ERP, MES, digital twins, and large artificial intelligence models.
The technical barriers are slowly disappearing.
And that is why the question becomes more serious.
Not only:
Can we?
But:
Should we?
🌌 Freedom as a Non-Optimal Solution
Perhaps freedom really is not optimal.
Perhaps it is inefficient and produces wrong decisions, risk, conflict, economic losses, injustice, and uncertainty. Perhaps a sufficiently powerful algorithm could find an arrangement in which we would live longer, safer, healthier, and more materially stable lives.
Perhaps it could even find a mathematical sweet spot in a space containing thousands of variables.
But more than a century ago, Zamyatin warned us that there is another variable here, one that is extremely difficult to place into an equation.
Human beings want the possibility of choosing wrongly.
To change a decision, to reconsider, to take risks, and to dream of something the model did not predict.
Perhaps freedom is not optimal. Experience teaches us that it is often inefficient, unpredictable, and costly.
But despite all of that, human beings need it. Not in a wild and unlimited form, of course.
And perhaps the greatest challenge for the cybernetics of the future will not be to create a system that can govern human beings perfectly, but a system intelligent enough to know when it should leave room for human free choice — even when that choice is not optimal.
Leave a Reply