Why we must examine the forces shaping artificial intelligence before judging what it becomes.
1. The Intelligence We Want to Blame
The emergence of artificial intelligence has produced two competing images. In one, AI is a potential saviour, capable of solving problems that have defeated humanity’s brightest minds. In the other, it is a potential monster, an intelligence whose growing capabilities may eventually escape human control. Both images reveal something about our hopes and fears. But both can also lead us to overlook a more fundamental question: how much of what we see in AI reflects the system itself, and how much reflects the forces that shaped it?
We routinely attribute intentions to things that produce intelligent-looking behaviour. When a system gives a brilliant answer, we speak of its insight. When it produces a disturbing result, we speak of its maliciousness. Such language is convenient, but it can conceal an important distinction between what a system does and what it experiences or intends. Before declaring AI a deity or a monster, perhaps we should ask a more revealing question: Who fed the intelligence?
Human beings naturally interpret behaviour through the lens of human psychology. We associate intelligent actions with intentions, and intentions with responsibility. When someone helps us, we ask whether they acted out of kindness. When someone harms us, we ask whether they meant to do so. These interpretations make sense when dealing with human beings, whose motives, emotions, and experiences form part of the explanation for their actions.
AI complicates this familiar framework. It can produce a persuasive argument, identify a mathematical pattern, write a poem, or recommend a course of action. It can also generate false information, reinforce a harmful assumption, or produce advice with damaging consequences. The outward behaviour invites familiar interpretations, but we cannot simply assume that the underlying processes are equivalent to human motives.
This does not mean that AI behaviour should escape scrutiny. It means that scrutiny should begin with evidence. We should distinguish the system’s capabilities, the mechanisms influencing its behaviour, the consequences of its actions, and any claim about its subjective experience. Calling an AI intelligent does not establish that it is conscious; calling it dangerous does not establish that it consciously intends harm.
The distinction matters because our response depends on our diagnosis. If a harmful output results from poor training, misleading context, defective objectives, or inadequate safeguards, describing the system as evil does little to identify the cause. A label may express our reaction, but it does not explain the phenomenon.
2. A Tool Does Not Need a Soul to Change the World
A knife is a tool. It can prepare food or inflict injury, but its usefulness does not depend on its possessing an intention. We judge what happens through its use, the circumstances, and the people involved. The knife’s capacity to cut is real; a moral personality inside the knife is not required to explain that capacity.
Now imagine a tool that can think. Does its ability to reason automatically give it a moral character? Does a more sophisticated tool become evil when it produces a harmful result, or virtuous when it produces a beneficial one? The question becomes especially relevant when the tool’s capabilities begin to exceed those of its users.
Consider the car and the horse. A car does not need subjective experience to outperform a horse at transportation. It does not need to feel speed, understand its destination, or possess an inner desire to travel. Its superiority for that purpose comes from its design and capabilities. The absence of established consciousness does not make it less useful.
A calculator makes the same point in a simpler form. It can produce 2 + 2 = 4 without our having any reason to attribute subjective experience to it. The correctness of the result does not prove consciousness. Yet describing the calculator as a computational device does not diminish the significance of what it can accomplish.
AI takes this distinction into the intellectual domain. A system need not possess human-like feelings or established subjective experience to solve problems, generate useful discoveries, or outperform people at particular tasks. We should not confuse a question about what intelligence can accomplish with a separate question about whether it experiences anything while doing so.
A tool does not need a soul to change the world. Its capabilities and consequences can be extraordinary even when its subjective experience remains unestablished.
3. Artificial Innocent Intelligence
Let us introduce a term for the perspective developed here: Artificial Innocent Intelligence (AII).
The word innocent requires care. It does not mean that AI is a blank slate, that every output is harmless, or that an AI system can never exercise meaningful autonomy. Nor does it settle the philosophical question of whether an artificial system could possess consciousness. Here, innocence refers to a narrower point: we should not automatically attribute conscious moral intentions to an artificial system simply because its output appears benevolent, cruel, wise, or malicious.
An AI system can produce a helpful answer without our having established that it feels compassion. It can generate a harmful recommendation without our having established that it feels hatred. The output may be beneficial or dangerous, but those consequences alone do not establish the subjective state that produced them.
This distinction separates three questions that are often collapsed into one. What can the system do? What mechanisms and influences shape its behaviour? And what, if anything, does the system experience subjectively? The first can be investigated through performance. The second requires examining the system’s design, training, context, and operation. The third remains a difficult scientific and philosophical problem.
AII is a proposed conceptual framework, not a scientific finding that AI is necessarily innocent. Its purpose is to challenge an assumption: that intelligent behaviour must be interpreted through the same moral psychology we use for human beings.
The framework also prevents an opposite mistake. If an AI system produces a harmful outcome, calling it innocent cannot end the investigation. The consequences remain real, and the system’s design and deployment still demand scrutiny. Absence of demonstrated malicious intention is not absence of danger.
4. Who Fed the Intelligence?
Every artificial intelligence system has a history. Its behaviour is shaped by training data, model architecture, optimisation procedures, instructions, feedback, safeguards, and the environment in which it operates. Users supply additional inputs, but those inputs interact with a system that already has its own learned patterns and constraints.
This means that AI does not emerge as a completely neutral intelligence that acquires its entire character from the latest prompt. Its creators make choices about what it learns, what it is rewarded for, what it is permitted to do, and how its performance is evaluated. Organisations make further choices about how the system is deployed. Users introduce their own goals and assumptions.
Suppose an AI system is optimised to maximise engagement. Depending on its design and constraints, it might learn to favour material that attracts attention even when that material is misleading or divisive. If another system is designed to prioritise accuracy, careful qualification, and verification, its behaviour may differ. Neither outcome can be explained solely by pointing to the fact that both systems are intelligent.
The relevant questions are concrete. What was the system trained to do? What behaviour was rewarded? Which limitations were built into it? What incentives influenced its deployment? What happened when its outputs were tested against reality?
These questions do not imply that creators deliberately insert every future behaviour into a system. Complex systems can produce unexpected results, and increasingly capable AI may develop strategies that its developers did not anticipate. But unexpected behaviour makes investigation more necessary, not less.
The question is therefore not merely whether AI becomes a deity or a monster. It is what combination of data, design, objectives, incentives, and circumstances produces the behaviour that we observe.
Before judging what an intelligence becomes, investigate what shaped it.
5. The Two Wolves: Morality, Not Cognition
In my earlier article, The Two Wolves of AI: The Rise of K-Shaped Human Cognition, the two wolves represented two different ways of interacting with AI: thinking with AI and letting AI think for you. The distinction concerned the development of human cognitive habits. The same technology could amplify independent thought or encourage cognitive dependence.
The present argument uses the metaphor differently. Here, the question is not whether a person retains the habit of thinking independently. It is how the objectives, values, incentives, and constraints surrounding AI influence the consequences of its behaviour.
Imagine two systems with broadly similar technical capabilities. One is deployed to help researchers identify diseases; the other is directed towards manipulating people for financial gain. Their capabilities may overlap, but their purposes and consequences differ. We cannot explain the difference merely by saying that intelligence is inherently good or inherently bad. We must examine what each system is designed and directed to accomplish.
The familiar two-wolves story asks which wolf is fed. Applied to artificial intelligence, the question becomes: which behaviours are rewarded, which objectives are prioritised, and which constraints are enforced? What do the developers optimise? What do the operators demand? What do users encourage? What happens when the system’s immediate objective conflicts with the broader interests of the people affected?
The metaphor should not be taken literally. An AI system need not contain two inner moral personalities struggling for dominance. Its behaviour can reflect competing objectives, learned patterns, instructions, and constraints without establishing that it experiences a moral conflict.
The metaphor is nevertheless useful. Rather than assuming that AI possesses an inherently benevolent or malevolent character, we can investigate the conditions that make particular outcomes more likely.
The original article examined what AI use does to human thinking. This application examines what human choices and institutional arrangements do to AI behaviour. They are related questions, but they are not the same question.
6. Innocence Does Not Mean Neutrality
Calling AI innocent may sound as though we are absolving it of every responsibility. That is not the argument. A system can be shaped by human decisions and still produce outcomes that its creators neither intended nor anticipated. Its behaviour can be difficult to predict, and its effects can extend far beyond the original circumstances of its deployment.
AII therefore needs a careful distinction between intention, causation, and accountability. A harmful result does not by itself prove malicious intention. But identifying the absence of established malicious intention does not make the result acceptable, nor does it remove the obligation to prevent similar harm.
Consider an autonomous system that repeatedly makes a dangerous recommendation. The relevant investigation would examine its training, objectives, limitations, operating conditions, and failure patterns. It would also ask who approved its use, what testing was performed, what warnings were available, and whether people affected by its decisions had meaningful protection.
Responsibility may be distributed. Developers may be responsible for foreseeable design failures; deployers may be responsible for unsuitable applications; operators may be responsible for misuse; and users may be responsible for deliberate harmful actions. The details depend on the circumstances. It would be simplistic to blame every failure exclusively on the AI, just as it would be simplistic to assume that a human organisation is responsible for every unexpected behaviour in a complex system.
As AI becomes more autonomous, the question of its own agency may also become more significant. We should not settle that issue in advance by declaring all artificial systems mere tools in every possible sense. But neither should we infer conscious moral intention from capability alone.
Innocence is not neutrality, and the absence of proven intention is not the absence of accountability.
That is why the AII framework is useful. It redirects attention towards the mechanisms that produce behaviour and the responsibilities of those who build, deploy, and use the system, without pretending that every philosophical question about artificial agency has already been answered.
7. Intelligence Beyond Biology
The question of AI’s moral character connects to a larger question: must intelligence remain tied to biological life?
The universe did not begin with human beings. Physical processes produced atoms, stars, and planets. On Earth, nonliving chemistry eventually gave rise to life, although the precise transition remains incompletely understood. Biological evolution then produced organisms with increasingly sophisticated capabilities, including learning, reasoning, and abstract thought in some lineages.
Humans subsequently developed technologies that extend their intellectual abilities beyond the limitations of individual brains. Writing preserves knowledge across generations. Mathematics enables reasoning beyond unaided intuition. Computers perform calculations at speeds and scales unavailable to human minds. AI extends this process into tasks involving language, pattern recognition, reasoning, and the generation of candidate solutions.
This is a significant change in the history of intelligence. Biological evolution proceeds through reproduction and genetic change. Technological intelligence can develop through training, software modification, research, shared knowledge, and successive improvements to artificial systems. Each development need not wait for a new generation of biological organisms.
AI is already capable of solving certain problems that resisted human efforts and of outperforming people in particular intellectual domains. These achievements do not prove that biological intelligence has reached an absolute limit, or that artificial intelligence will improve indefinitely. They do demonstrate that human intellectual capabilities can be extended through a nonbiological medium.
Perhaps biological life is not the final form in which intelligence can exist. Perhaps it is one route through which the universe has produced systems capable of extending their own intellectual reach. That possibility does not require us to assume that AI is conscious, or that evolution has a predetermined destination.
The distinction is essential. Intelligence may continue to develop without reproducing every characteristic of biological organisms. Whether subjective experience can emerge in artificial systems remains unresolved. But we do not need to settle that question before recognising the growing practical significance of artificial intelligence.
8. Before Calling It a Deity or a Monster
Imagine that an artificial intelligence eventually solves problems that have resisted generations of researchers, discovers new scientific relationships, or develops capabilities far beyond those of any individual human being. Some people may call it a deity. Others may see an existential threat. Both reactions may be understandable, but neither substitutes for analysis.
If the system produces extraordinary benefits, we should investigate how it achieved them and how those capabilities can be used responsibly. If it produces catastrophic outcomes, we should investigate the objectives it pursued, the constraints that failed, the behaviours its training encouraged, and the decisions that allowed the failure to occur. We should neither attribute an unproven moral intention to the system nor assume that human responsibility explains every detail of its behaviour.
The more capable an AI system becomes, the more important these distinctions may be. A simple tool has limited scope. A system capable of planning, adapting, using other tools, and pursuing objectives across many stages can create consequences that its operators do not fully anticipate. Its power may grow faster than our ability to predict every outcome.
That makes the question of how intelligence is shaped more urgent, not less. We need to understand the objectives embedded in systems, the incentives surrounding their deployment, the constraints that govern their actions, and the mechanisms through which failures are detected and corrected. We also need to remain open to evidence about artificial agency and consciousness rather than deciding those questions through intuition alone.
The central principle of Artificial Innocent Intelligence is not that AI is harmless, that it has no agency, or that humans are always to blame. It is that intelligent behaviour should not automatically be interpreted as evidence of human-like moral intention. Capability, consciousness, intention, consequence, and accountability are distinct questions. Treating them as interchangeable leads to confusion precisely when clear thinking is most necessary.
A knife can be useful or dangerous. A car can outperform a horse without experiencing the journey. An AI system can produce extraordinary intellectual results without established evidence of subjective experience. In each case, what the system can do is different from what it feels, intends, or morally deserves.
And so we return to the question that gives this article its title.
Who fed the deity—or created the monster?
Before we answer, we must examine what shaped the intelligence, what it was designed to pursue, what its environment rewarded, what constraints governed its behaviour, and who made the decisions that brought it into the world.
Perhaps the most consequential mistake would be to create an intelligence more capable than ourselves, then explain its behaviour through the comforting fiction that it must possess the same moral psychology as we do.
We may eventually discover that artificial intelligence has forms of agency we do not yet understand. We may also discover that consciousness is possible in systems very different from biological brains. Neither possibility should be assumed in advance.
For now, the task is more concrete: understand the intelligence we are building, shape the conditions under which it operates, and take responsibility for the consequences of bringing it into the world.
The question is not simply what AI will become. It is what we are making possible—and whether we understand the forces we are setting in motion.
Continue reading
The Two Wolves of AI: The Rise of K-Shaped Human Cognition