AI is strongest when the problem is already sufficiently represented. The real world is harder when the information needed to solve the problem has not yet been discovered.
1. The Strange Advantage AI Has in Mathematics
Mathematics gives AI an unusually favourable environment. A mathematical problem can often be specified before solving begins. Its variables can be defined, its assumptions stated, its rules made explicit, and its result independently checked.
If we ask an AI to solve an equation, the information needed to solve it is contained in the equation. The machine does not have to leave the problem, interact with the physical world and discover another piece of information before it can proceed.
That does not make mathematics trivial. Some mathematical problems are extraordinarily difficult. Research mathematics can involve discovering new representations, finding unexpected connections and deciding which questions are worth asking in the first place.
But once a problem is sufficiently formalised, AI has a major structural advantage: the problem has already been converted into information that can be processed.
This is very different from many real-world situations.
2. The Real World Does Not Hand You the Problem
Consider starting a business. You might have market research, financial data, customer surveys and years of historical information. Yet none of this tells you exactly what will happen after you launch.
Which customer segment will respond? Which competitor will change its strategy? Will customers actually behave as they said they would? Will a seemingly insignificant feature determine whether people buy? Will a new regulation, technology or social trend change the market six months later?
There is no complete problem statement sitting on the table waiting to be solved. The entrepreneur has to discover the problem while trying to solve it.
This is a fundamentally different situation from solving a well-defined mathematical question. The difficulty is not necessarily that the calculation is harder. The difficulty is that some of the information required for the calculation does not yet exist in usable form.
3. Information-Incomplete Problems
This suggests a useful distinction. An information-complete problem is one in which the important variables, constraints, relationships and rules are sufficiently available before the solution process begins.
An information-incomplete problem contains important unknowns. Some information may be missing. Some may be ambiguous. Some may change over time. Some may exist only in another person’s behaviour. And some can be obtained only by interacting with the environment.
That last category is particularly important.
Suppose you want to know whether customers will buy a product. You can analyse demographics, study competitors, examine historical purchasing patterns and ask potential customers what they think. But eventually, you may have to put the product in front of people and observe what happens.
The action itself generates information. The problem was not merely waiting to be solved. It was waiting to be revealed.
4. Action Is Also an Information-Gathering Process
This changes the traditional picture of problem-solving. We often imagine intelligence as a sequence:
Information → reasoning → answer
But many real-world problems look more like:
Observe → hypothesise → act → receive new information → update → act again
The action is not merely the final step. It is part of the investigation.
A scientist runs an experiment because the information required to answer the question is not yet available. An entrepreneur launches a product because market behaviour cannot be completely known beforehand. An engineer tests a machine because simulations and specifications cannot capture every condition. A negotiator makes an offer partly to discover what the other side is willing to accept.
In all these cases, the world becomes part of the computation.
This may be one of the most important differences between processing information and operating in reality.
5. Why a Vending Machine Can Be Harder Than a Mathematical Proof
Consider something as apparently simple as operating a vending machine. Compared with advanced mathematics, it seems almost embarrassingly easy.
But put an intelligent system in front of a real vending machine and the problem suddenly expands. What if the payment fails? What if the product is stuck? What if the customer inserts the wrong amount? What if the machine reports an error incorrectly? What if the temperature changes? What if someone shakes the machine? What if the customer’s behaviour is unexpected?
None of these problems necessarily requires sophisticated mathematics. The difficulty comes from the fact that the complete state of the world was never supplied in advance.
This gives us an important distinction:
Complexity ≠ Incompleteness
A problem can be mathematically enormous and still be completely specified. Another problem can be extremely simple but impossible to fully specify beforehand because the missing information is sitting outside the problem—in the physical world, in another person’s behaviour, or in events that have not happened yet.
6. The Problem Changes When You Act
There is an even deeper complication. The world does not merely contain unknown information. The world reacts to what you do.
A business enters a market and competitors respond. A company changes its pricing and customers change their behaviour. A government introduces a rule and businesses adapt. A person makes an offer and the other person changes their position. An autonomous machine moves through an environment and the environment changes around it.
The agent is therefore not solving a fixed puzzle. It is participating in a system.
This creates a feedback loop:
Your model → your action → the world’s response → new information → revised model → new action
The original problem may therefore cease to be the same problem a few minutes later. That is fundamentally different from a static question whose answer remains unchanged while you calculate it.
7. The Boundary of AI May Be Information, Not Intelligence
This leads to a more interesting way of thinking about AI. The question is often framed as:
How intelligent will AI become?
But there is another question hiding underneath it:
What information can the AI actually obtain?
Suppose an AI becomes vastly better at reasoning. It can analyse millions of possibilities, write sophisticated plans, simulate outcomes and combine enormous quantities of knowledge. There may still be situations where the critical variable simply isn’t available.
The customer has not yet reacted. The competitor has not yet moved. The machine has not yet failed. The experiment has not yet been performed. The physical environment has not yet revealed what happens.
No amount of reasoning can substitute perfectly for information that has not yet been acquired. The bottleneck has shifted from computation to information acquisition.
That is a very different limitation.
8. The Human Advantage May Move Into the Loop
This does not mean that humans will remain uniquely capable of every information-incomplete task. AI systems can increasingly interact with tools, sensors, software, robots and environments. The boundary will therefore move as machines acquire better ways of obtaining information.
But it does suggest a useful way to think about the future.
Human importance may increasingly lie not in possessing more stored information than AI, nor necessarily in performing calculations faster. It may lie in participating in the observe–act–learn loop.
A human enters an uncertain environment. They notice something unexpected. They decide what to investigate. They act. The environment responds. They update their understanding. And the problem itself becomes clearer.
Perhaps the future contest is therefore not simply:
AI intelligence versus human intelligence.
A more interesting question is:
Who—or what—can enter the world, act, obtain new information and continuously redefine the problem?
That may be one of the places where the difference between knowing an answer and discovering what needs to be answered becomes crucial.
AI Superintelligence Constraint Problems