Question the Question: The Absent-Minded Professor Problem in AI

There is a famous story about Albert Einstein that captures a peculiar kind of absent-mindedness.

Einstein was travelling by train when the conductor came through checking tickets. Einstein searched his vest pocket. Nothing. He checked his trousers, his briefcase and the seat beside him. Still nothing.

The conductor recognized him and said, “Dr. Einstein, I know who you are. We all know who you are. Don’t worry about the ticket.”

Einstein nodded appreciatively.

The conductor continued down the carriage. Then, just before moving to the next coach, he looked back. Einstein was down on his hands and knees, searching under the seat.

The conductor hurried back. “Dr. Einstein, don’t worry. I know who you are. You don’t need the ticket.”

Einstein looked up and replied:

“I know who I am. What I don’t know is where I’m going.”

“I need the ticket for that!”

It is a wonderful joke about an absent-minded genius. But there is a deeper idea hidden inside it.

The problem was not Einstein’s ability to think. The problem was what his attention was currently attached to.

A man capable of thinking about the deepest questions in physics could temporarily lose track of something as mundane as a train ticket—and yet that mundane detail suddenly mattered enormously.

That is the Absent-Minded Professor problem.

And it may be a surprisingly useful way to think about AI.

An AI system can perform sophisticated mathematics, write complex code and sustain a long chain of reasoning—and still overlook something elementary because its reasoning is operating at the wrong level.

The question is not always whether the system can reason. Sometimes the question is whether it noticed what it was supposed to be reasoning about.

1. The First Blind Spot: Accepting the Frame

Consider the baseball-cap puzzle used by SilentRoom. A buyer supposedly purchases a baseball cap for $10 and pays with a counterfeit $10 bill. The seller obtains change from a neighbor, gives the buyer the cap and $4 change, and later repays the neighbor $10 when the counterfeit bill is discovered.

At first glance, it looks like a familiar counterfeit-money calculation. But the first thing a careful reader should notice is much simpler: the cap costs $10, the buyer pays $10, so why is the seller giving $4 change?

That is not an arithmetic question. It is a question about the question itself.

2. The Counterfeit Puzzle Is Itself Counterfeit

The joke almost writes itself. The puzzle is presented as a counterfeit-bill puzzle, but the bill is not the only counterfeit element. The problem statement itself is defective.

It looks like a legitimate mathematical problem because it contains all the familiar signals: a transaction, a counterfeit bill, a neighbor, some change, a repayment and a question asking for a total. Those signals encourage the solver to start calculating before examining the transaction.

The natural response should therefore be: “Why would he go to the neighbor for change?”

3. Humans Have Been Falling for Blind-Spot Puzzles for Years

The phenomenon is hardly unique to AI. Humans have a long history of falling for questions that exploit intuitive shortcuts.

Take the famous bat-and-ball problem. A bat and a ball cost $1.10 together, and the bat costs $1 more than the ball. Most people immediately say that the ball costs 10 cents. The correct answer is 5 cents.

The important point is that this is a valid question. The trap lies in the solver’s intuition. The counterfeit-cap puzzle is different: there the solver can be trapped by accepting a questionable frame before doing any mathematics.

4. The Missing-Dollar Hotel Puzzle

Another classic is the three-friends hotel puzzle. Three friends pay $75. The hotel later reduces the bill to $70. The clerk sends back $5, but the bellboy keeps $2 and gives each friend $1.

Each friend therefore ends up paying $24, making a total of $72. The familiar question then asks: if they paid $72 and the bellboy kept $2, where did the missing dollar go?

There is no missing dollar. The $72 already consists of the $70 paid to the hotel plus the $2 kept by the bellboy.

5. Other Classic Blind-Spot Traps

The same pattern appears in many familiar puzzles. Ask where survivors of a plane crash should be buried, and the mind may immediately begin thinking about geography and international borders. The word “survivors” should have stopped the reasoning before it began.

Ask, “Mary’s father has five daughters: Nana, Nene, Nini, Nono and…?” and many people begin looking for the missing vowel. Yet the answer, Mary, was supplied in the first word.

These puzzles work because the brain is efficient. Once it recognizes a familiar pattern, it does not normally inspect every assumption from first principles.

6. The Wolf, Goat and Cabbage Problem

This is where SilentRoom’s earlier examples become particularly interesting. The original Wolf, Goat and Cabbage puzzle is familiar enough that a model can retrieve a standard solution almost automatically.

SilentRoom describes modified versions in which important conditions are changed or removed, while some models continue applying elements of the original solution.

The blind spot here is different from the bat-and-ball problem. The model is not simply making a quick numerical mistake. It is recognizing the template and continuing to solve the template instead of the actual problem.

7. Gowers’ Chickens and the Problem of Overthinking

SilentRoom also discusses Timothy Gowers’ experiment involving a simplified river-crossing problem with two chickens. The problem was reduced enough that it required very little elaborate strategy, yet ChatGPT produced an unnecessarily complicated solution.

There is another lesson here. A reasoning system can possess enormous computational resources and still fail to ask whether all that machinery is necessary. Humans do this too. We sometimes solve a much harder problem than the one that was actually asked.

The problem is therefore not simply insufficient reasoning. It can also be misallocated reasoning.

8. What Does “Common Sense” Actually Mean?

Calling these failures a lack of common sense is tempting, but the phrase covers several different abilities.

Sometimes common sense means recognizing that the conditions of a familiar puzzle have changed. Sometimes it means avoiding double-counting. Sometimes it means noticing that survivors are not buried. And sometimes it means asking why a seller is giving change after receiving the exact selling price.

These are related capabilities, but they are not identical. A more useful description would be premise validation, frame checking and context-sensitive reasoning.

In ordinary language, the principle is much simpler: before solving the problem, check what problem you have actually been given.

9. The Ferrari Test

Consider a deliberately silly example. Someone looks at a very fast Ferrari and asks whether it can move forward while its transmission remains in reverse gear.

A system could answer the question literally by discussing engine power, torque and transmission mechanics. A more sensible response would first challenge the operating condition: change the gear first, stupid human.

The Ferrari does not need more horsepower. It needs the right gear.

The same principle applies to AI reasoning. More tokens and more computation can help when the problem is genuinely difficult. They do not automatically help when the system has accepted the wrong frame.

10. For a Trivial Puzzle, Nobody Wants to Spend $10 Worth of Thought

There is another reason these traps work so well. If someone gives a human a $10 puzzle, the human is unlikely to spend ten minutes formally auditing every assumption. The problem looks trivial, so we use fast thinking.

We recognize the genre, recognize the familiar transaction and begin calculating. That is a perfectly sensible allocation of attention under ordinary circumstances.

A blind-spot puzzle exploits precisely that instinct:

simple-looking problem → low scrutiny → fast reasoning.

The Domain-Search Problem

There is a deeper issue hiding here. When a problem looks trivial, the search domain itself may become small. A $10 puzzle looks like a small arithmetic problem, so a system may search a narrow neighborhood of arithmetic and transaction patterns. Unless something signals that the question is adversarial, malformed or deliberately designed as a test, there may be little reason to activate a much broader search for hidden assumptions.

Tell the system, “I am testing you,” and the situation changes. It now has a reason to consider traps, inconsistencies and alternative interpretations.

The test has therefore changed the reasoning process before the reasoning has even begun.

This suggests that some apparent reasoning failures are actually problem-classification failures: the system did not search the right domain because the surface form of the question told it that such a search was unnecessary.

The reasoning pipeline is therefore deeper than simply:

Question → Reason → Answer

A more robust pipeline is:

Question → Classify the Problem → Choose the Search Domain → Reason → Verify

11. Designed to Exploit Weakness Is Not the Same as Testing Strength

SilentRoom’s experiments are useful precisely because they deliberately search for situations in which models behave badly. That is a legitimate way to expose weaknesses.

But finding a construction that makes a system fail does not, by itself, establish that the system is generally stupid. It establishes that the system has a vulnerability to that construction.

The constructive question is therefore not simply, “Did the model fail?” It is, “What exactly caused the failure, and what capability would prevent it?”

12. Question the Question

This leads to a simple reasoning protocol:

Q → Question Q → Reason → Question A → A

The first stage asks whether the question is internally coherent. The second examines hidden assumptions and the frame being supplied. The third performs the actual reasoning. The final stage asks whether the resulting answer survives an independent sanity check.

The usual model of problem solving is Q → A. A more robust model is: Question the Question → Solve → Question the Answer.

The first question an intelligent system should sometimes ask is therefore not “What is the answer?” but “Is this actually a valid question?”

13. The Absent-Minded Professor Problem in AI

The absent-minded professor is not unintelligent. He is intelligent inside the wrong frame.

That may be one of the most useful metaphors for understanding certain AI failures. A system can perform sophisticated mathematics, write complex code and sustain long chains of reasoning while still overlooking an elementary inconsistency because the inconsistency appears before the reasoning process it has chosen to perform.

The solution is not necessarily to make the professor think harder. It is to make him occasionally look up from the blackboard.

That is the deeper lesson behind these silly puzzles. AI does not merely need the ability to reason. It needs the ability to decide when the reasoning should begin, what assumptions it should operate under, and whether the result makes sense when it returns to the real-world situation.

The challenge is therefore not simply whether AI can solve the puzzle. It is whether AI can notice what puzzle it has actually been given.

Before answering the question, inspect the question. Before trusting the answer, inspect the answer. And before declaring a machine stupid because it fell into a carefully constructed trap, ask what the trap actually measured.

That is how a puzzle becomes more than a puzzle. It becomes a test of how an intelligent system decides what deserves to be reasoned about in the first place.

Question the Question.

Further reading: SilentRoom’s “The Wolf, Goat and Cabbage Problem”.