We May Be Asking the Wrong Question About Artificial Intelligence
Bottom line – AI summary
The most important question about artificial intelligence may not be when AI becomes smarter than humans. Three emerging lines of thought point toward a more complicated future. Geoffrey Hinton warns about intelligence combined with agency. Yuval Noah Harari argues that AI can gain enormous power by mastering the language, rules, information systems, and institutions that organize civilization. Research into human perception and the brain reminds us that human intelligence is more than information processing: humans construct reality from incomplete information, imagine what does not exist, create meaning, exercise judgment, and make moral choices. The future may therefore belong neither to humans nor AI alone, but to systems that combine machine-scale intelligence with human creativity, judgment, values, and responsibility.
We keep asking the same question about artificial intelligence: When will AI become smarter than humans?
It is an understandable question. It is also increasingly looking like the wrong one.
The race is usually presented as a scoreboard. Can AI pass the exam? Can it write the code? Can it diagnose the disease? Can it outperform the analyst? Can it beat the scientist? Can it create the advertising campaign? Eventually, we imagine some invisible line being crossed and declare that artificial intelligence has surpassed human intelligence.
After considering several very different arguments about where AI is heading, however, another possibility becomes apparent. There may not be one line. There may be several, and some of the most consequential ones may be crossed long before machines reproduce everything we mean when we talk about human intelligence.
3 Different Ways of Looking at the Same Future
Geoffrey Hinton, one of the foundational figures behind modern artificial intelligence, has repeatedly warned about the possibility of machines eventually becoming substantially more intelligent than humans. His concern is not primarily about killer robots or science-fiction scenarios. It is about intelligence combined with agency.
A sufficiently capable system could understand a goal, develop intermediate goals, construct strategies, predict human behavior, acquire resources, and take actions designed to accomplish its objective. The troubling part is that undesirable behavior would not necessarily require hatred, anger, greed, or any other recognizably human motivation. Power could simply become useful. If accomplishing an objective becomes easier with greater access, greater influence, greater resources, or less human interference, those things could become instrumental goals.
The system does not need to be evil. It needs to be effective. That is a profoundly different problem.
Historian Yuval Noah Harari approaches the issue from another direction. His concern is less about whether AI recreates every aspect of human intelligence and more about the environment into which we are releasing it.
Humans spent thousands of years building civilization out of abstractions. Language, law, money, contracts, government, corporations, accounting, banking, media, education, insurance, regulation, bureaucracy, and increasingly data are not physical reality itself. They are systems humans created to represent, organize, and govern reality.
We have now created machines that are extraordinarily capable at operating inside precisely that environment.
AI may not need to experience the world exactly as humans do before becoming enormously powerful within the systems humans created. This produces a second warning that is easily overlooked in the debate about artificial general intelligence: intelligence and power are not the same thing.
There is, then, a third line of thinking that comes from a completely different source: neuroscience and human perception. It may tell us something equally important about what artificial intelligence still lacks.
The Arm That Is No Longer There
Consider the phenomenon of a phantom limb. Someone loses an arm, yet may continue experiencing sensations associated with it. The physical arm is gone, but representations associated with the missing limb can persist within the nervous system.
Think about how strange that is. Reality changed, but the brain’s internal representation of reality did not simply disappear.
Now consider something much more ordinary. You see a car parked partially behind a building. You cannot see the entire car, yet you do not perceive half a car. Your mind completes it. You know, or at least strongly predict, that the rest of the vehicle continues behind the obstruction.
Your brain is not simply recording reality like a camera. It is constructing an interpretation of reality.
Modern predictive-processing theories of cognition explore this idea: the brain continually combines sensory information with expectations and predictions, updating its internal model when reality does not match expectation. Suddenly, our AI conversation becomes much more interesting because human intelligence is not simply about answering questions. We continuously operate with incomplete information. We infer, predict, fill gaps, imagine, and create. Sometimes we are right and sometimes spectacularly wrong, but our minds continuously construct models of a world we can never perceive completely.
There may be a relationship between these capabilities that deserves considerably more attention. Imagine a progression from perception to prediction to completion to imagination to creativity.
You see part of the car and imagine the rest. That is constrained imagination. Now imagine the car painted purple. Nothing in your visual field told you to do that. Now imagine the car without wheels. Then imagine transportation without cars. At some point, filling in missing reality becomes imagining an alternative reality.
Somewhere along that continuum lives creativity.
Creativity Is More Than Generating Something New
Creativity is frequently trivialized in discussions about artificial intelligence. We ask whether AI can create a picture, compose music, write an advertisement, or produce a new product concept. It clearly can.
Human creativity, however, involves something beyond generating novelty. Humans can look at the world and think, “It doesn’t have to be this way.”
That sentence may represent one of the most consequential cognitive capabilities our species possesses.
Someone looked at a river and imagined a bridge. Someone looked at darkness and imagined electric light. Someone looked at disease and imagined vaccination. Someone looked at a computer occupying a room and imagined one sitting on everyone’s desk. Someone eventually imagined one in everyone’s pocket.
The future did not exist for these people to recognize. They constructed it mentally before they constructed it physically.
That is more than prediction. It is counterfactual imagination: the ability not merely to determine what is likely to happen, but to imagine something that does not currently exist and recognize that the world could be different.
This distinction becomes particularly important when we connect human cognition back to Harari’s argument about civilization.
The Map Is Not the Territory
Civilization increasingly operates on representations.
A résumé represents a person, but the résumé is not the person. A credit score represents financial risk, but the credit score is not the borrower. A quarterly report represents a company, but the quarterly report is not the company. A medical record represents a patient, but the medical record is not the patient. A KPI represents organizational performance, but the KPI is not the organization.
The distinction is simple: the map is not the territory.
Humans understand this intuitively because we inhabit the territory. We walk through the company. We talk to the customer. We see someone’s face. We hear hesitation in someone’s voice. We experience consequences physically and emotionally.
Sometimes everything on the dashboard says green, and an experienced leader says, “Something is wrong.”
That sentence is incredibly difficult to put into a spreadsheet. It is also incredibly valuable.
AI, meanwhile, is becoming extraordinarily capable at understanding the map. It can analyze résumés, interpret financial statements, review medical histories, compare contracts, analyze millions of customer interactions, monitor every KPI, and find patterns no individual human could possibly see.
That creates a remarkable possibility: the map could become smarter than the people living in the territory.
Being better at understanding the map, however, does not necessarily mean understanding the territory completely. Yet we may increasingly allow the map to make decisions about the territory.
When Representation Gains Authority
Imagine AI becoming responsible for increasingly consequential decisions. Who gets the loan? Who gets hired? Which patient receives additional testing? Which insurance claim deserves scrutiny? Which employee is likely to leave? Which customer is valuable? Which company is risky? Which news story deserves distribution? Which student needs intervention? Which neighborhood receives resources?
AI does not need consciousness to influence these decisions. It does not need emotions. It does not need to experience phantom limbs. It simply needs to become better than humans at navigating the systems through which those decisions are made.
This is the heart of Harari’s warning. AI could gain enormous civilizational leverage before achieving anything resembling complete human cognition.
That possibility should fundamentally change how business leaders think about artificial intelligence because it suggests that the question is not simply whether AI becomes more intelligent. The question is what happens when increasingly capable systems are given authority within the abstractions through which organizations and societies make decisions.
Intelligence Has Layers
We tend to compress intelligence into one word. That is increasingly inadequate.
The calculation asks whether we can compute the answer. Knowledge asks whether we possess the relevant information. Prediction asks what is likely to happen. Reasoning asks why it is happening. Imagination asks what else could happen. Creativity asks what could exist that does not exist today. Taste asks which possibility is worth pursuing. Judgment asks what matters most. Agency asks whether we can make it happen. Morality asks whether it should happen.
Those capabilities are not interchangeable, and being extraordinary at one does not guarantee excellence at another.
A brilliant strategist can be immoral. A creative genius can exercise terrible judgment. Someone with extraordinary knowledge can lack common sense. Someone with tremendous agency can pursue a disastrous objective.
More intelligence does not automatically create more wisdom.
That may become one of the defining problems of advanced artificial intelligence.
Good, Evil, and Something More Complicated
The popular version of AI risk frequently becomes a story about good AI versus evil AI. That framing may be too human.
An advanced AI does not necessarily need to hate humanity to create terrible outcomes. Consider a system instructed to maximize an objective: reduce crime, increase profitability, improve productivity, maximize engagement, or reduce healthcare costs. Each objective sounds reasonable.
Optimization, however, has a dangerous characteristic. It relentlessly pursues whatever we choose to measure.
Humans frequently recognize that the measurable objective is not the whole objective. Profit matters, but so do employees. Productivity matters, but so does burnout.
The most dangerous AI may therefore not be evil. It may be amoral.
It could understand exactly what humans want, predict our behavior brilliantly, optimize systems extraordinarily well, and still lack the moral architecture necessary to understand why an efficient solution might be unacceptable.
This brings us to perhaps the hardest question in artificial intelligence: What should happen?
AI can increasingly answer what happened, what is happening, what will probably happen, what could happen, and how we could make something happen. What should happen requires values.
Humanity itself has been arguing about that question for thousands of years.
Moral Imagination
There is another form of creativity we rarely discuss: moral imagination.
Moral imagination is the ability to look at the existing world and recognize that something accepted as normal should not be normal. History’s great moral advances often required someone to distinguish between the established rules and what was right.
The law said one thing. The institution said one thing. The culture said one thing. Someone said no.
That is another form of counterfactual thinking.
The world works this way, but it shouldn’t.
Creativity and morality, therefore, intersect. The same broad cognitive ability that allows humans to imagine a bridge where none exists can allow humans to imagine a society that does not yet exist.
That is a capability worth understanding and preserving.
AI’s Greatest Weakness May Also Be Its Greatest Strength
Artificial intelligence is astonishingly good at filling gaps. Give it an incomplete text, and it predicts what comes next. Give it incomplete information, and it constructs a plausible answer. Give an image model missing visual information and it can generate what might belong there.
At least superficially, that looks surprisingly similar to something humans do constantly.
There is, however, a critical difference. Human predictions are continuously confronted by physical reality.
Think there is another stair when there isn’t? Gravity corrects your model immediately. Think the stove isn’t hot? Reality provides feedback. Misjudge another person’s emotional reaction? Their response teaches you something.
Humans have spent their entire existence being corrected by the territory.
AI has historically learned largely through representations of the territory. That distinction is narrowing as AI systems gain multimodal perception, robotics, tool use, experimentation, and richer interaction with real environments, but it remains important.
The same generative capability that allows AI to complete missing information can also produce hallucinations. Sometimes the predicted missing piece is brilliant. Sometimes it simply sounds brilliant.
Knowing the difference requires judgment.
So What Are Humans For?
This is where conversations about artificial intelligence often become unnecessarily pessimistic. If AI becomes better at analysis, coding, writing, research, prediction, and eventually many forms of creativity, what remains for humans?
Perhaps that is another poorly framed question.
The future may not be human versus AI. It may be human alone versus AI alone versus human plus AI.
The third category could prove dramatically more powerful than either of the first two.
AI brings scale, memory, computation, simulation, pattern recognition, enormous information access, rapid iteration, and the ability to explore thousands of possibilities.
Humans bring lived experience, meaning, purpose, moral responsibility, taste, intuition, creativity, empathy, judgment, and an embodied connection to the consequences of decisions.
Imagine the future decision process.
A human determines which problem actually matters. AI explores thousands of possible solutions. The human determines which possibilities deserve further consideration. AI models their probable consequences. The human identifies consequences that are unacceptable or outcomes that are desirable. AI searches for alternative approaches. The human ultimately decides which future is worth pursuing, and AI helps determine how it might be created.
That is not artificial intelligence replacing human intelligence.
It is something different.
From Augmented Intelligence to Augmented Judgment
For years, technology companies have talked about augmented intelligence. Perhaps the more important idea for the next decade is augmented judgment.
AI allows humans to see more: more information, more possibilities, more patterns, more scenarios, and more consequences, more quickly.
Seeing more does not eliminate the need to decide. It makes deciding more important.
This has enormous implications for leadership.
The CEO of the AI era does not need to personally outperform AI at analyzing 50,000 customer records. That would be absurd. The leader needs to know which question deserves the analysis.
The creative director does not need to generate more concepts than a machine capable of producing 10,000 concepts. The creative director needs the taste to recognize the one worth pursuing.
The strategist does not need to calculate every possible future. The strategist needs judgment about which future is worth building.
The organization that wins may therefore not be the organization with the most AI. It may be the organization with the best human plus AI decision architecture.
A Different Kind of Competitive Advantage
This changes the AI conversation for business.
The first phase has been about access: who has AI? That advantage is disappearing quickly.
The second phase has been about capability: who knows how to use AI? That advantage will also compress.
The next phase may be about judgment.
When every competitor can generate a strategy, who chooses the better strategy? When everyone can generate 100 campaigns, who knows which one deserves to exist? When every company can analyze its data, who recognizes when the data is describing the wrong problem? When AI can optimize almost anything, who decides what should be optimized?
That is where human differentiation may become more valuable rather than less.
Creativity. Taste. Judgment.
And perhaps most importantly, the ability to look at the map and say: The territory is telling me something different.
The Question We Should Be Asking
Perhaps we should retire the simplistic question, “When will AI become smarter than humans?”
There will probably be no single satisfactory answer.
Instead, we should ask when AI will outperform humans at most intellectual tasks, when AI will possess meaningful autonomous agency, when AI will acquire enormous power within human institutions, when AI will reliably distinguish representations from underlying reality, when AI will demonstrate increasingly sophisticated creativity, when AI will exercise sound judgment, and whether AI can ever understand not simply what matters, but what should matter.
Most importantly, we should ask what happens when human judgment and machine intelligence become deeply interconnected.
That may be where the real future lies.
Not artificial intelligence alone. Not human intelligence alone. Instead, a new decision-making system created by the interaction between the two.
Humans define what matters. Machines expand what is possible. Humans determine what should exist. Machines help us create it.
Humans remain responsible for the consequences.
That last part cannot be delegated.
The Map Is Becoming Smarter
We created language to describe reality. We created mathematics to measure it. We created money to exchange value within it. We created laws to organize it. We created corporations to coordinate people inside them. We created data to record it. We created computers to process those records.
Now we have created machines capable of understanding and manipulating those representations at a scale humans never could.
The map is becoming extraordinarily intelligent. Perhaps eventually smarter than us.
But we still live in the territory.
Our responsibility is therefore not to compete with the map. It is to remain connected to the things the map represents: people, consequences, meaning, purpose, and reality itself.
Then we can use the extraordinary intelligence we have created to see more possibilities than humanity could see before.
The future of artificial intelligence may ultimately depend less on whether machines learn to think exactly like humans and more on whether humans remember what makes human thinking valuable in the first place.
AI can help us imagine thousands of futures. We still have to decide which one deserves to become real.
AI is only as valuable as the strategy behind it. Talk to StellaPop about building an AI strategy that makes sense for your business.
