What AI misses when it matters most
In brief: When situations exceed the frameworks built to manage them, a different kind of capacity becomes relevant. This piece examines what that capacity is, why AI cannot replicate it, and why that gap is likely to matter more, not less, as conditions become less predictable.
When the task changes
A few days ago I responded to a building fire at two o’clock in the morning. Eighteen firefighters were on scene, and we worked through the night until the fire was extinguished. Unfortunately by then the building was gone. The owner was standing outside in the dark. A woman in her seventies, physically composed but clearly disoriented in the way people often are when something irreversible has just happened and the implications have not yet fully registered.
From an operational perspective, everything that could be done had been done.
I then walked over, introduced myself, and asked her name. We spoke for a few minutes and I learnt that it was her birthday, and that detail landed with a certain weight. She then said, almost in passing, that the burning down was perhaps a form of renewal. I was deeply touched by such a perspective. As we spoke, her breathing slowed and her attention seemed to settle, not in the sense that anything was resolved, but in the sense that the moment became more stable and inhabitable, with a slight shift from disorientation towards something she could begin to take in.
That interaction did not change the outcome. The house was still gone. But it changed the situation in a way that felt more significant than anything else I did that night.
Experiences like this are not unusual in disaster work, but they are instructive, because they make visible a shift that is otherwise easy to miss. When systems are functioning, the task is largely technical, and competence is measured by the ability to apply the right procedure at the right time. When those systems fail or are exceeded, the task becomes less about intervention and more about orientation — about recognising what is actually happening, in context, and responding in a way that fits the situation as it is rather than as it is assumed to be.
I have been thinking about this in parallel with my use of AI.
(this picture is not from the actual incident)
Representation and orientation
I rely on it regularly for writing, research, structuring ideas, or creating images, and in many contexts it is genuinely useful, particularly where the problem is well-defined and the parameters are clear. Over time, however, a limitation becomes apparent, which is not so much about accuracy as about orientation. AI can produce responses that are coherent and relevant to the prompt, but it does so by operating on representations of a situation rather than being situated within it, which means that it can generate plausible answers without registering what is actually at stake.
In stable conditions, this distinction has limited consequences. Where feedback loops are clear and problems are bounded, this form of ‘intelligence’ performs well and can be (somewhat) relied upon.
It becomes more consequential under different conditions, particularly where the situation itself is still forming, where multiple dynamics are interacting, or where people are trying to make sense of something that has just disrupted their assumptions. In those moments, the task is not to generate answers, but to remain oriented to what is unfolding, which requires a form of attention that is both perceptual and relational, and which cannot be reduced to pattern recognition or response generation. It involves registering subtle shifts in context, emotion, and meaning as they emerge, and adjusting one’s response accordingly, something still profoundly human. Whether AI systems will eventually replicate that capacity in any meaningful sense remains genuinely open. For now, the gap is real and consequential.
The context we are moving into makes this distinction nonetheless very relevant. Climate disruption is increasing the frequency of events that do not fit established expectations, and these events rarely occur in isolation; they overlap, interact, and evolve while responses are still underway. At the same time, geopolitical and economic conditions are becoming less predictable, which further reduces the reliability of inherited frameworks. AI is part of this landscape. It accelerates the production of information and the pace of decision-making, but it also increases the volume of signals that need to be interpreted and can obscure the underlying dynamics that matter. Under these conditions, the difficulty is not simply technical. It becomes harder to read situations, to distinguish signal from noise, and to respond in ways that are grounded rather than reactive.
This is where the limitation becomes visible. AI can assist with processing and generating responses, but it cannot determine what matters in a situation where meaning is still emerging, nor can it stand in relation to another person whose world has just shifted and respond in a way that stabilises that moment.
The woman standing outside the remaining of her house did not require analysis or interpretation. What mattered was that someone noticed her, recognised her as a person in that moment, and remained present long enough for the situation to settle slightly.
What this means for leadership under disruption
I have watched leaders lose situations not because they lacked information or capability, but because they could not stay oriented when the situation stopped resembling anything familiar. The response fragments — sometimes gradually, sometimes dramatically — as people begin managing their own disorientation rather than the situation in front of them. What holds a response together in those moments is rarely technical. It is the capacity to remain present to what is actually unfolding, and to hold enough stability for others to begin making sense of it too.
For people in leadership roles, this has practical implications. In disrupted conditions, the expectation that leaders will provide clear answers becomes less realistic and, at times, counterproductive. What becomes more important is the capacity to stay oriented, to recognise when situations exceed existing frameworks, and to hold enough stability for others to begin making sense of what is unfolding. These are not capabilities most organisations explicitly develop, but they are often what determines whether a response holds or fragments under pressure.
* * *
The question is whether we are treating that capacity as incidental, or recognising it as something that will increasingly sit at the centre of how we respond when things stop working as expected. At present, most systems assume it will be there without explicitly cultivating it.
That night, the most important thing I did was walk over, ask a woman her name, and stay with her for a few minutes while the reality of what had happened began to take shape. Nothing about that scales easily, and it is not something that can be optimised in the usual sense, but it remains central to how people navigate disruption when it occurs.
It however points towards a form of effective presence in leadership that is not typically defined or developed explicitly, and which I will examine in more detail in a subsequent piece.