AI Architect Series, Part 2: Embracing the Unknown in the Age of AI

AI Architect Series, Part 2: Embracing the Unknown in the Age of AI

One of the most difficult questions I face when working with AI is surprisingly simple: What should I ask next?

Sometimes I can describe the outcome I want, but I do not understand the unfamiliar territory well enough to ask a useful question. At other times, I know exactly how I would approach the task. That certainty creates a different problem: I may be overlooking a better way to do it.

And when several agents return pages of analysis, there is another challenge. I still need to understand enough of their work to make a sound decision.

In the first article in this series, I explored how to build an AI ecosystem around tools, skills, context, and governance. As that ecosystem becomes more capable, the architect needs to grow alongside it.

For me, that means developing three habits: know your unknowns, unlearn familiar methods, and verify your own understanding. Together, they help turn unfamiliar territory into something we can investigate, test, and learn from.

Know your unknowns: learn enough to ask the next question

There is an uncomfortable stage in learning where you know too little to describe what you are missing. You can ask an AI to explain a subject, receive a polished answer, and still have no idea which part matters to your problem.

The immediate goal is to develop enough understanding to recognize a useful question.

Consider an illustrative task: building a demo that shows how an AI assistant investigates a performance incident. An initial request might be, “Help me build an AI incident-analysis demo.” That leaves almost everything open. The assistant must supply assumptions about the audience, data, architecture, and meaning of success.

Before choosing tools, I would work through a few gaps:

What is unclear?	A question that moves the work forward
What the demo must prove	What should the audience understand or be able to judge at the end?
What evidence is available	Do the sample logs and metrics distinguish the suspected cause from other explanations?
What the assistant may do	Is it explaining evidence, investigating through tools, or proposing a change?
What would count as success	How will we recognize a correct result and a convincing but unsupported one?

Each answer makes the next question more specific. “Build an incident demo” might become: “Using these synthetic logs, show why a slow response could come from either a database delay or a downstream dependency, and identify the observation needed to distinguish them.”

That is a much more useful task. It gives the investigation a direction and makes a missing piece of evidence visible.

George Loewenstein’s information-gap account of curiosity offers a useful connection: recognizing a gap in understanding can create the desire to resolve it. Applying that idea to AI architecture is my interpretation, rather than a tested learning formula. A little knowledge can help us see what to investigate next. The Psychology of Curiosity.

When I cannot yet formulate that question, a prompt like this provides a starting point:

I understand cloud architecture, but this topic is unfamiliar to me.
Explain the essential concepts in terms I can connect to my experience.
Identify assumptions in my request and important gaps in the evidence.
Suggest three questions that could change the design, and explain why.
Include a question that challenges the approach I appear to prefer.

The resulting questions are candidates to investigate. An agent can share my blind spots or invent a concern, so I would check the important ones against documentation, a small experiment, or a colleague with relevant experience.

I also need a stopping point. For a bounded demo, enough understanding might mean I can explain the design, identify its main limitation, and define a test that could show it is wrong. A production decision demands deeper evidence. The depth of learning should follow the decision and its consequences.

Unlearn: reconsider the method you already do well

Building demos is a useful example for me because I already know ways to do it successfully. I can choose familiar components, assemble the environment, write the supporting code, and prepare the story.

That experience is valuable. It can also make one route feel inevitable.

In the age of AI, I need to ask whether the method still fits the task. Some steps may be automated. Others may disappear when I describe the desired outcome more clearly. AI might also make it practical to explore several approaches before committing to one.

Research on the Einstellung effect, or fixation on a familiar solution, gives this concern a useful foundation. In a study of expert chess players, participants’ attention remained drawn to features of a solution they already knew while they reported searching for a better one. Chess is not demo development, but the finding illustrates how a successful idea can constrain further exploration. Bilalić, McLeod, and Gobet.

Someone who has never built a demo my way may try a different route with AI. That possibility should make me curious. It does not establish that less experience produces better results; it suggests a reason to compare approaches before defending the familiar one.

For the incident demo, I might instinctively ask an agent to generate the components of my usual architecture. An alternative is to start with the audience’s learning outcome:

Design a five-minute incident-investigation demo for cloud architects.
Use synthetic data. Show an observed symptom, two plausible explanations,
and the evidence that distinguishes them. Label simulated components.
The demo must be repeatable from a clean start.

Compare three approaches before building. For each, explain what it proves,
what it cannot prove, the setup effort, and how we would verify it.

One approach might use a static walkthrough. Another might replay prepared telemetry. A third might use a small live sandbox. Which one is appropriate depends on what the audience needs to see. If the objective is to demonstrate a real integration, a mockup cannot establish that it works. If the objective is to explain an investigation, an elaborate deployment may add little value.

This is what I mean here by intent-driven development: make the outcome, constraints, and evidence of success explicit before prescribing the steps. It gives AI room to propose a method while keeping the architect responsible for the requirements.

The comparison must include the work after generation. Did I spend less time overall, including review and rework? Can someone else reproduce the result? Are the assumptions and simulated components clear? Does the demo actually teach the intended lesson?

Research with 758 consultants described AI capability as a “jagged technological frontier”: assistance was useful for some tasks and unreliable for others. The practical implication I draw is to evaluate the particular workflow, rather than assume that adding AI makes every step better. HBS AI Institute.

Unlearning requires keeping my knowledge available while loosening my attachment to one way of applying it. Experience still helps me recognize an unrealistic design, a missing dependency, or a result that does not make sense.

Verify your own understanding: make human review work

We often discuss agents hallucinating. I also worry about my own unsupported certainty: accepting an explanation too quickly, filling in a missing detail, or approving a conclusion I have not understood.

“Human hallucination” is a provocative way to describe that concern, but the distinction matters. Here I mean mistaken assumptions, overconfidence, and misplaced trust—not the same mechanism as a model generating false content, or a clinical phenomenon.

The problem becomes harder when several agents produce detailed outputs at once. A human can remain in the loop yet have too little attention to scrutinize every claim. Merely assigning someone to review the result does not ensure that useful review occurs.

A CHI 2025 study surveyed 319 knowledge workers and found that greater confidence in generative AI was associated with less reported critical thinking. This was a self-report study, not proof that AI causes people to lose that ability. It does, however, give us a reason to examine how trust affects scrutiny. Lee and colleagues.

The research process for this article supplied a small example. A NotebookLM overview described newer developer-productivity evidence as a “significant productivity uplift.” That wording was stronger than the underlying evidence justified. METR’s February 2026 update explicitly says selection effects and measurement problems make its new data an unreliable estimate of the current productivity effect. The qualification changes what a reader can conclude. METR’s study update.

The summary was easy to read. It still needed checking.

Give attention a structure

In Part 1, progressive disclosure helped control how much context an AI needed at once. I would apply the same principle to the human reviewing its work: present what matters for the decision, with a clear route to inspect the supporting detail.

A useful handoff has three levels:

1. Decision summary: what happened, what decision is needed, and the most important limitation.
2. Evidence: the sources, observed results, tests, and unresolved disagreements behind that summary.
3. Detailed artifacts: the analysis, logs, code, or other material needed for deeper inspection.

For the illustrative incident demo, an honest summary might read:

The demo identified the condition deliberately inserted into the synthetic data. A replay check passed. The explanation is ready for review, but reset behavior has not been tested and no live integration was exercised. Next decision: accept the explanation, then test repeatability before presenting it.

That tells me what was demonstrated and what remains uncertain. In a real handoff, the replay result and artifacts should be linked directly, with the input or version identified so that I can check what was tested.

Instructions such as “Explain this to me as a CEO” or “Explain it as simply as possible, as if I were a caveman” can help make an answer accessible. But a role or style request is incomplete. A CEO still needs to see the uncertainty that affects a decision, and a simple explanation still needs to preserve the distinction between an observation and a guess.

Instead of asking only for brevity, I would specify the information that must survive:

Lead with the result and the decision I need to make.
Use plain language and keep the screen summary brief.
State what was observed, what was inferred, and what remains unknown.
Keep failed checks, conflicting evidence, and material limitations visible.
Save detailed analysis to files and link to the evidence for the key claims.
If a short summary would hide something decision-critical, say so.

This can become a reusable skill so I do not have to restate the handoff format for every task. Writing details to files improves navigation; it does not automatically reduce the model’s token use or make the content more accurate. Likewise, a positive tone or sentiment says little about whether the conclusion is supported.

Give review something concrete to test

A concise answer can still be wrong. For an important conclusion, I want to identify the claim on which the decision depends, inspect its evidence, and ask what would change my mind.

For the demo, that could mean rerunning the scenario from a clean state or checking whether the explanation changes when the distinguishing evidence is removed. For a technical article, it could mean opening the cited source and checking whether it supports the exact claim. These checks examine the result outside the agent’s own description of it.

Anthropic’s evaluation guidance makes a useful distinction between what appears in an agent’s transcript and the actual outcome in its environment. It also describes combining automated checks, model-based evaluation, and human judgment. That is a helpful foundation for deciding what kind of evidence a task needs. Demystifying Evals for AI Agents.

Agent loops and teams can help gather evidence and challenge an answer. But several agents may repeat the same assumption, especially when they rely on the same source or framing. Agreement among them cannot substitute for checking that source or observing the actual result. If they disagree, the final summary should retain the disagreement that matters to the decision.

Human review also needs an active step. Before reading the recommendation for a consequential choice, I can write down what I expect the evidence to show. Research on cognitive forcing interventions found that requiring more deliberate engagement reduced overreliance in the studied task, although participants liked some of those designs less. That supports thoughtful review design, not adding approval clicks everywhere. Buçinca, Malaya, and Gajos.

The amount of review should match the consequences. A reversible draft and a production change need different levels of scrutiny. In both cases, I should be able to explain what I am accepting and why.

Growing into the next level

Embracing the unknown becomes practical when I can turn it into a question, an experiment, or a piece of evidence to inspect.

On the next familiar task, I can start by naming one thing I do not yet understand. I can ask whether the method I normally use still serves the outcome. And before accepting the result, I can check one important claim against the evidence behind it.

Those habits reinforce each other. Better questions reveal alternatives. Trying an alternative tests an old assumption. Verification tells me what deserves to become part of my understanding—and what I still need to learn.

That is how I want to keep growing as an AI architect in the age of AI.