The problem
Preface
Imagine this: An internal research team sends out a report containing an in-depth SWOT, market sizing, and competitor analysis for a new product your company is working on. This is a key piece of research that will directly inform your go-to-market (GTM) strategy, including how much to invest in the launch campaign, which markets to prioritize, and how to position the product.
There's already been lively debate among your team about your GTM strategy, splitting everyone between two options. This research report is supposed to unblock the decision and clarify which path to pursue.
The report goes out, and you and your leadership team start to dig in.
At face value, the report looks polished and thorough. The writing is clear, the structure makes sense, and it contains extensive citations, facts, and claims. The conclusions and recommendations look compelling. Overall, it appears to be solid work, and you have no reason to question its accuracy.
And great news. The report provides strong evidence supporting one of the two GTM strategies under review. The debate is settled. Your team mobilizes and prepares for a product launch informed by the data and insights in the report.
Fast-forward six months. Your team is now three weeks post-launch, and you're facing strong headwinds. Traction is poor, and you're burning cash on advertising that isn't landing.
As you and your team try to recalibrate and salvage the flailing product launch, you return to the research report that informed your GTM strategy. On closer review of several claims used to validate your strategy, you discover that two of the sources can't be verified as they don't seem to exist outside the report. Another key stat that was used comes from a verifible source, but it appears the data was taken out of context and doesn't actually validate the strategy you pursued. You quickly realize that the core body of evidence your team relied on either doesn't exist, or doesn't really your chosen strategy.
Upon checking, it turns out the report was generated by AI, and some of the sources and claims were completely made up (what we often call an AI hallucination). In other cases, stats and claims could be traced back to a real source, but the source wasn't reputable, or the stats were taken out of context and didn't align with the report's conclusions.
How could this happen? The team that worked on the report was staffed by smart people with a track record of delivering great work. Your company has also embraced AI, giving every employee access to the best AI models and tools, along with training on how to prompt effectively. And, of course, the report looked polished and thorough.
Sloppy work would have been easier to question: a spelling error or run-on sentence, a chart that looks wrong, a claim without evidence. Many of us have spent years learning to spot those red flags. We read, mark up, ask for changes, and read again.
But this piece of work didn't contain any of the tell-tale signs that traditionally trigger your Spidey-sense. The parts that were wrong looked just as convincing as the parts that were right, maybe even better.
So, what went wrong? As you discovered, the report was generated in part by a large language model (LLM). This is the same kind of model that can solve complex math problems, write code, build applications, generate images, and analyze massive datasets. We all know that AI can achieve miraculous things, yet it can still make mistakes. The problem is that those mistakes can be hard to spot, either because the AI gives you a confident, polished response, or because those errors are buried deep, and invisble to the naked eye.
You've just learned a critical lesson: AI output can be, and often is, deceiving. On the surface, almost every line of generated text or code looks polished, accurate, and confident. It's only when you dig in and scrutinize the work that you start to see the seams and imperfections.
You might think this hypothetical is extreme. Who in their right mind would deliver and publish work with such egregious errors? Well, in 2026 alone, Deloitte, EY, KPMG, and PwC all published (and later retracted) reports that contained sources, claims, staistics or citations that were entirely fabricated. These are some of the biggest firms on the planet, and the very ones generating substantial revenue by teaching other companies how to use AI.
Working successfully with AI isn't just a matter of technical skill, like knowing how to prompt effectively or build an agent loop. It requires judgment, discernment, and the ability to know when to use it and when not to.
A fabrication can look like a compelling fact. A codebase riddled with vulnerabilities can still compile and run without errors.
When we do the work ourselves, we tend to check and verify what we did. But when AI does it, we're more likely to accept what it gives us without checking its work.
This problem is compounded by two factors. First, many consider productivity gains and speed to be core benefits of AI. However, this mindset can lead us to prioritize completing a task quickly—without checks—when there are no visible issues with the output. Second, the miraculous capabilities of modern AI lead people to take on work they have no underlying expertise in. A non-technical marketer can now code an app; a developer without marketing expertise can generate compelling web copy and imagery for a product launch.
These factors make us far more susceptible to shipping work that looks good on the surface, but underneath, may be flat-out wrong and full of risk.
However, just because you can catch mistakes doesn't mean AI should have done the task in the first place. You can still exhibit poor judgment by exposing your company's sensitive data to a frontier lab's AI models, or by handing AI a task that requires personal attention and care. Correctness is only one dimension of good judgment when using AI.
The task matters
A recent study conducted by Harvard researchers in collaboration with Boston Consulting Group (BCG) examined how AI tools can improve or degrade the productivity and performance of business consultants. The experiment involved 758 knowledge workers who were split into groups and given a series of tasks to complete. Some participants had access to AI tools, while others completed tasks without AI assistance.
The study found that AI clearly improves human performance on certain tasks but degrades performance on others. This distinction is what the authors call the "jagged technological frontier."
Tasks within the frontier are well-aligned with AI capabilities, allowing the technology to act as a performance booster. Conversely, tasks outside the frontier require nuanced contextual synthesis, complex logic, or deep domain judgment that current LLMs struggle with. On these tasks, AI acts as a performance disruptor due to human overreliance.
So what exactly did the consultants do that resulted in degraded performance? They were given a complex business strategy case asking them to recommend which brand or channel a client should invest in for future growth, based on financial spreadsheets and qualitative insider interviews. The spreadsheets provided a surface-level impression, but the qualitative interview notes contained subtle, critical details that contradicted the raw spreadsheet data. When prompted directly, the AI model defaulted to an incorrect recommendation based on the surface-level quantitative data. Because consultants overrelied on the AI's persuasive logic, those using AI were 19% less likely to arrive at the correct strategic recommendation compared to those working without AI.
In summary, AI was a proven productivity booster when the right tasks were matched to the right model. But AI actually degraded productivity when the user was over-reliant and didn't thoroughly scrutinize its work. This is the jagged frontier of AI.
This is one of the strange realities of working with AI: something that looks difficult to us can be easy for it, while something we consider trivial can trip it up. You cannot judge whether to hand over a task based on how long it would take you to do it manually.
Dumping AI onto every task without understanding that boundary is a recipe for failure. Perhaps even disaster.
We humans have little patience for either the fear or the hype surrounding AI. Neither helps when you have an actual job to do, with deadlines approaching. To cut through the hype and achieve great results with AI, you need to understand where it is useful, where it struggles, and where you will need to check its work. Being impressed or frightened doesn't answer any of those questions.
Calling every failure a "hallucination" doesn't get you very far either. An invented fact, faulty logic, an omission, and an inappropriate use of AI are completely different problems requiring different responses. An AI hallucination is the most talked-about and visible type of error, but it's far from the only one.
That's why AI training focused solely on technical skills and tools can hurt more than it helps. Focusing only on which button to press or what to type into the prompt box will only take you so far. Using AI tools is just the beginning. You need to understand what the AI is doing, how to work alongside it as the task develops, how to evaluate the results, and to take responsibility for the final output.
Four things to learn
We've established that being "good" at AI requires more than technical ability. You need sound judgment and an eye for the hard to spot mistakes AI can make. To that end, you can think about AI fluency across four key dimensions:
Understand AI before you hand over the work. Which part of the job is suitable for AI? What does it need to do that part well? What kind of errors should you expect? This is a judgment about the task as much as the technology.
Work with AI while the work is happening. How far should it go before you review it? Where do you need to provide context, correct it, or stop it? If you leave everything to default settings, the tool's setup determines how the work proceeds.
Evaluate AI after it produces something. A quick glance is still just a quick glance, regardless of your experience. Check facts against their original sources. Work through the logic. Look for what was left out. The review must match the specific failure you are trying to catch.
Own AI risk across the entire process. Your name goes on the final work. Is it fit to send, to whom, and for what purpose? What happens if it is wrong? Something adequate for your own rough exploration may need much deeper scrutiny before someone else acts on it.
"Understand AI" comes first for a reason. If you misunderstand the task or the capability, you will waste considerable effort supervising and checking the wrong thing. A check is only useful if it catches the failure that actually matters.
How this book is structured
This book follows the same logic. Rather than opening with the four dimensions, we start where the work starts, with one person and one task. From there, we zoom out to the team and, finally, the whole organisation. The four dimensions apply at every level; what changes is who makes each judgement, and how. A short interlude after this preface introduces the three animals we use for those levels: the elephant (you, on your own work), the pelican (your team), and the dragon (your organisation). Here's what each part of the book covers.
Part I: The Problem. We show what happens when you hand work to AI with a quick prompt and accept the first polished result. Every time you hand over work, there are decisions to make: what AI should do, how far it should go before you review it, how you'll check the result, and whether you're prepared to put your name to it. If you don't make those decisions, AI fills the gaps with its own assumptions, which are easy to miss when the output looks finished.
Part II: Tasks. We break work into individual tasks and look at what AI is actually doing in each one, where it tends to fail, and why it performs well on some tasks and poorly on others. This matters because you can't judge whether AI will help with a whole job, but you can judge it for a specific task, including which mistakes to look out for.
Part III: Dimensions. Each of the four dimensions gets its own chapter, covering the four sub-skills that make it up, sixteen in all. We also explain why being strong in one dimension doesn't make you strong in the others, and why feeling confident with AI isn't a reliable sign that you're using it well. Together, these chapters give you specific skills to work on, instead of a vague goal to get better at AI.
Part IV: Measurement. We cover how to find out whether someone can make these judgements. Self-ratings measure confidence and quizzes measure knowledge, but neither shows how a person handles real work, so we explain how to build assessments around realistic tasks. Without that evidence, it's hard to know where you or your team need to improve.
Part V: Archetypes. We introduce twelve job archetypes, such as the Strategist, the Analyst, and the Guardian, and show how AI's usefulness varies across the tasks within a single job. The four dimensions are the same for everyone, but the tasks you do, and the mistakes you're most likely to run into, depend on your role.
Part VI: The Team. We look at how AI skills combine across a team, including blind spots that everyone shares and handoffs where each person assumes someone else has checked the work. A team of capable people can still let errors through if nobody is clear about who checks what.
Part VII: The Organisation. We explain what an organisation needs in place for people to work well with AI. That includes measuring whether the work is improving, not just counting licences and active users, and making sure its processes match what its people can do. Buying tools and running training won't pay off if nothing else about the way work gets done changes.
The final chapter brings these ideas together. It returns to the exercise at the end of this preface and offers practical questions for everyday conversations about AI at work, so you can put what you've learned into practice with your colleagues.
Who is this book for?
This book is for people whose work now involves AI and who need a way to judge whether they are using it well. It assumes you have work to get done, rather than an AI system to build.
Reading can help you understand the core concepts; the exercises are where you find out if you can apply them. Each exercise asks you to work with a real task of your own.
Start with one thing you handed to AI in the last fortnight—one specific piece of work. Write down what it was, what you asked the AI to do, and how you decided the result was good enough. Keep that written down so you can refer back to it as you work through this book.