Don't treat AI like an employee — treat it like a trainee
Why a model that's 'wrong 1 in 8 times' needs guardrails, a watcher, and a plan for where saved hours actually go.
You just hired someone with a great resume who never takes lunch, works weekends, and never calls in sick — and one out of every eight things they tell you is simply made up.
That was Nick Dreyfus's opening. He uses that image to pull the conversation away from hype and toward one simple management truth: you wouldn't put a person who confidently lies on the phone with your best client. Don't put unchecked AI in the same seat.
The 1-in-8 problem and why it matters
Published hallucination rates for large models sit in the low double digits to about one in eight, depending on task and model. Nick doesn't present this as a technical bucket to fear; he presents it as an operational reality leaders must manage.
The crucial detail: a human who is wrong usually gives you a tell — hedges, pauses, “let me check.” Models give wrong answers with the same polished confidence as correct ones. They invent prices, capabilities, and numbers in clean, professional language. When that goes out under your company name, it costs more than the error; it costs trust.
The invoice story every owner should picture
Nick offers a short illustration you should imagine clearly: an AI is told to apply a 5% price increase. The system interprets “5%” without context — per run, per day, compounded — and the number compounds until the invoice your client opens is orders of magnitude off.
Fixing the invoice takes minutes. Fixing the relationship doesn't. The client now checks every number you send. You did not lose money as much as you lost the benefit of the doubt — and that fracture is expensive in ways P&L spreadsheets don't capture.
"The value in these systems is not intelligence, it's guardrails." — Nick Dreyfus
What an agentic engineer actually does
Nick names a role that barely existed three years ago and defines it precisely: the agentic engineer doesn't make the AI smarter; they make it safer. The craft is not to let the model ‘think’ freely, it's to narrow decisions until there is nothing left to invent.
Concrete examples he gives:
Wrong build for marketing: “Here is our brand, go write a campaign” — agent invents claims and statistics.
Right build: choose from approved copy and structures; flag anything outside those lines.
Accounting: never allow rounding; build verification on every figure and block anything outside expected ranges.
Those constraints are not a prompt you type on Tuesday. They are engineering decisions plus human-approved policies that prevent confident, silent failures.
Hours saved ≠ dollars earned
This is the spine of Nick's business argument. Vendors sell hours saved — “we saved your team 400 hours.” But an hour recovered translates into value only if the organization has a place for that time to go.
Nick walks through the arithmetic: give five hours back to accounting and nothing changes in the top line. Give those same five hours to sales and you might cut a 90-day sales cycle to 45 days. Same headcount, same marketing spend, doubled capacity to close deals. That's the difference between efficiency and capacity.
The operational choice you need to make is not where you can remove the most clerical hours; it's where freed time will be redeployed into revenue, faster closes, or scalable pipeline.
Three real choices — and one common accidental decision
Nick lays out the practical options:
Do nothing — which is not absence of AI but unmanaged, shadow AI on personal accounts and free tiers. You didn't avoid risk; you avoided knowing about it.
Build in-house — hire multiple agentic engineers, embed them across functions, and budget for continuous maintenance. This is legitimate for larger organizations that can recruit and retain the talent.
Use a partner — get an engineering team plus business judgment handed to you. (Nick describes Fortify AI, their managed offering, as an example of this approach.)
The one “not-really-a-choice” is doing nothing intentionally: AI will already be in your business, and that passive path usually becomes the most expensive over time.
The simple three-question starting point
Nick's actionable assignment is low-effort and high-returnful: go ask your team, honestly and without penalty — what AI are you using, and what are you putting into it? Then ask a second question: if something the AI outputted was wrong, who would catch it?
If nobody has an answer to that second question, you've found where to begin.
Closing thought
AI right now is not a drop-in employee. It's a fast, confident system that can save enormous amounts of low-value work — but only if you treat it like a managed trainee: narrow the decision space, build guardrails, assign continuous ownership, and decide where the hours you save will actually go. Do that, and you won't just avoid expensive failures; you will open pockets of capacity that change how the business grows.
If you want to walk through what you find, Nick offers an AI usage policy template and a conversation starter — that detail sits with the episode for anyone who wants the worksheet.