AI will tell you that a competitor's revenue grew 40% last year — with complete confidence — when it made that number up entirely. It will cite a study that doesn't exist. It will describe a market trend that contradicts the actual data. And it will do all of this in the same calm, authoritative tone it uses when it's completely correct.
This is called hallucination, and it's the number one reason small business owners get burned when they try to use AI for research and analysis. The output looks like analysis. It reads like analysis. It just isn't always true.
This guide teaches you how to use AI as a genuine analysis partner — while building in the verification habits that keep you from acting on fiction.
The three types of AI analysis errors
Not all AI errors are equal. Understanding the types helps you know where to verify and where to trust...
The three types of AI analysis errors
Not all AI errors are equal. Understanding the types helps you know where to verify and where to trust:
- Fabrication: The AI invents a fact, statistic, or citation that doesn't exist. Most common when asking for specific numbers, recent data, or named sources.
- Distortion: The AI takes a real fact and misrepresents it — gets the direction right but the magnitude wrong, or applies a finding from one context to a different one.
- Staleness: The AI gives you accurate information that was true 18 months ago but has since changed. Especially common for market data, regulations, and competitive positioning.
HIGH HALLUCINATION RISK
- Specific revenue/market size figures
- Recent news or announcements
- Named studies with citations
- Competitor pricing specifics
- Regulatory or legal details
- Anything after the model's cutoff date
LOW HALLUCINATION RISK
- General strategic frameworks
- Synthesizing information you provide
- Identifying patterns in your own data
- Generating hypotheses to test
- Structuring your thinking
- Drafting questions to ask humans
The safe AI analysis workflow
Protocol uses a four-step workflow for any AI-assisted analysis in client work. It's simple enough to become a habit:
PROTOCOL ANALYSIS WORKFLOW
📥
Step 1 — Feed it your data, not your questionsGive AI the actual information you have — your sales data, customer feedback, market notes — and ask it to analyze what you've given it. Don't ask it to fetch information it might hallucinate.
INPUT FIRST
🔍
Step 2 — Ask for patterns, not factsAsk: "What patterns do you see in this data?" not "What is the average conversion rate in my industry?" One question uses your verified information; the other invites fabrication.
PATTERN ONLY
✅
Step 3 — Verify any external claim before you use itIf the AI references something external (a statistic, a competitor's move, an industry trend), treat it as a hypothesis, not a fact. Spend 5 minutes finding a primary source before you act on it.
VERIFY FIRST
🧪
Step 4 — Use AI to challenge your conclusions, not confirm themAfter you've drawn a conclusion, ask AI: "What are the three strongest arguments against this?" Confirmation bias is as dangerous as hallucination.
CHALLENGE IT
Prompt templates for safe analysis
Copy these and adapt for your business:
The most dangerous AI output isn't the obviously wrong answer — it's the plausible-sounding wrong answer. Build verification into the habit, not the exception.
Part of Protocol's AI Without the Slop series. See all six resources →