When to distrust the AI
The model is confident even when it's wrong. Here's how to catch it before it costs you.
AI coding tools are genuinely great — and they will also, with total confidence, tell you things that are wrong. Learning where that happens is what keeps you out of trouble. This isn’t about distrusting the tool; it’s about knowing its failure modes so you can double-check the right things.
Confidence is not correctness
The most important thing to internalize: the AI sounds equally sure whether it’s right or wrong. There’s no wobble in its voice when it invents a function that doesn’t exist. So you can’t use “it seems confident” as a signal. You need other checks.
Where to be extra skeptical
Turn up your skepticism when the AI is dealing with:
- Anything security- or money-related — auth, payments, permissions, deleting data. Wrong here is expensive. Verify carefully.
- Exact facts — version numbers, API parameter names, config values. These are easy to hallucinate. Check the official docs.
- Very new or obscure tools — the model may have little or outdated knowledge of them.
- “It’ll definitely work” claims — the only real proof is running it, not the AI’s promise.
The AI writes code using a function called db.autoBackup() and assures you it exists. You've never seen it in the docs. What's the right move?
Your best lie-detector: run it
Here’s the good news — you don’t have to be an expert to catch most mistakes. Running the code tells you the truth. The app either works or it doesn’t. A test either passes or fails (that’s the next module). The AI’s confidence is an opinion; the running program is a fact.
What's the most reliable way to know whether AI-generated code actually works?
Putting the module together
You now have the full loop for staying in control while coding with AI: small steps, review every diff, prompt with clarity, and verify instead of trusting. That’s not a limitation on the tool — it’s what lets you use it fearlessly, because nothing it does can get past you unchecked.
Module project: catch a mistake on purpose
Build the skeptic’s reflex:
- Ask your AI tool to build a small feature that touches something specific — e.g. “format this date as DD/MM/YYYY.”
- Don’t trust the result — run it. Does the output actually match what you asked?
- If it’s wrong, notice that it probably looked right. That gap between looks-right and is-right is exactly what verifying closes.
- Fix with a specific follow-up, run again to confirm, then commit.