Why AI Agents Need to Make Mistakes

Every developer hates AI hallucinations. But what if those mistakes were the reason AI seems intelligent at all? Look at how real intelligence works in the brain, in evolution, even in human behavior. It’s full of errors. That’s not a flaw. It’s a feature.

Maybe the best AI design docs aren’t on GitHub — they’re buried in neuroscience and evolutionary biology papers about bias, noise, and how the brain decides.

Brains Are Noisy — and That’s a Good Thing

The brain is a messy calculator. Ask it to estimate probabilities and it fails spectacularly, as shown by Kahneman and Tversky’s work on cognitive biases¹. Neuroscientists now know that the brain’s natural variability, or neural noise, isn’t just random error. In a 2005 paper published in Nature Reviews Neuroscience², Stein et al. explain that variability in neuron firing is part of the signal. The noise in brain activity is what lets us detect weak signals, adapt fast, and stay creative. As neuroscientist Aldo Faisal put it³, noise isn’t a bug — it’s how the brain stays flexible.

Deep learning borrowed that same principle early on — stochastic gradient descent, dropout, random weight initialization — randomness wasn’t an afterthought but a foundation. It proved essential for generalization⁴. An agent that never fails never learns.

💡 Takeaway: Embrace useful noise.

• Let your agent explore by asking open-ended questions.
• Keep its temperature high enough.
• Provide a few examples but not too many.
• Be liberal in constraining its response format.

Evolution Built Layers, Not Monoliths

Some parts of our brain are less prone to error. For routine actions, our brain can switch to an “autopilot” mode where mistakes are rare. This is because the brain is not a single, monolithic decision-maker.

The brain’s decision-making system was built in layers over millions of years. It started with primitive structures like the mushroom bodies⁵ in insects, which integrate sensory input with past rewards. Mammals later evolved the basal ganglia⁶ to arbitrate between actions. At the top, humans developed a prefrontal cortex⁷ for planning, self-awareness, and abstract thought.

This evolutionary process resulted in a multi-modal decision architecture. Daniel Kahneman calls it System 1 and System 2 thinking⁸. System 1 is quick, intuitive, and provides good-enough answers, while System 2 is slow, rational, and more accurate — but costlier.

Our agents need the same flexibility, a mix of reflexive heuristics and deliberate reasoning.

💡 Takeaway: Architect your agent with sub-agents using thinking modes.

• Reflex: Use a small, fast model for lightweight heuristics in most cases.
• Reflection: Escalate to Reasoning models when uncertainty or stakes are high.

Humans Know When to Switch Gears

Humans are good at switching between chaos and focus. When we need to concentrate, the brain literally reduces neural noise⁹. When we need creativity, we let variability back in. Intelligence, in humans, is the ability to choose when to let randomness speak.

Agents can do the same. A good agent doesn’t just act, it monitors itself. Low confidence? Switch to deep reasoning or ask for clarification. High confidence? Move fast and commit.

💡 Takeaway: Build feedback loops.

• Feed your agent’s output back into itself for self-evaluation.
• Teach models to estimate how confident they should be in their own outputs (a.k.a. “uncertainty calibration”).
• Align a model’s confidence with reality with temperature scaling.
• Use a routing agent to let your agent switch strategy.
• Only respond to the user when the agent is confident.

Tool Use Is Intelligence in Action

The human revolution started in the Early Paleolithic (3.3 million years ago) with tools¹⁰ made of stone, bone, and wood. They made hunting more efficient and allowed for the development of complex societies. Later, humans used notebooks to extend memory, abacuses for calculation, and computers for simulation. Tools amplify the impact of our brains on our environment.

A GenAI model alone is probabilistic and creative, but not reliable. Combine it with deterministic tools (calculators, APIs, databases), and suddenly it’s both inventive and accurate. That’s what papers like Toolformer¹¹ are all about. Agents are LLMs + tools in a loop, which allows them to turn fuzzy thinking into precise action.

💡 Takeaway: Give your agent hands and instruments.

• Add tools for precise tasks (math, search, retrieval).
• Let the model chain tool calls to perform complex workflows.
• Use MCP servers to integrate third-party tools seamlessly.

Learning Comes From Being Wrong

Addiction¹², OCD¹³, and frontal lobe damage¹⁴ show what happens when biological decision loops go wrong: too much impulsivity or too much control breaks adaptability. Every brain learns through error. Dopamine neurons fire when outcomes are better or worse than expected. That prediction error is the engine of adaptation.

The same goes for agents: treat mistakes like bugs, and you’ll kill their ability to learn. If you design agents to notice, measure, and correct their mistakes — they’ll adapt.

💡 Takeaway: Don’t suppress errors. Capture them.

• Log agent output in production, especially errors
• Request user feedback to identify failure modes
• Include a self-critique tool for your agent to detect its mistakes
• Use reinforcement learning from human feedback (RLHF) to improve over time

Context is not Limitless

In modern life, we make an large number of choices each day — what to eat, what to wear, which emails to respond to, and so on. Research shows that making too many decisions can erode the quality of later decisions (this is called the hungry judge effect¹⁵). Too much information can also impair decision-making¹⁶.

LLMs are no different. Feed them too much context or conflicting data, and they’ll hallucinate or miss the needle in the haystack.

💡 Takeaway: Create decision filters and smart memory tools

• Add the ability to track context size and summarize when needed.
• Embed filtering instructions directly in prompts (what to ignore, what to focus on).
• Let them ask for missing info instead of processing everything blindly.
• In RAG, limit the number of documents retrieved and chunk size.

The Developer’s Playbook for Smarter Agents

What evolution and neuroscience tell us boils down to this: intelligence is not precision, it’s adaptive error management. If your AI never gets it wrong, it’s not thinking; it’s just memorizing.

Here’s a checklist for building agents that actually think:

• ✅ Combine creativity and control: let agents explore, then verify.
• ✅ Build feedback loops: learn from errors instead of avoiding them.
• ✅ Use tools wisely: delegate precision, keep LLMs for judgment.

Do these tips work? Again, AI and the brain are different beasts. But the parallels are striking. And if you look at the most successful AI agents today, like Claude Code, you’ll see these principles in action: they mix exploration and exploitation, use tools effectively, and adapt based on feedback (see Claude Code’s system prompt and tools for details).

Want to dig deeper? At Marmelab, we build agents the same way — fast learners, useful failures, adaptable minds. Check out our explorations on AI agents for more.

Footnotes

1. Kahneman & Tversky, “Prospect theory: An analysis of decision under risk”, Econometrica, 1979 ↩

2. Stein et al., “Variability in neuron firing is part of the signal”, Nature Reviews Neuroscience, 2005 ↩

3. Faisal, Selen & Wolpert, “Noise in the nervous system”, Nature Reviews Neuroscience, 2008 ↩

4. Chen et al., “Randomness and generalization in deep learning”, Information Sciences, 2024 ↩

5. Mushroom bodies - Wikipedia ↩

6. Basal ganglia - Wikipedia ↩

7. Prefrontal cortex - Wikipedia ↩

8. Kahneman, “Thinking, Fast and Slow”, 2011 ↩

9. Arazi et al., “Neural Variability Is Quenched by Attention”, Journal of Neuroscience, 2019 ↩

10. Human evolution: Refinements in tool design - Britannica ↩

11. Schick et al., “Toolformer: Language Models Can Teach Themselves to Use Tools”, arXiv, 2023 ↩

12. Bechara, “Decision making, impulse control and loss of willpower to resist drugs: a neurocognitive perspective”, Nature Neuroscience, 2005 ↩

13. Pushkarskaya et al., “Decision-making under uncertainty in obsessive-compulsive disorder”, PubMed, 2015 ↩

14. Damasio, “The somatic marker hypothesis and the possible functions of the prefrontal cortex”, Philosophical Transactions of the Royal Society B, 1996 ↩

15. Hungry judge effect - Cambridge University Press, 2018 ↩

16. CHE Jingshang et al., “Information overload damages decisions”, Advances in Psychological Science, 2019 ↩