Sonatafy
An AI feedback loop breaking apart

Confidence Is Not Capability: Steve Taplin on Breaking the AI Echo Chamber

November 16, 2026Senior ExecutiveRead the full article on SeniorExecutive.com

Sonatafy CEO Steve Taplin was featured in "AI Training Data: How to Break the Model Echo Chamber," published October 30, 2026 by the Senior Executive AI Think Tank. The piece asks a question more AI leaders should be asking out loud: as models learn more and more from content other models produced, are they getting smarter, or just getting better at repeating themselves?

Steve's answer is short. A model can grow more confident without growing more capable, and from the outside those two things can look identical.

Read the full article here: Read the full article on SeniorExecutive.com

Below is what Steve said, and why we think it matters to anyone responsible for AI that is already running in production.

When a model starts agreeing with itself

The article opens with a 2024 study published in Nature. Researchers found that repeatedly training models on AI-generated output can cause them to lose information from the original data, and the rarer patterns tend to disappear first. They call it model collapse.

Synthetic data still has real uses. It can simulate rare events, fill in specialized tasks, and test situations that would be too expensive or risky to recreate in the real world. The danger is the closed loop, where a model keeps learning from material that only reflects what it already believes.

Steve put it this way:

"The biggest risk with synthetic data is an AI echo chamber, where models grow more confident without becoming more capable," Taplin says. "Synthetic data isn't the problem. Unvalidated synthetic data is."

That distinction is the whole point. Synthetic data is a tool. What determines whether it helps or hurts is whether anyone checked it against something the model did not produce.

What makes a synthetic example worth training on

In the article, Steve sets two conditions. A valuable example has to be grounded in something independent: outside evidence, expert review, or a verifiable outcome, like code that actually passes its tests. And it should go after the edge cases where the model struggles, not reinforce the areas where it already performs well.

That gives any team a quick filter. Can something outside the model confirm this example is correct? Does it teach the model something it gets wrong today?

If the answer to both is no, the data adds volume. It does not add capability.

The AI Validation Gap

Steve gave the problem a name in the article: the AI Validation Gap. It describes AI output reaching production without evaluation that decision-makers trust.

It is one of four diagnostic frameworks Sonatafy uses with product and engineering leaders. In plain terms, it sounds like this: we spent on AI and cannot point to the return.

Synthetic data feeding a closed loop is one version of the gap, at the training level. The same pattern shows up inside companies deploying AI, not just the labs building models. Success gets defined as “we released the model” instead of “it produces correct outputs.” Evaluation turns into a vibe check.

Then the symptoms appear. Hallucinations reach production undetected. Model drift goes unwatched. AI features get quietly pulled after launch. A board asks whether the AI is working, and nobody has data behind the answer.

The question that actually matters

The line from the article we keep coming back to:

“The question isn’t how much synthetic data we generate,” he says. “It’s whether we can independently prove the next model is better.”

Steve's point is that evaluation has to be built into development, deployment, and ongoing monitoring. It cannot wait until a system is already in use.

For leaders who own AI in production, that turns into a handful of questions worth putting to your team this quarter:

  • What independent check tells us this version is better than the last one?
  • Does anyone outside the team that built it trust that check?
  • Where did the correct answers in our test cases come from?
  • Would we notice if performance slipped next month?

If those answers are vague, more data will not fix it. Validation will.

Where to start

If your AI is already live and nobody can show, with evidence, that it is working, that is the AI Validation Gap. Sonatafy's AI Validation Assessment is built for exactly that situation. It looks for evidence that your AI is accurate, monitored, governed, and producing measurable value. Fee and scope are set in a scoping conversation.

If you are earlier in the journey, the AI Readiness & Value Snapshot is complimentary. It is a 30 minute executive session with a senior US based Sonatafy leader, followed by a one page written read within two business days. The question it answers: where can AI create measurable value, and are you actually ready to implement it?

Either way, start with the question Steve asked. Can you independently prove your next model is better than the one you have?