The problems with vibe-coded MMMs

A bridge designed by a structural engineer and one designed by someone who used an AI tool to generate blueprints might look identical in a photograph. Both have the right number of spans. Both clear the required width. The difference doesn't show up at the ribbon cutting. It shows up when the bridge is under load.

Budget decisions guided by a flawed MMM work exactly the same way. The numbers look plausible. The charts are clean. The model even produces confident recommendations. The problem only becomes visible later, when you've scaled spend in the wrong direction or cut a campaign that was quietly holding up your funnel. In a marketing environment where every budget dollar is under scrutiny, measurement that fails silently is a serious liability.

If you're not already familiar with what vibe-coded MMMs are, this article on the topic is a good starting point. The short version: they're marketing mix models built by prompting AI coding tools to generate code on top of open-source MMM frameworks, often without a data scientist or marketing scientist involved. The problems with these models aren't random or unpredictable. They're structural, inherited directly from the frameworks they're built on, and the person who built the model may not know enough to recognize them.

Key takeaways

• The failure modes of vibe-coded MMMs are predictable because they inherit known structural limitations from the open-source frameworks they're built on
• Forced saturation assumptions baked into most open-source frameworks can lead to systematic underspending on campaigns that still have room to scale
• Channel-level attribution, rather than campaign-level, makes it impossible to identify which specific campaigns to cut or grow
• Attribution instability, where outputs shift noticeably with small configuration changes, signals a deeper problem that more data and tuning cannot fix
• Missing halo effects means upper-funnel campaigns are routinely undervalued, which can lead to budget cuts that quietly damage lower-funnel performance
• A model that fails silently is more dangerous than one that fails visibly, because confident-looking bad recommendations will be acted on

The failure modes aren't random

One of the most important things to understand about vibe-coded MMMs is that their problems aren't flukes. They're predictable. Because these models are built on open-source frameworks with known structural limitations, anyone familiar with those frameworks can tell you in advance where they'll break down. The question isn't whether a given vibe-coded MMM will have these problems. It's whether anyone involved in building or selling it knows enough to recognize them.

There are four failure modes worth understanding in detail.

Forced saturation leads to underspending

Most open-source MMM frameworks use response functions that assume, by mathematical design, that all campaigns will eventually stop generating incremental returns. The diminishing returns curve is built into the model before it ever processes your data. This isn't a setting you can toggle off. It's a structural property of the model class.

When saturation is imposed by the model rather than observed in the data, budget recommendations cap out earlier than your actual marketing performance warrants. Prescient's own research has found that linear response patterns frequently outperform saturating parametric forms, and that defaulting to saturation can push recommendations toward systematic underspending. A vibe-coded MMM produces the same bias, and nothing in the build process surfaces it or flags it to the user. You'll receive a budget recommendation that looks reasonable and is built on an assumption that may not reflect your campaigns at all.

Channel-level attribution isn't granular enough to act on

Most open-source frameworks, and by extension most vibe-coded MMMs, produce attribution at the channel level. They can tell you that Meta drove a certain share of your revenue last quarter. They can't tell you which Meta campaigns were responsible, or how your prospecting campaigns compare to your retargeting campaigns.

In practice, marketers don't cut channels. They cut campaigns. The decision to pull back on a specific underperforming creative, or to double down on one prospecting campaign over another, requires campaign-level data. Channel-level attribution produces a number that sounds useful but can't actually drive the decisions that move the needle.

The compounding problem is that channel-level models also can't distinguish between campaigns that are genuinely driving performance and campaigns that are simply running during high-revenue periods. Without that distinction, there's no reliable way to know which campaigns are worth defending when budget conversations get difficult.

Attribution instability signals a problem that tuning can't fix

A well-documented complaint about open-source MMMs is that outputs shift noticeably when model configuration changes slightly. Channel contributions jump around. Budget recommendations reverse. This is commonly treated as a tuning problem, something to address by running more optimization iterations or adjusting model parameters.

It isn't a tuning problem. Attribution instability is a sign that the model's structure doesn't have enough information to produce a unique, reliable decomposition of what drove revenue. Different configurations of the model all fit the observed data equally well, which means the model is converging on one of many plausible-but-unverifiable explanations rather than the most accurate one. More data doesn't resolve this. More tuning doesn't resolve it. The only path forward is a model built with the structural capacity to represent how marketing systems actually work.

With a vibe-coded MMM, there may be no one involved with the statistical background to recognize instability as a structural signal rather than a configuration issue. The natural response is to keep tweaking until the outputs look stable, which produces a model that looks well-behaved and may still be systematically wrong.

Halo effects are invisible to most vibe-coded MMMs

Upper-funnel campaigns don't convert directly. Their impact shows up downstream: in branded search volume, in organic traffic, in direct visits, and in the performance of the lower-funnel campaigns running alongside them. Someone sees a TikTok prospecting ad, doesn't click, and searches for the brand by name three days later. That conversion gets credited somewhere else, and the prospecting campaign looks weaker than it is.

Open-source frameworks model channels independently and additively, which means these spillover effects either get absorbed into baseline demand or disappear entirely. The model has no mechanism to connect upper-funnel investment to its downstream revenue impact.

If an MMM can't see that an awareness campaign is driving branded search and direct traffic, it will undervalue that campaign. A recommendation to cut or underfund it will look defensible in the dashboard. The downstream effects on lower-funnel performance will be slow enough that the connection isn't obvious. For DTC brands where upper-funnel spend is already the most contested line item in budget conversations, a measurement tool that systematically undercounts its impact doesn't just fail to help. It makes the wrong case.

A model that fails silently is the most dangerous kind

Each of the problems above shares a common characteristic: none of them announce themselves. Forced saturation produces a budget recommendation, not a warning. Missing halo effects produce an attribution number, not a gap. Attribution instability produces a result that looks stable once you've finished tuning.

The real risk with vibe-coded MMMs isn't that they'll generate obviously wrong outputs. It's that they'll generate outputs that look confident and reasonable while the structural problems remain hidden. Marketers act on what the model tells them. If the model is wrong in ways that aren't visible on the surface, those decisions will be wrong too, and the feedback loop is slow enough that the model rarely gets blamed.

Where Prescient comes in

Prescient was built specifically to address the failure modes that open-source frameworks leave unresolved. Campaign-level attribution gives marketers the granularity to make real decisions. Flexible response functions let saturation emerge from data rather than be imposed by the model. Halo effects measurement connects upper-funnel spend to the downstream revenue it actually drives across organic search, branded search, direct traffic, and Amazon. And daily model updates mean the picture you're working from reflects what's happening now, not last quarter.

These weren't features bolted onto an existing framework. They were the reason Prescient's founding team decided to build from scratch in 2019, after assessing the available MMM landscape and concluding that none of it was built for how modern marketing actually works. If you want to see what that looks like in practice, book a demo.