Trust is the bottleneck: why we built a data model, not another chatbot

A number that's wrong, and nothing goes red

Open Claude or ChatGPT right now and ask it to write a Python script that pulls last month's spend from the Meta Ads API. You'll have working code in under a minute. Ask it to add Google Ads, LinkedIn and TikTok. Still under ten. The barrier to getting marketing data has genuinely collapsed, and that's a good thing.

Now feed all that data back in and ask a question a client would actually ask: "What was our blended cost per acquisition last month across every channel?"

You'll get a number. It will be confidently formatted, probably to two decimal places, possibly with a nice little commentary about efficiency trends. Nothing will error. No pipeline will fail. No cell will turn red.

And there's a good chance it's wrong — because somewhere in that answer, Google's cost in micros got added to Meta's cost in dollars, or a conversion counted by Meta's 1-day-view window got stacked on top of a Google data-driven conversion, or Google Business Profile got a zero for "clicks" instead of being left out of a metric it doesn't report.

The reason this is dangerous isn't that AI makes mistakes. It's that the only person who can catch this particular mistake is someone who already knows the answer — which rather defeats the point of asking.

Data is not software

Anthropic's data team put their finger on why this happens in their write-up on self-service analytics, and it's the cleanest framing we've seen: data is not software.

When an AI writes application code, the world pushes back. A compiler throws. A test fails. A page won't load. The feedback loop is immediate and mechanical, which is exactly why coding agents have got so good so fast — hallucinations get caught by the environment, not by a human reading every line.

When an AI writes an analytics query, nothing pushes back. The SQL is valid. The rows come back. The chart renders. As Anthropic put it, there's often only one correct answer and no deterministic way to prove you got it. Their team named the failure mode that swallows most errors: concept-to-entity ambiguity. The model can't reliably map the words in your question to the right things in your data.

This is not a model problem

The tempting response is to wait it out. Models get better every few months; surely this fixes itself?

The evidence says no. In April 2026 Cube published a paired benchmark: 100 natural-language questions over a retail dataset, run against three frontier models under two conditions. In one, the model got the raw warehouse schema. In the other, it got the same schema plus a four-kilobyte markdown document describing what the measures meant. No code, no runtime, no tools. Just the document, pasted into the prompt.

Every model improved by 17 to 23 percentage points. But the far more interesting result is what happened between the models: with the definitions present, the three models became statistically indistinguishable from each other. Without them, also indistinguishable — just worse. Nearly all the variance came from whether the definitions were there. Almost none came from which model you picked.

If your team hasn't agreed on what a conversion means, no amount of model upgrading will produce a correct answer. The gap is in the context, not the model.

Anthropic's numbers point the same direction from a different angle. Without curated domain knowledge, their internal analytics agent didn't clear 21% on their evaluation set. With it, they run consistently above 95%. Same models. Different context.

This is the part worth sitting with if you're deciding what to build: this is an architecture decision, not a model decision. Get the definitions right and you can choose whichever model suits your budget and latency. Skip them and the most expensive model available won't save you.

Why marketing data is a worst case

Everything above applies to any warehouse. Marketing data is harder than most, for four specific reasons.

1. The same concept has a different name on every platform

"Spend" is the simplest metric in advertising, and no two platforms agree on how to express it. Google Ads reports metrics.cost_micros — the cost multiplied by a million, as an integer. Meta reports metrics.spend as a plain decimal. LinkedIn reports costInLocalCurrency. X reports total_spend_micro. Two of those need dividing by a million before they can be added to the other two, and one of them may not be in the currency you think it is.

An LLM handed four raw API responses has to infer all of that from field names. Sometimes it does. Sometimes it adds a micro figure to a dollar figure and reports a campaign that spent nine hundred million dollars, which at least you'd catch. The dangerous version is the one that's only slightly wrong.

2. Some platforms don't have the metric at all

Google Business Profile doesn't report "clicks." It reports website clicks, call clicks and direction requests as separate things. It doesn't report impressions either — it reports searches and map views. If you want a cross-channel clicks number that includes local, someone has to decide what counts, and then apply that decision the same way every single time.

Left to an LLM, that decision gets made fresh on each request, and differently depending on how the question was phrased.

3. Conversions from two platforms are not the same unit

This one is genuinely unsolvable by naming alone, and it's worth being blunt about. Ad platforms attribute a conversion back to the date of the ad interaction. GA4 records it on the date it actually happened. So the same purchase can land on two different days depending on which system you ask, and the further apart your click and your conversion, the worse the divergence.

Attribution windows compound it. In January 2026 Meta removed the 7-day-view and 28-day-view attribution windows entirely. Google Ads defaults to data-driven attribution. Adding those two conversion counts together produces a number, but it isn't a measurement of anything.

4. It all changes underneath you

Which brings us to the part everybody underestimates. More on that shortly.

The four options, honestly

We build one of these things for a living, so treat the following with appropriate suspicion — but we'd rather lay out the actual landscape than pretend there's one answer. There are four sensible ways to get AI-assisted answers out of marketing data, and three of them are legitimately the right call for somebody.

Why a stack of MCP servers isn't a data model

Option 1 deserves its own section, because it's the one most agencies are quietly trying right now and it's the one where the gap is least visible.

MCP is genuinely excellent. It solved a real problem — how an agent reaches a system it wasn't trained on — and Jepto runs its own MCP server precisely because we think that's the right shape. But an MCP server is a transport. It hands the model the platform's own vocabulary, unchanged. Connect five of them and you haven't built a cross-platform model; you've given one agent five phrasebooks and asked it to negotiate a treaty.

What actually happens when you ask a five-MCP setup for blended ROAS:

• The join happens in the context window. Raw rows from five platforms get pulled into the conversation, and the model does the arithmetic in prose. That's the least deterministic place in the entire system to perform a sum.
• Nothing forces consistency between sessions. Ask the same question on Tuesday and you may get a different reconciliation, because nothing wrote the first one down. Cube calls this the difference between descriptive context and executable context: a description helps the agent choose, but only an executable definition makes the chosen answer correct.
• Access rules are advisory. A permission the model is merely told about in a prompt is a permission it can reason its way around. In a governed layer, the rule is applied when the query is compiled, before any data moves.
• It gets expensive in exactly the wrong way. Cross-platform questions mean pulling large volumes of raw rows into context on every single ask, then paying a per-token rate to have a language model add them up.
• It fails hardest on the questions you care most about. Single-platform questions work well — the agent is just reading one system's own numbers back. The failures cluster on cross-channel questions, which are the ones clients actually ask.

When code is free, maintenance is the whole bill

This is where the build-versus-buy conversation has genuinely shifted, and where we think most in-house projects go wrong in 2026.

Historically, "we'll build it ourselves" was gated by the cost of writing the extraction code. That gate is gone. An AI can write a competent Meta Ads connector in an afternoon. So the project gets approved on the strength of a build estimate that is now, accurately, very small.

The problem is that the build was never the expensive part. AWS has been making this argument about infrastructure for years through its total cost of ownership framing: real TCO is direct costs plus indirect costs plus operational costs, and operational labour is a first-class line item, not a rounding error. Ownership costs are always significantly greater than acquisition costs. Marketing data just makes the ratio unusually brutal.

What "API maintenance" actually looked like in 2026

This isn't hypothetical, and it isn't occasional. Here's a partial list of what a marketing data pipeline had to absorb in the first seven months of this year alone.

Google Ads moved to a monthly release cycle in January 2026, lifting major versions from three a year to four, each supported for roughly twelve months. That sounds generous until you notice what it means in practice: a major version reaches end of life every few months, and when it does, requests simply begin to fail — no grace period, no soft degradation. v19 died on 11 February. v20 in June. v21 on 5 August. v22 is scheduled for October.

Meta's changes were subtler and, honestly, worse.

The dangerous API changes aren't the ones that break your pipeline. They're the ones that don't.

Look again at the two entries marked silent. When Meta removed the 28-day view-through window, nothing failed. Your sync ran. Your dashboard rendered. Your year-on-year comparison quietly started comparing two different definitions of a conversion, and your client's ROAS trend developed a step change that nobody could explain because nobody was looking at a Meta changelog.

A breaking change is a ticket. A silent semantic change is a credibility problem six weeks later, in a meeting.

And at the end of all that, you have data — not answers

Here's the part that catches teams who do successfully build the pipeline: you're now roughly where you started this article. You have clean rows in a warehouse and an LLM that still doesn't know what your agency means by a qualified lead.

The obvious shortcut is to have the AI generate the semantic layer too. Anthropic tried exactly that — bootstrapping metric definitions from raw tables and query logs — and reported that it produced plausible-looking definitions that encoded the very ambiguities they were trying to eliminate, and scored worse on their evaluations than a smaller, human-curated layer. Their conclusion: generate the documentation with AI, but have a human own the definition.

That's the sentence we'd underline for anyone scoping an in-house build. The valuable artefact is a set of human decisions, and AI can't make those for you. It can only help you write them down faster.

What we built, and why it's shaped like this

The Jepto Data Model is our answer: a governed catalogue of every metric and dimension available for a client, in three layers.

A few decisions inside that are worth calling out, because they're the ones that took the longest to get right.

Labels speak marketing, keys speak machine. Every field carries a human label matching the language you'd see in Ads Manager, and a stable key like googleAds.clicks underneath. You pick by label; cards and agents resolve by key. That separation is what lets us change how a metric is computed without breaking every report that uses it.

Aggregation lives on the field, not the chart. How a metric rolls up — sum, average, count distinct — is a property of the metric itself. It's the kind of thing that seems fussy until you've seen an averaged rate metric summed across a hundred campaigns.

Missing means excluded, not zero. If a platform doesn't report a concept, it contributes nothing rather than a zero. Google Business Profile isn't in the spend total. It's a small decision that keeps blended averages honest.

Definitions are account-wide by default when you want them to be. A custom field created under All Clients is available to every client, so the definition of "blended paid spend" is written once and can't drift between the report your senior strategist built and the one a new starter built last week.

The second trust problem: nobody can read the SQL

Suppose the AI gets it right. There's still a question sitting underneath, and it's the one that comes up every time an agency starts letting AI near client reporting: how would you prove it?

The stock answer is "show the SQL." Most tools now do — a little expandable panel under the chart with the generated query in it. It looks like transparency. In practice it's theatre, because the person who most needs to check the number is an account manager or a client, and they cannot read SQL. Neither can most senior strategists, and there's nothing wrong with that; it isn't their job.

Even for people who can read it, the format works against them. OpenAI made this point vividly by publishing one of their own analysts' queries — a 180-line statement — with the note that it isn't easy to tell whether you're joining the right tables and querying the right columns. Anthropic put the structural version of the same problem plainly: the end user of the data model is now an agent acting for someone who isn't a data expert, so the result can't require the user to validate the underlying correctness, because the user doesn't know how.

That's the audit problem. An audit trail nobody can audit is just a longer receipt.

What an auditable request looks like

The alternative is to make the request itself the artefact, not the SQL it compiles into. In Jepto, every chart and table stores a data query — the saved recipe behind the card, visible in a Query tab beside the preview. It's declarative: it says what you're asking for, not how to fetch it. There are no joins in it, no unit conversions, no grain decisions, no timezone handling — all of that lives in the data model, resolved once, the same way for everyone.

Which means the thing you check is short enough to check.

WITH g AS (
  SELECT campaign_name,
         SUM(cost_micros)/1000000 AS spend,
         SUM(conversions)         AS conv
  FROM   google_ads_campaign_daily
  WHERE  segments_date BETWEEN @from AND @to
  GROUP  BY 1
), m AS (
  SELECT campaign_name,
         SUM(spend)       AS spend,
         SUM(conversions) AS conv
  FROM   meta_ads_insights
  WHERE  date_start BETWEEN @from AND @to
  GROUP  BY 1
)
SELECT COALESCE(g.campaign_name, m.campaign_name) AS campaign,
       COALESCE(g.spend,0) + COALESCE(m.spend,0)  AS spend,
       COALESCE(g.conv, 0) + COALESCE(m.conv, 0)  AS conv
FROM   g FULL OUTER JOIN m USING (campaign_name)
WHERE  campaign_name LIKE '%Brand%'
ORDER  BY spend DESC
LIMIT  10;

{
  "metrics": [
    { "field": "unified.spend",       "aggregation": "sum" },
    { "field": "unified.conversions", "aggregation": "sum" }
  ],
  "dimensions": [
    "unified.source",
    "unified.campaignName"
  ],
  "timeDimension": {
    "field":       "unified.date",
    "dateRange":   "last30days",
    "granularity": "day"
  },
  "filters": [
    { "field":    "unified.campaignName",
      "operator": "contains",
      "values":   ["Brand"] }
  ],
  "comparison": "previousPeriod",
  "order":      { "unified.spend": "desc" },
  "limit":      10
}

Two properties of this fall out for free, and both matter more than they sound.

Disagreements become diffs. Because identical queries always produce identical numbers, "these two reports don't match" stops being a forensic exercise. You open both Query tabs and look at what's different — a stray filter, a different date range, a source-specific field where a unified one was intended. It's usually a thirty-second job.

AI output gets reviewed at the right altitude. When you describe a chart in plain language and let AI build it, the thing you check afterwards is the query, not the SQL. That's a review a marketer can actually perform. Our own docs are blunt about this: AI drafts are a starting point, and the Data tab is the source of truth.

Worth saying plainly, though: readable is not the same as right. A data query being legible doesn't mean someone read it. It just removes the excuse that nobody could.

Write once, use anywhere

Here's the part we care most about, and it's the reason the data model is deliberately decoupled from any one interface.

Historically, semantic models lived inside a BI tool. If you wanted the logic, you used that tool. Cube's Artyom Keydunov puts the objection well: a context layer that ships as a feature of a single BI tool wraps trust around a seat you're already locked into. Anthropic reached the same conclusion internally — their rule is that the same knowledge must produce the same answer in Slack, in the IDE, in a dashboard tool and in a standalone agent session. OpenAI's data agent is available in Slack, on the web, inside IDEs, and in ChatGPT through an MCP connector.

Three organisations with unlimited engineering resources, solving this independently, and all three landed on the same shape: one governed core, many front doors.

The practical version for an agency: a strategist can build a client dashboard in the report editor, a junior can ask Data Chat what happened to cost per lead last week, and you can point Claude at the MCP server to diagnose a sudden performance drop at eleven o'clock at night — and all three are reading the same definition of cost per lead. Nobody has to reconcile anything, because there was never more than one number.

What a data model can't do

It would be a poor learning piece if we stopped at the good bits. Four honest limits.

It can't decide what you mean. A semantic layer records a decision; it doesn't make one. If your agency hasn't agreed whether a lead form submission counts as a conversion, the model will faithfully encode whichever answer you type into it. That conversation is still yours to have, and it's the highest-leverage hour you'll spend on reporting all year.

Normalising names doesn't reconcile attribution. We can make Google's and Meta's spend directly comparable, because money is money. We cannot make their conversions the same unit, because they're measured with different windows and different models. Blended ROAS is a useful directional number and a bad forensic one, and any tool that tells you otherwise is selling.

Bad inputs stay bad. Inconsistent UTM tagging, conversion actions counting the same event twice, a GA4 property with a broken key event — a governed layer makes those failures more visible, not less real.

It doesn't remove the human check. Anthropic runs adversarial review, provenance footers and daily sanity checks against blessed dashboards, and still describes silent wrong answers as the failure mode they haven't fully solved. Anyone claiming they have is not being straight with you. Read anything before it goes to a client.

You don't need to reinvent the wheel

The AI wave didn't make data engineering less important. It made it the whole game. An agent is only as good as the structure you hand it, and the benchmark evidence is unusually consistent on this point: the definitions move accuracy far more than the model does.

What genuinely changed is which part is worth your time. Writing the extraction script used to be the hard bit, and it's now nearly free — which is precisely why it's no longer where your advantage lives. The advantage is in the decisions: what your agency counts as a conversion, how you blend paid channels, which metrics you'll defend in a client meeting. Those are yours, they're worth writing down carefully, and no model will generate them for you.

The plumbing underneath them is a solved problem. Someone is already maintaining it, watching the sunset calendar, and migrating the field names when Meta changes its mind. That doesn't have to be you.

Own your definitions. Rent the pipes.

Want the detail? The Data Model and Unified Fields docs list every field mapping we use, platform by platform — including the gaps.