The symptom

You point Claude or Gemini at a client's data, ask the kind of question that would take an analyst half a morning, and get an answer in four seconds. It reads well. It has a number in it. Nobody in the room can tell you whether it is right.

A week later somebody checks. The revenue the model reported for paid social does not match what the client sees in their own back office, or the top performing segment it named has been switched off since spring. Nothing was flagged along the way, because a model cannot tell you that what it was handed was incomplete.

That is the part agencies underestimate. A broken tag throws an error, a failed pipeline sends an email, a missing column breaks a query. A wrong AI answer arrives as a fluent sentence with the same confidence as a correct one, and from there it walks straight onto a client call.

Why it happens

In almost every setup we look at, the model is the only component behaving exactly as documented. Three things underneath it are not, and they fail in different ways, so it is worth knowing which one you have before you promise a client anything.

The rows that were never collected

An answer can only be as complete as the dataset behind it. On a typical account, 15 to 30% of conversions are never captured at all. Ad blockers strip 30 to 40% of tracking events before they leave the browser, browser tracking prevention cuts cookie lifetimes short enough that a returning visitor looks like a stranger, and consent handling drops more on top. We took that apart in why conversions never reach GA4.

Here is what it does to an AI answer. The model has no concept of a conversion that was not recorded, so it treats the total it was given as the total that happened. Ask which channel drives the most revenue and it will rank the channels by how well they were measured, not by how well they performed. The channels that lose the most signal, usually the ones carrying Safari and in-app traffic, come back looking weaker than they are, and nothing in the output hints at it.

Tables that arrive already summarized

The second failure is quieter, because the data exists, just not at the resolution the question needs. If your AI setup reads GA4 through the reporting interface or the Data API, it inherits every artifact of that layer.

Two of those are documented by Google and worth knowing by name. Sampling: reports and explorations switch to sampled data once a query exceeds the property quota, and Google Analytics Help puts that quota at "10 million events for standard Google Analytics properties". Cardinality: an (other) row appears "when the number of rows in a table exceeds the table's row limit", and per Google's documentation any dimension with more than 500 values should be treated as high-cardinality. That covers most landing page and campaign dimensions on a mid-sized account. The values pushed into (other) are exactly the long tail that a question about underperforming campaigns is trying to surface.

An analyst sees the sampling icon and the (other) row and adjusts the conclusion. A model reads both as data.

Columns nobody ever defined

The third failure produces the most convincing wrong answers. Say the dataset is complete and event level. There are still four columns with revenue in the name, test events from a staging container sitting next to real ones, two conversion actions counting the same lead, and a timezone that differs between the ad platform tables and the analytics tables. None of that is written down anywhere.

So the model guesses. It picks a revenue column, joins two tables on the field that looks like it should join them, and decides for itself what counts as a conversion. Every one of those guesses is invisible in the answer, and a guess that lands on the wrong column still returns a number in exactly the right shape.

This is not one vendor's weakness either. The BIRD benchmark, which tests systems against large, messy, real-world databases instead of tidy sample schemas, still puts the best submitted systems below the human baseline set by data engineers, and the distance grows on multi-step questions where each step depends on the last. Those benchmark databases come with documentation. Your client's BigQuery project probably does not.

What good looks like

The repair runs in the same order as the failures, and not one of the three steps involves changing model.

Establish what is missing before you trust any answer. Measure per account what share of conversions actually arrives, which browsers and channels lose the most, and where consent handling drops events. That is a measurement job rather than an AI job, and it is what a tracking audit covers. Until it is done, every AI answer is a confident statement about a dataset of unknown completeness.

Give the model events, not reports. Sampling and the (other) row belong to the reporting layer, so the way past them is event level data in a warehouse. Archon Pixel captures around 25% more than a standard GA4 install, and that extra 25% is mostly long tail, which is precisely the part AI questions go looking for.

Define the dataset once, in the data, not in the prompt. One agreed definition per metric, every column documented, test traffic excluded, PII stripped before anything queries it. That is the job Archon AI does: materialized, documented, PII-free BigQuery datasets that Gemini, Claude and MCP based tools can query without guessing. The prompt stops carrying the business logic, because the dataset carries it instead.

What that is worth compounds, because everything above it inherits the fix. For one performance agency we worked with, cleaning up the measurement layer brought 38% of affected client traffic back into measurement, lifted measured conversions by 26% on average, and saved 14 hours per project against handling it in-house. AI on top of that answers questions. AI on top of raw tables invents them.

Frequently asked

Can we not fix this with better prompts?

Up to a point. Tell a model which revenue column to use and it will use it, for that one question, in that one chat. What no prompt reaches is a conversion that was never recorded or a row that was collapsed into (other) before the model saw it. Prompting also moves your business logic into a place nobody reviews and nobody versions.

We already have the GA4 BigQuery export, is that not enough?

It is the right foundation and it is not the finished job. The export gives you event level rows, which removes the sampling problem, and it arrives as nested event parameters with no column definitions, no exclusion of test traffic, and no agreement on which revenue field is the one you report. That is what a model trips over.

Which model should we use for this?

It is the least important decision in the chain. The frontier models are close enough to each other that swapping one for another will not turn a wrong answer into a right one when the cause is missing rows or undefined columns. Put the effort into the dataset, then use whichever tool your team already works in.

Is client data safe if we point a model at it?

It is when the dataset is built for it. Strip PII before anything queries it, keep the datasets in the client's own BigQuery project, and expose modelled tables rather than raw event streams. What a model might retain stops being a governance question once there is nothing sensitive for it to see.

A client already acted on a wrong AI number, how do we handle it?

Correct it with the cause attached, not just the number. "The model read the sampled report instead of the event data" is a sentence a client can accept once. "The AI got it wrong" invites them to distrust everything you present next quarter, including the parts that were right.

If you want to know how complete a client's data really is before you put AI on top of it, request a free tracking audit and we will measure it per account.