The symptom

Reporting week arrives and somebody has a better idea. Export the analytics tables, drop the ad platform numbers in beside them, hand the lot to an AI tool with "write the monthly commentary for this client". A minute later there is a draft, structured, readable, in better English than the version written at 11pm last month.

So it goes out. The client reads one line, paid search softened in the second half of the month, and asks what happened. Nobody re-checks it, because the sentence around it sounds considered. Two weeks later that number has moved in the platform and the explanation no longer fits.

The model did not write badly. It wrote fluently about a number that was provisional, partly withheld, or measuring something other than what the sentence claimed. No prompt tuning reaches that, because the problem arrived before the prompt did.

Why it happens

An AI report writer is a language layer over whatever you hand it. It reads rows and produces sentences, with no independent view of the platform, no memory of what the figure said yesterday, and no way to separate a measured number from an estimated one. Three properties of marketing data make that worse than it sounds.

The warnings stay behind in the interface

GA4 does tell you when a number is not what it appears to be. It tells you in the interface, not in the data. Data thresholding withholds rows to stop anyone "inferring the identity or sensitive information of individual users", and announces that as an indicator on the card. Sampling starts on a standard property once a query exceeds 10 million events, signalled by an icon carrying the percentage of data used. Any dimension with more than 500 values counts as high cardinality, and its tail is condensed into a single "(other)" row.

Copy that table into a sheet and every one of those signals is left behind. What reaches the model is tidy columns with nothing to indicate that a row is missing, a total was scaled up from a subset, or a third of the landing pages are hiding in one bucket. Google's note on the (other) row is blunter still: you can see the warning "but no (other) row appears in the report", because a filter hid it. A human who has run that account for a year reads past that. A model has no year.

The numbers are not finished when the report is written

Monthly reports get written in the first days of the following month, the worst possible moment to freeze a number. Google states that "data processing can take 24-48 hours" and that during that window data in your reports may change. The BigQuery export has a longer tail: Google updates the daily tables "with events for those dates for up to three days after the dates of the events", and events arriving later are not recorded.

Google Ads moves in a different direction again. Its primary conversion columns are, per Google's documentation, "calculated based on the time of the click, not the time of the conversion". A conversion landing this week is written back to the click that produced it last week, so last month's total keeps climbing for as long as the conversion window stays open. A closed month is not closed.

Underneath sits the input problem: roughly 15-30% of conversions are never captured at all, and where consent is refused GA4 fills part of that space with modelled key events. Google's behavioural modelling documentation is explicit that these appear only when there is high confidence of quality, and go unreported without enough traffic. A segment can appear one month and vanish the next with no campaign change behind it.

The model cannot flag what it cannot see

This is not a swipe at the tools, it is what the vendors publish. Anthropic's documentation notes that "even the most advanced language models, like Claude, can sometimes generate text that is factually incorrect or inconsistent with the given context". Its recommended fixes are all grounding, letting the model say it does not know and making it quote its source, and it says plainly that those techniques do not eliminate the problem entirely.

Grounding works on the source you supplied. Ground an explanation in a wrong number and you get a well cited wrong explanation, harder to catch than an obviously invented one. The tone stays level whether the input was solid or scraped off a sampled screenshot, and that flatness is the opposite of what a reviewer needs.

What good looks like

The agencies getting real value out of this all made the same split. Arithmetic comes from a governed source, narrative from the model, and nobody asks the model to judge its own input.

Every metric has one definition, agreed before an AI sees it. If "conversions" means one thing in the ads export and another in the analytics table, a model handed both will pick one and write a confident sentence about the gap. That reconciliation is a data modelling job, done once per client.

Caveats travel with the figures. If a number was sampled, thresholded, modelled or pulled inside the three day window, that belongs in the same row as the number, not in an icon two systems away. Then the model can be told to surface it and a reviewer sees it in the draft.

You hand over documented aggregates, not a dump of everything. This is where the privacy question gets answered, and it is governance, not tool preference. Passing client data to an AI vendor makes that vendor a processor: GDPR Article 28 requires a written contract covering the subject matter, duration, nature and purpose of the processing, binds the processor to documented instructions, and forbids a sub-processor without the controller's authorisation. Terms also differ by tier, not by brand. OpenAI states that data sent to the OpenAI API is not used to train or improve its models unless you opt in, which is a statement about the API, not about every product that company sells. Check the tier your team pastes into.

All three need the same thing underneath: a modelled, documented, personal-data-free dataset the reporting layer reads from. That is what Archon AI builds, materialised BigQuery datasets with defined columns and consistent metric logic, in the client's own project, designed to be queried by a model rather than screenshotted for one. The performance agency in our case study saved 14 hours per project once reporting stopped being reassembled by hand.

Where the line falls depends on the client. For a small account on one channel with settled definitions, a careful prompt over a clean export is genuinely fine, and building a warehouse first is an expensive way to save an hour. The case for the foundation grows with the number of sources being reconciled, and with how much budget moves on the report.

FAQ

Which parts of a client report can we safely automate today?

The parts that need no judgement about whether a number is trustworthy: drafting commentary from figures a human has validated, summarising what changed against last month, rewriting technical findings for a non-technical reader, holding tone steady across twenty accounts. What stays human is the check on any figure that leaves the building, and any sentence assigning a cause to a change.

Is it safe to paste client data into ChatGPT or Gemini?

It depends on the tier, the contract and what is in the data, and your client is entitled to a straight answer on all three. Under Article 28 the vendor becomes a processor and needs a written agreement, and sub-processors need authorisation. The safer pattern is to keep personal data out of what you hand over, which is far easier when the reporting dataset was built without it.

If the dataset is clean, will the model still invent numbers?

Less often, and not never. Vendor guidance is consistent that grounding reduces hallucination rather than removing it, so review does not disappear when the data improves. What changes is what you review for: an interpretation to sanity-check, rather than a set of figures to re-derive.

Do we need a warehouse before AI reporting is worth trying?

No, and starting small is how you find out where your definitions disagree. Run it on one client and note every place the AI draft differs from the report you would have written. Those differences are your data problems written down, usually the same handful repeating across the client base. We covered why AI gives wrong answers on marketing data and what sampling does to client reports separately.

If you want to know whether your clients' data could survive being handed to a model, that is what a free tracking audit answers.