AI didn't replace our analysts. It made them harder to replace

We've been building something at Fresh Egg. It's called Inspectre, and it's changing how quickly we can get from a question about digital performance to understanding what's actually happening.

It can interrogate data across multiple sources, investigate anomalies and follow the questions that naturally come next. But the interesting part isn't the AI itself. It's what happens when you put that technology in the hands of experienced specialists.

Keep reading as I explain what we've built, what we've learned from getting it wrong, and why we think it's an early glimpse of where intelligent digital experiences are heading.

Someone spots an 18% drop in purchases.

If you work in digital, you probably know what happens next.

Analytics starts digging. Paid media checks whether anything changed. Someone looks at the website releases. Tracking gets questioned. Messages fly around Teams or Slack. Maybe somebody changed something in GTM. Maybe it's consent. Maybe performance really has fallen.

Eventually, you get to an answer.

Inspectre is an AI-powered analytics capability we've developed at Fresh Egg. It connects to the data our specialists use to understand digital performance, including GA4, Search Console, Google Ads and offline conversion data, and lets us interrogate it using normal language.

You can ask it a question. Then another. Then follow the thread wherever it takes you.

And that changes things more than I expected.

Imagine asking: "Why did organic conversions fall last week?"

Inspectre can query the underlying data and return what it finds.

But analytics rarely ends with the first answer.

• Was the fall sitewide?

• Was it concentrated on particular landing pages?

• Did traffic change?

• Was the same thing happening in paid?

• Did anything else move at the same time?

You keep asking. Inspectre keeps investigating, retaining the conversation context as it works to get to the root cause.

Underneath it all, serious engineering is happening. Inspectre works out what data it needs, writes and runs SQL against BigQuery, and returns the findings with tables or charts where they're useful.

But you don't need to know SQL. You don't really need to care that it's writing SQL. You care that the question you had five minutes ago is already turning into something you can act on. For me, that's where AI in digital starts becoming genuinely interesting.

One of my favourite examples came from an alert showing purchases down by around 18%. That's enough to get people moving.

But Inspectre dug deeper. It found a drop in consent after a change to the site's consent banner.

What initially looked like a conversion problem was actually a measurement problem. That's an important distinction.

Finding the real cause quickly changes the conversation. We've had other cases where Inspectre helped diagnose tagging issues, including by taking a GTM JSON file to investigate what was happening in the implementation.

That's the kind of work I want this technology doing. Get through the mechanics of the investigation quickly. Give our specialists the evidence. Let them concentrate on what it means and what we should do next.

The chat interface is probably the easiest part of Inspectre to understand, but it isn't necessarily the bit I'm most excited about. It can also keep watch.

We run checks every morning, before anyone has opened a report. Here's the part people don't expect. That bit has no AI in it at all.

It compares each metric against what normally happens on that day of the week, works out how far outside normal the current figure sits, and ignores anything where the numbers are too small to mean much.

Deliberately boring maths. It can't invent a problem and it can't talk itself out of one.

The AI only turns up afterwards, when something needs explaining.

That split matters more than it sounds. The part that decides whether something is wrong is arithmetic you could check by hand. The part that works out why is where a language model earns its place.

It also changes the starting point.

Instead of discovering something unusual during a reporting meeting and starting the investigation afterwards, you can arrive at the meeting already knowing about it.

Potentially, already knowing why.

For a marketing or digital team with several channels, platforms, campaigns and agencies in play at the same time, that matters. There are already enough things competing for people's attention. Your data shouldn't rely entirely on somebody remembering to go and look for problems.

So that's the what's and whys covered – but how do you actually go about creating a content hub?

Find out in the second blog post in this series - How to Create an Effective Content Hub in 4 Steps.

If you've used AI much, you'll know it has a slightly unnerving ability to be confidently wrong. That isn't much of a problem when you're asking it to suggest somewhere for lunch.

It's a fairly substantial problem when somebody is about to make a commercial decision based on the number it just gave them. We discovered early on that you can't fix an AI tool by asking it nicely. You can write instructions telling a model what it should and shouldn't do. You can improve the prompt.

You can give it examples. At some point, you need to engineer the risk out instead. Arithmetic is a good example. We don't want the language model doing calculations that BigQuery can do deterministically. So it doesn't. BigQuery does the calculation, and the model works with the resulting figures.

That principle has spread into other parts of Inspectre. Where conventional software can give us certainty, we use it. Where AI is genuinely useful for exploring, interpreting and working with language and context, we use AI.

When a decision needs human judgement, a human is involved. It's less exciting than telling you we've built an autonomous AI analyst. It's also a much better idea.

Another problem is asking a generic AI model questions about your analytics.

• It doesn't know your business.

• It doesn't know that you changed your tracking six months ago.

• It doesn't know what your organisation actually means by a lead.

It doesn't know that one section of your website behaves differently from another, that a particular campaign uses an unusual naming convention, or that something which looks anomalous happens every year.

A good analyst accumulates that knowledge.

We've designed Inspectre to do the same. The system has deep layers of context, including knowledge specific to individual clients. We can update that context as we learn more, and client-specific knowledge takes precedence over generic assumptions.

That might sound like an implementation detail. In practice, it's fundamental.

The more relevant context you can give the system, the more useful the investigation becomes. And that knowledge doesn't have to disappear because somebody closed a ticket, finished a project or can't remember what happened nine months ago.

It can compound.

This was one of the first questions we had to take seriously. We're dealing with client data. “We've plugged it into AI” isn't an acceptable architecture.

The language model doesn't need to see all of the raw, row-level analytics data to answer a question.

There is more engineering behind this than I'm going into here, but that's intentional.

Inspectre is designed to make complicated things feel simple without pretending the complexity has disappeared.

The same applies to the data underneath it.

We've built our own reporting structures on top of the GA4 BigQuery export, including what we call the FE Mastertable. That gives Inspectre a reliable foundation and lets us enrich and correct the data where appropriate.

Here's a small example of what "correct" means.

GA4 quietly files a chunk of your paid traffic as "Unassigned" when the click identifier gets lost in a redirect. It's common, and most people never notice. Our reporting table puts it back where it belongs. If a visit carries the Google Ads click ID, it's paid traffic, regardless of what GA4 decided.

So when someone asks Inspectre how paid search performed, it answers from corrected data. Point a general-purpose AI tool at the same raw export, and it will confidently report a decline that never happened.

That's one correction. There are many, and they took years to accumulate. The foundations matter enormously. Put AI on top of poor data, and you've mostly developed a quicker way of being wrong.

This is the question people inevitably ask.

In our experience, the analyst becomes more valuable.

Getting to a good answer has always involved a lot. Pulling information together, querying it, checking it, validating it, tracking, comparing sources and working out whether the thing you're seeing is even real.

That work matters. We can now accelerate significant parts of it.

That means our analysts can reach the point where their expertise really counts much sooner, for example:

• Why did this happen?

• Does it matter?

• What else should we look at?

• What should the client do?

Those aren't database questions.

You need judgement. You need commercial context. Quite often you need knowledge from outside analytics altogether.

That's why I don't think the interesting story here is AI replacing analysts.

Give a good analyst technology like this, and you amplify what they can do by some considerable margin.

We're not building Inspectre because we think the world needs another dashboard or another SaaS login. There are plenty of those.

We've built it as part of how Fresh Egg works with and serves our clients. That distinction matters because Inspectre becomes far more interesting when you combine it with the people around it.

Our analytics specialists can use what it finds. Our CRO team can turn an insight into something worth testing. SEO can investigate what's happening in organic performance. Developers can get involved when the evidence points towards something technical.

The question doesn't have to stop when you reach the edge of one team's remit. That's closer to how digital problems actually behave, after all:

• A CMO doesn't have a "GA4 problem". They have a performance problem.

• A Head of Marketing doesn't really want an attribution report. They want to know where to put the next pound.

• A Head of Data doesn't need AI to produce more numbers.

They need people across the organisation to get more value from trusted data without creating chaos around it.

And a CTO probably isn't short of technology. They need technology that earns its place. That's the standard we're holding Inspectre to.

At Fresh Egg, we're working towards what we call Intelligent Experiences, or IX.

The premise is that digital experiences should get smarter and more effective as you learn more about the people using them and what produces results. Inspectre is an early, tangible part of that. It helps us shorten the distance between data and understanding.

Elsewhere, we're already using AI-assisted development to move from ideas to working solutions at speeds that would have sounded fairly ridiculous to me a couple of years ago. In CRO, we're looking at how technology can help us identify and test opportunities sooner.

I've personally used AI to get working solutions off the ground in around an hour that, approached conventionally, could have represented tens of thousands of pounds of development effort. That's not about making expertise cheaper.

It's about what happens when you give experienced people a serious accelerator. You can explore ideas that previously wouldn't have survived the business case. You can find out whether something works before months disappear into a project. And when the evidence tells you something needs changing, the gap between knowing and doing can become dramatically smaller.

For me, that's where this gets exciting.

Access to an LLM isn't particularly special anymore. The difficult bit is turning the technology into something you can trust with a real business problem.

That means having the right data underneath it, giving it the right organisational context and knowing where deterministic systems should take over—knowing where a human needs to stay involved. Then, when it finds something, you need the capability to do something useful with what you've learned.

We've spent a lot of time working through those problems with Inspectre. There have been dead ends. Some answers looked convincing but weren't. We've tried to solve some things with prompts, only to find they needed engineering.

That's why I'm comfortable talking about it now. It isn't a concept deck. We're using it.

And we're learning what becomes possible when you start removing the friction between a question and the people capable of doing something about the answer

Even if you never work with Fresh Egg or build anything resembling Inspectre, I'll give you one piece of advice for free. If you use GA4, turn on the raw BigQuery export. Do it now.

You can't switch it on in two years and ask Google to give you all the raw historical data you didn't collect.

AI capabilities are moving ridiculously quickly. Your organisation may have no immediate plans for this kind of analytics capability. Still, the data you retain today will determine what you can do tomorrow.

Inspectre has taught us that the shiny AI bit is rarely the whole story. The interesting part is what you can build around it.

For us, that's increasingly about acceleration: getting from a question to understanding sooner, getting from understanding to action. At the same time, it still matters to use what we learn to make the next decision better.

That's what we're building towards with IX. Inspectre is an early proof point. And if you're wondering what it could do with your data, your business context and your digital team, that's probably a conversation worth having.

I'll be talking more about the journey of building Inspectre over the coming weeks, including at MeasureCamp and BrightonSEO. If you’re there and want to talk about what we've built, what we've learned, or what something like this could do with your data, come and find me.