How to evaluate an email platform's AI agent

You can give any AI model a few brand colours, your guidelines and a tone of voice, and it will build you an HTML email. In a demo, that looks impressive. But the demo skips what your team has to answer before sending. Is it on brand? Can legal approve it? Does the personalisation still work? Does it render in Outlook?

A general-purpose model has nothing to measure its work against, so it can't iterate toward a campaign you could actually send. The result looks plausible, and your team has to check the HTML, the brand and the content. Done this way, AI adds to the checking you do.

Here are the nine questions I would ask of any agent, each with a test. Bring a real brief from last month to the demo, because a vendor's own template won't tell you much. There is a scorecard to take with you at the end.

1. Does it give you less work?

If your team spends more time reviewing the AI's work than they used to spend building the campaign, something is wrong.

2. Does it stay inside your design system?

The mistakes that worry me most are the ones nobody can see. Say an agent writes the HTML from scratch for every campaign. Even with a good prompt that reminds it what Outlook needs, the model generates something new each time, and at some point it will slip up.

Nobody will notice, because nobody inspects the HTML in detail on every send. That would add far too much work, and only a developer or email specialist could do it.

<td style="border-radius:999px">
  <a href="https://..."
     style="padding:12px 24px">
    Shop the offer
  </a>
</td>

So how can you trust HTML that gets made up again for every campaign? You can trust it when the agent doesn't write it. If the agent only sets the inputs on approved modules, the HTML is code your team has already tested.

Your reviewers should review what they can see, not what they can’t.

Betty, our AI agent, builds from your approved modules by setting their inputs. When a campaign needs something new, she can build an AI Block, a new module for that one campaign. That is new code, so it has to pass automated checks first, and admins can switch AI Blocks off or require approval for campaigns that use one.

3. Does it know what a good campaign is?

An AI checking its own work isn't enough. It needs a definition of quality outside itself that it can measure its work against and can't talk its way past. Some of that can be checked by a machine, and some needs understanding:

Betty runs pre-flight checks on her draft, fixes what they find, and looks at the rendered campaign before handing it back. A person's campaign and an agent's campaign go through the same checks, and each check can be set to block export.

4. Does it handle what makes your campaigns hard?

A newsletter in one language is the easy case. Large teams need personalisation, variants for different segments, language versions, and the syntax of their ESP.

5. Does it work consistently?

One good result in a demo doesn't tell you much. You need high quality every time, for everyone on the team.

6. Does it fit into how your team works?

Your team has to work next to the agent. Humans and AI move faster when each has a defined job, and the system holds the rules for both:

In Better Email, you can undo a turn of Betty's work, and she can't create a revision, approve a campaign or export it. When she builds several variants or campaigns at once, the work keeps running if you close the tab.

7. Does it learn from its mistakes?

Agents make mistakes. What matters is that you can correct the agent so it doesn't happen again, for you and for your teammates: how to build a certain type of campaign, how to configure a module, which modules you like to combine.

In Better Email, teams teach Betty through tone of voice, skills, and AI metadata on the design system and its modules. These are shared across the organisation.

8. Can you trace what it did and control what it sees?

You need full traceability: who did what, and who the agent was working for. The agent should act with the permissions of the person who asked, those permissions should be revocable, and there should be a version history you can go back through.

Ask about data as well: which model runs, in which region, whether you can use your own provider, and whether anything is used for training.

Betty acts as the person who asked, with their permissions, and our security page covers the data side: no training on your content, and the option of your own provider in an EU region.

9. Does it connect to the rest of your stack?

MCP has become table stakes, for two reasons. The agent has to reach the systems where the source material lives, like briefs in project management or images in a DAM. And more companies will build their own marketing agents, running in their own environments. To produce high-quality campaigns, those agents need an MCP connection to a system that holds the design system and the checks.

Having an MCP server says little by itself. What matters is what an outside agent can do through it, and whether your rules still apply.

Through our MCP server, an outside agent signs in as you, builds from your modules, and runs the same pre-flight checks. Approval and export stay in Better Email.

The scorecard

AI on top of chaos is just faster chaos. Build the harness first, then turn up the speed.

See how this works in practice on our page about Betty.