Synthetic audiences fall short: What marketers need to know
The AI-generated audience simulations may make it more difficult for brands to reach new audiences, writes RYA co-founder Mark Himmelsbach.
This audio is auto-generated. Please let us know if you have feedback.
The following is a guest piece co-founder of creative AI marketing tool RYA and advertising agency Episode Four. Opinions are the author’s own
Artificial intelligence has proposed a novel technique for testing that’s cheaper, faster and whose proponents say is as accurate — perhaps even more so — as “real” research: synthetic audiences.
CMOs have good reasons to want this. Research is expensive, deadlines are short, marketers need to do even more with less and they increasingly need to justify their decisions using data. Synthetic audiences are simulated panels of customers, built from a set of data like customer records, interviews, demographic descriptions or combinations of those inputs.
These simulations can evaluate campaigns and explain their preferences in minutes. Marketers can then revise their work, ask again and repeat at a fraction of the cost of recruiting a human panel. But do synthetic audiences constitute high enough quality evidence?
The market seems to think so. In July, synthetic data company Simile raised $200 million at a $2 billion valuation, five months after announcing its series A. Even Adobe now lets marketers test creative, messaging and more against AI-generated customer profiles inside its Brand Intelligence product.
Now, properly identifying a target audience and codifying their preferences is hard enough. A simulation whose predictions hold up against real customer responses can help improve familiar campaigns. Indeed, for those teams who’ve successfully modeled their customers, synthetic audiences will save them considerable amounts of time and money.
There may even be additional creative possibilities on the margin — think more precise demographics, more targeted campaigns — but it basically helps you do more of the same.
A brand trying to acquire new customers, however, needs to understand people who haven’t yet made a decision to spend money with them. This is where simulations get tricky.
Synthetic audiences are being sold with that kind of solution in mind as well. Evidenza, for example, markets itself as “AI market research for impossible audiences,” as in impossible to do without AI.
But synthetic audiences still need evidence about the people they’re supposed to represent. Giving the model good evidence upon which to train is called grounding in the AI industry. For example, Bain reports that pairing simulated customers with a company’s own customer data substantially improved their overall performance.
But whose data are we talking about? What separates a useful synthetic audience from a generic simulation is data quality over time: whose experiences are represented, how recently were they observed, how relevant are they to the task at hand, and how often does new evidence get added to the training data?
A detailed snapshot of your existing customers may not actually provide much insight into potential customers who considered your brand but walked away. I’m reminded of the classic financial services industry disclosure: “past performance is not indicative of future results.”
Synthetic audiences – and synthetic data writ large – propose to use statistics and AI to create the data required to model those missed customers using external sources and observations. But again, part of the art of marketing is identifying trends and finding creative ways to stand out by doing something different that appeals to your target audience.
Whether or not AI can help humans do so is the trillion-dollar question.
Activision CMO Tyler Bahl, for example, asks what his team could do that only “Call of Duty” could do. His “stature, swagger, and scale” criteria express an ambition for the brand to delight their customers with singular, spirited ideas. A synthetic audience can help test how an idea might land, but can it decide what’s worth trying and how far to push it?
What’s more, there’s a self-reinforcing dynamic here that must be considered. Even if the data is high quality, and even if the model itself isn’t overtly biased, how might it eventually condition its users to test ideas likely to get accepted?
Constant scoring risks making a team more conservative and risk-averse rather than more creative and productive.
If campaigns need good “scores” to get approved, marketers will have every reason to learn what the panel rewards. This is commonly known as survivorship bias.
Under pressure, marketers may — consciously or not — begin incorporating those preferences before proposing something. Eventually, the process might invert: whereas the simulation was intended to improve creative decision-making, it may begin to influence creative decision-making, often regressing to the mean.
In the end, CMOs have to decide how much authority to give these tools, and leave their teams room to both use these models and pursue ideas the models wouldn’t have picked. They can thus derive considerable value from synthetic audiences while preserving the judgment and imagination that make their teams valuable. A few practical recommendations:
- Know your audience: Understand where the information any model or AI tool is trained on, where it comes from, and how often it’s refreshed. If the brief is reaching new customers, the simulation must include people outside your existing customer base.
- Continue doing real research: Compare what the model expected with what people actually said and did. If they disagree, investigate why. A small test with real people can sometimes tell you more than another round of simulated feedback.
- Continue pursuing moonshots: Develop experimentation as a portfolio. Reserve a proportion of budget to pursue unfamiliar approaches, radical ideas and moonshots.
- The “Inverse Cramer” AI strategy: You may consider deliberately pursuing certain ideas that are rejected by the models to keep the team and the creative direction honest.
Cheaper, faster research should undoubtedly give creative teams more room to explore. Use simulation to test more ideas, learn from the results and improve the work. But marketing leaders must refrain from delegating too much of the process to the machines and remember to seek ways to ground their simulations in reality.
