What Every Brand Wants to Know About AI-Powered Creative Testing

We’ve listened to hundreds of sales calls and identified the 10 most asked questions.  

Kate Nunan, Senior Customer Success Manager, Rebecca Shepherd, Senior Account Executive, and Stuart Pecksen, Product Owner, get together to answer them for you. 

Here are just a few of the answers, for the full episode and all the insights – watch above or head over to Spotify. 

Q: How does Dragonfly AI validate its predictions? and why should we trust AI models?  

Stuart: We have spent a lot of time, over a decade's worth of research we're talking about here has gone into this at Queen Mary's University. What this has meant is that we've got a model now that is accurate to 89 % in terms of predicting human attention. This has been verified against MIT's independent saliency benchmarks as well.  

Becky: You don't want to be using a tool that is only going to be a certain amount accurate to test all your data. It's so important. There's a lot of money on the line here. We are constantly asked about how we're validated; how accurate we are.  It's always really nice to be able to reassure customers about how accurate we are as an AI tool, always within 10 % of human interpretation, which those that are familiar with AI tools, that is pretty bang on. 

Kate: We're very neuroscience based, which I think is my favorite thing to bring to people that it's not just our word for it. It's academically based 

Q: How does Dragonfly AI compare to traditional methods like A-B testing, focus groups or eye tracking studies?  

Becky: Some of the benefits of using Dragonfly, and I guess probably the same with a lot of AI tools in general, are time saving and cost efficiency. Where Dragonfly is brought into brands or businesses is where they realize that they're spending a lot of money on, say, traditional techniques and waiting a lot of time to get those results back. 

We know with these kinds of decisions in creative testing that they need to be made quickly, and they need to be made often. So that means that it gets so expensive. Obviously one of the benefits of using Dragonfly and how our model works is you get a license for Dragonfly and that's unlimited testing. So, you immediately have that cost saving and also time saving because it's instant.  

Kate: I completely agree with what you're saying. I think from the clients that I work with, a lot of the time when they don't have money or budget or the time to do a panel test or focus group, think that's when Dragonfly comes in. If you think of A-B testing on something like an e-commerce site, that can take you, best case, four weeks to get back or worst case, be up to 10 or 16 weeks. 

Using Dragonfly for that really quick validation point is ideal. We have different features and different things on the platform itself that helps with A-B testing.   

Stuart: I would say that AB testing or multivariant testing and Dragonfly are not necessarily mutually exclusive and it's not that you do one instead of the other. In fact, I'd say that Dragonfly enables you to do better versions of this at a much earlier stage. 

Q: Can Dragonfly AI test creative assets across multiple formats? For example, e-commerce, social, packaging, retail, and video? 

Stuart: We're very good at supporting clients across specific use cases using our collective expertise. We've got fantastic support functions within the business to help those different use cases. The core algorithm applies the same universal analysis to anything that you show it. So, this could apply to use cases we haven't even considered yet, which is quite exciting. 

Becky: It's one of my favorite things when it comes off on a sales call. Being truly omnichannel is really one of the main things about Dragonfly. We can test anything and all to the same level. It's not that one is particularly better than the other. That is why and how we're able to scale so quickly.  

Kate: I find a lot of the times when I start working with clients, they might bring us in as a packaging solution or as an e-commerce testing tool. But when they get into that algorithm or understand it, you can see how clients get very excited when they say, OK, great, I have all these different considerations or projects that I need to deliver, and I finally have one solution that can go across everything.  

Q: How do teams actually use Dragonfly AI in their workflow, and does it replace or complement existing tools?  

Kate: I always say we're not here to replace any expertise. We're here to add to it. So, working with brand managers or brand teams or insight teams, they're the experts in their customers and they're the experts in their industries. You use Dragonfly to validate that you don't need to make someone work harder to see what you need them to see.  

For example, we work with a lot of beauty brands. If they have a benefit on pack like it has an SPF or a certain ingredient and you know that's how your customers shop, why would you make someone work harder?   

Stuart: I think the key thing here is that different teams and organizations have got different goals and perhaps depending on their ambition, there could be an impact on different areas of their workflow. From my experience speaking to clients, speaking to prospects, they seem to have a preference to try and keep humans in the loop at some stage.  

Becky: Our clients are always testing every single piece of content that goes out. And that's the main goal or the the crux of using Dragonfly is that it is literally used for every single thing, every single time. 

Q: How does Dragonfly handle tech-heavy creatives like packaging, claims or call to action buttons? 

Stuart: The key thing to note here is that the attention algorithm is pretty cognitive. So, it's agnostic to the content. However, we are looking at potential ways to add layers to our analysis to help the analysis to understand what it's looking at and to help with the taxonomy of tagging up these different things and looking at the location of them and assessing the location and the impact that that has.   

Kate: The Dragonfly platform doesn't read the text, but it can let you know if it's easy to see. If you do need someone to see it, don't make them work harder than they need to. think anyone knows if you must look for information, you're going to give up and that's potentially a mis-sale or a mis-click or whatever it might be. 

Stuart: If those text elements do form part of an intended visual hierarchy related design, for example, we can still analyze that. We can still rank all your assets based on that as well.  

Q: What's the ROI of using Dragonfly? Can you prove that optimized creative leads to better conversion rates or sales? 

Becky: There's different success metrics dependent on your content type. But it's not rare that we see a lot of our customers seeing incredible returns on investments. Results of increasing attention on X element and the changes it has in click through rates, for example.   

Stuart: It's not rare to hear really good success stories from our clients and our clients, they want to stay with us and we're hearing more and more that they want to roll us out across different regions, different teams as well. That speaks volumes to the ROI that they're already seeing in the way they're using it.   

Q: What are the biggest challenges companies face when adopting Dragonfly AI and how do you help to overcome them? 

Becky: One of the main challenges is not really specific to Dragonfly. I think it's specific to any tool that anyone's bringing in is it's just different, right? People have got something else that they need to be using. So, it's more sort of like the usage of how to use it, which only takes like a training session, super easy to do.  

Kate: When a client starts to engage with Dragonfly, there is that real, it can do anything across any use case and kind of go anywhere. It's trying to align the expectations with the right outcomes. It's a really exciting challenge to have. And I think Dragonfly does well from that initial how they speak to us, how they engage with us, they get a clear idea of where we fit in workflows and what are the answers that you can get from Dragonfly and why is AI useful within an existing workflow.   

Stuart: The trust in AI piece has definitely shifted over the last few years. There's still a little bit of skepticism, but it's a very, from my perspective, it's a very quick conversation. Most of the time when people are referencing AI, and obviously we've got AI in our title, so it's a logical question, but most of the time when they're questioning it, they're thinking of these large language models like chat GPT and things like this, because obviously it's blown up. 

It's a very quick conversation to say, look, we're not generative. Our core IP is very much predictive to help you to save time and money. And that's it.   

We hope you found this helpful! 

If you’d like to hear the team answer more commonly asked questions watch the full episode above or listen on Spotify here. 

Don’t forget to subscribe for further episodes!