The challenge of testing creative

I am a brand strategist. I believe that research is valuable and necessary. And I think we’re giving creative testing way too much power.

Allow me to explain. 

Research is a useful way to learn about your audience’s current perceptions and needs. What do people think about your brand today? What are they craving from your competitors? Asking smart research questions can shape brilliant strategies. 

Creative testing, however, is a whole different beast. It’s a nice idea to think that creative testing will lead to a successful brand that’s universally loved. The idea is that focus groups will perfect your brand, and your trustees will weep tears of joy. 

It’s also a nice idea to think that polls will predict election victories. Sometimes they do… and sometimes they don’t. There’s a reason that we still hold elections, no matter what the polls say. 

One of the biggest risks with creative testing is a bias toward safety.

Participants are typically drawn to concepts that feel familiar. Phrases like “caring community” and “cutting-edge research” are a slam dunk in testing. But given what we know about branding, safe phrases like these won’t cut it.

For example: I’m convinced that the name Hulu would have failed a creative test miserably. The streaming giant’s CEO’s has said that the name is a Mandarin word for gourd. Gourds hold precious things. Cool! But even if you explained that to a focus group in 2007, would they have chosen Hulu out of a list of names? Would they have chosen it over something like StreamFast or TVNow? I can imagine the report: A majority of participants found Hulu difficult to pronounce. Many associated it with the nonsense word “hooey.” I’ve written many of those reports. Hulu was not safe or familiar. But now it is. Now, we don’t even question its nonsensicalness. Now, we love it! 

The reality is that no amount of creative testing can guarantee branding success. 

Branding has an emotional component that is difficult to articulate and makes it ill-suited to rigorous, scientific study. You can still love a movie that has a low score on Rotten Tomatoes. 

Testing creative certainly won’t guarantee that you’ll break new ground. In fact, safe brands are usually the least successful. It’s Apple versus Microsoft, or Ben & Jerry’s versus Breyers. (If you don’t mention Apple at least once in a branding blog, your LinkedIn account gets deleted.) 

Plus, making a splash in branding is trickier now than 40 years ago, because then our attentions weren’t fractured in 1,000 different ways. You might adore MrBeast, or you might have no idea who that is — even though he has more followers on YouTube than there are people living in Nigeria.

Here’s my argument: Don’t let creative testing water down a bold strategy, a groundbreaking idea, or a stunning design. 

To return to my polling analogy, if a candidate changes their stance on taxes because of what the polls tell them, then they probably aren’t a great candidate. But they can tweak their messaging about taxes to get noticed and get elected.

So we need to shift attitudes that creative testing can identify the best concept and focus efforts on identifying modifications to make concepts better. 

With that in mind, here are my best tips for effective creative testing:  

1. Get specific about what you want to learn. The bar for success shouldn’t be set at “pleasing everyone.” In fact, it should almost be the opposite of that. Good branding should elicit strong reactions, which are hard to come by in an artificial environment. But if you are lucky enough to get reactions, good or bad, take note. 
2. Don’t test for appeal; test for associations. What does each concept make you think of? What other brands does this remind you of? That’s how you end up with the name Parachute for a bedding company, instead of something like GoodSheet. 
3. Bring audiences into the fold and ask them to check for authenticity. Describe the challenges you face, and test against those challenges: Would this convince you that we’re more cutting-edge now? Does this convey how beloved we are by small businesses? People are very good at telling you when something feels authentic to them, which is a valid research goal. But when they gravitate toward the familiar, as they are prone to do, remind them of the specific heights you’re trying to reach by doing something new. 
4. Don’t ask audiences to pick the winning idea. Focus groups in particular can offer a muddy picture of success. For example, let’s say you run three focus groups. Two of the focus groups like Direction A best. The third focus group loves Direction B, but a few participants say that they loathe Direction A. Now what? With this kind of creative testing, you run the risk of sowing doubt about all of your concepts and having to start over. (Sigh.) Remember: Reactions like this don’t automatically imply that the directions were weak. It might mean that they felt too new and different to audiences. And in the marketplace, being new and different is often a good thing. 
5. Separate words, images, and design elements. A stock model who looks too old for your product can be hard for participants to see past. The use of the word “nonsensicalness” might put people in a sour mood about an entire concept (or blog post). Similarly, consider testing something completely new in the context of something familiar to get a better sense of how it’s working. 
6. If a concept is too abstract, don’t test it. Imagine explaining to a focus group in the ’80s that you’re going to stitch the outline of an athlete onto a shoe. (“A majority of participants do not find the Air Jordan concept appealing.”) A/B testing and “shop-alongs” tend to work well because people see concepts in context, and you can measure the real actions they take or don’t take. Ask fewer hypothetical questions and make more real-world observations. 
7. Use testing to check for red flags. For example, people from a different background might see something in a word or image choice that you didn’t. Or you might uncover a deep attachment to a symbol that you didn’t think mattered. Research is great at spotting potential landmines. 
8. For branded concepts, keep the group small. Don’t make your creative testing an unintentional premature brand launch — or, worse, a public leak. If you feel the need to test with a wide audience, consider blinding the concepts (hiding any identifiable information of the brand). 

Above all, don’t use creative testing to make decisions for you. 

Instead, do thorough research during the discovery phase to arrive at a stronger brand. Trust your partners and your teams to know your brand, your industry, and your goals best. 

Then, once concepts are in the field, do A/B testing or “shop-alongs” to modify them based on measurable results.

Nothing can guarantee brand success. And trying to please everyone almost certainly guarantees that you’ll create a forgettable brand. Trust that your branding partners have put in a lot of work to arrive at a brilliant strategy and that you have guided them wisely. Anything else is just nonsensicalness.