ad testing +OdinText
[Authors note. As I am writing this blog post early Monday morning after the Super Bowl, I have already completed the initial ad testing analysis. It’s the case where modern AI and analytics software (OdinText) is faster than the data collection process/vendor we’re relying on. We’ve asked an open ended comment question among n=3,000 respondents about which super bowl ads they like/dislike and why. Eager to have the analysis complete as soon as possible, the analysis is already done, and blog written based on n=1,011 initial responses received. But since 1,998 more are expected I’m painfully waiting to publish results until the rest of the fielding comes in. The bad part is waiting for the sample. The good part is knowing that now repeating the analysis will literally take less than 1 minute. Just uploading the data into OdinText, and then the brand names and advertisement likes and dislikes will automatically be coded, analyzed and charted in seconds. I just have to review if anything has changed materially and make small updates in my copy below in such case. As it turned out, more data did change findings, and so I did have to change my blog copy. Ah, the joys of modern analytics!]
The Advertising Pundits weighed in on which ads were best and worst even before the Super Bowl aired. We tend to do things a little differently at OdinText and allow data, not opinion to drive.
Of course, for “best” and “worst” not to be subjective, we need some definition of desired outcomes. Last year we looked at a simple formula to evaluate efficacy consisting of Awareness + Positive Sentiment/Liking of the ads.
For instance, you may not remember this because of the low sentiment, but last year 85 Lumber was one of the companies with the highest Awareness after the super bowl. However, because it also had low sentiment and relevancy (as it dealt with the explosive issue of immigration/Trump’s wall in a somewhat ambiguous way). It’s probably the case that it ended up doing better among its core customer segments than among the general population, but since Super Bowl Ads are expensive, I argued that all things equal, a strategy with a broader target in mind, which aims to leave a positive impact among this broader group, should provide a better ROI. Looking at it another way, to have the most significant positive impact we want to maximize both awareness and sentiment almost equally.
With those assumptions and comments from over 3,000 respondents, OdinText’s AI predicted which of the Super Bowl Ads were successful, and which were not. Below I’ve shown 10 Brands/ads, the best performing 6 and the worst 4.
OdinText Ad Ratings
THE WINNERS
#1. THE NFL
In a year where there has been a lot of controversy surrounding NFL players taking a knee, and with a few of our respondents explicitly stating that they had boycotted the Super Bowl this year, it interesting to see the NFL advertising, and doing it so well. The NFL’s Dirty Dancing with Manning and Beckham performed best, I believe in part because of its high relevance to the audience, but also for garnering high awareness together with very high positive sentiment/liking. In fact, only one other ad came close in sentiment.
#2 AMAZON ALEXA
https://www.youtube.com/watch?v=J6-8DQALGt4
That second most well-liked ad was Amazon’s Alexa. Not as much because of its awareness (which was rather low in comparison), but because of its extremely high sentiment. The audience loved the various famous actors playing the voice of Alexa at least as much as they enjoyed NFL players Dirty Dancing.
#3 TIDE
https://www.youtube.com/watch?time_continue=1&v=doP7xKdGOKs
In 3rd place we have Tide. They earned the spot less so for sentiment (though viewers did like the ad). The reason Tide did so well was primarily due to the awareness it garnered. Tide had THE HIGHEST awareness of any Super Bowl Ad. However contrary to some of the Advertising Pundits opinions, it just wasn’t quite as consistently well-liked by viewers as Dirty Dancing, and Alexa.
If you are in the camp who believe Awareness is everything, then Tide should have an even higher spot.
#3 DORITOS (& MOUNTAIN DEW ICE)
https://www.youtube.com/watch?time_continue=10&v=4eKYR_iL5eU
Doritos + Mountain Dew Ice was so close in our model, that I’m going to give them a tie for 3rd. Not awareness like Tide, but for balancing both positive sentiment and awareness perfectly. It’s mix of awareness and liking was in the same proportions as NFL Dirty Dancing, just at a slightly smaller scale.
Obviously considering the audience and occasion, just like the NFL ad, Doritos especially is a highly relevant product, and as importantly the humorous approach with two extremely popular yet not commonly seen together stars (Namely Morgan Freeman and Peter Dinglage/Tyron from Game of Thrones) succeeded in the unique Combo messaging of Fire & Ice.
#5 BUD-LIGHT (NOT BUDWEISER)
https://www.youtube.com/watch?v=CxGUmtRLm5g
https://www.youtube.com/watch?v=lvTE_5c7buk
Budweiser is almost expected to do well. So, in a way, it may be surprising to see it doesn’t make it into our analysis (Beer, in general, did poorly especially Miller Ultra). It really should be so easy for Budweiser though. Here’s a case where the occasion is more than just relevant, it’s almost as if the brand has a historic Super Bowl halo effect. That said, their performance was less than impressive.
While the idea of stopping the Budweiser line to make water in an emergency could be touching for some, reminding consumers you have a good fun product may be a safer strategy than asking for kudos for merely being a good corporate citizen?
And that’s where Bud-Light’s Knight did better. Beer should be about fun…
#6 TOYOTA
Here’s a case where awareness was quite low, but the ad was still more liked than average compared to the other brands. Toyota and our #8 brand just barely made the list. While the setting was right “The Super Bowl,” in the end perhaps the ‘Priest, an Imam, a Monk and a Rabbi’ may have felt a bit less like a joke, and more like preaching…
THE LOSERS
#1 COCA-COLA
https://www.youtube.com/watch?v=-R-EEdvDrUU
Like Budweiser, we expect a lot from Coca-Cola when it comes to advertising. They’ve been pushing the diversity message for a few years now. It may be that pulling at heart strings is far harder to do than making people laugh. Coca-Cola had lower than average sentiment coupled with relatively low awareness. Not a winning combination.
#2 DODGE RAM
https://www.youtube.com/watch?v=SlbY1tGARUA
Dodge Ram did better than Coca-Cola at least, especially on awareness, but even concerning sentiment/liking.
The negative aspect of course in large part was the appropriateness of Martin Luther King’s message at the beginning.
When it comes to ads like these though, I think we must assume, as was the case for 85 Lumber last year, that perhaps the brand knows what it’s doing. They aren’t there to please everyone (as you would hope is the goal of Pepsi and Coca-Cola), but to message their core audience with a ‘We Get You – Even if Everyone Else Doesn’t’. And so, awareness wise, Ram did better than Amazon, Bud-Light, and Pepsi. But on an overall basis, they get dinged by the overall sentiment due to the some would say clumsy ‘MLK + Patriotic’ messaging. Only time and sales will tell…
#3 T-MOBILE
https://www.youtube.com/watch?v=C-rumHvmqCA
T-Mobile was less well liked than you’d think, who doesn’t like babies right? Turns out people are getting tired of the “social responsibility ads” in their entertainment, at least that’s what they told us.
#4 DIET COKE (TWISTED MANGO)
The Booby Prize. Ok, so here is a bad ad. In PR they used to say, any PR is good PR. But Diet Coke didn’t do too well on either of our metrics. It had low awareness combined with even lower sentiment/liking. Diet Coke Mango, because, just no…
HALF TIME
A WORD ABOUT PEPSI
Pepsi, what can I say. You may be surprised that yet again, Pepsi performed poorly compared to the other brands I mentoned considering that their name was all over the Super Bowl during the Half Time Show. And yet, it may be that the real winner of Halftime is the brand of the performer, which this year was Justin Timberlake. We saw a similar pattern last year as well.
PS. See Your Data +OdinText