Experimentation is a Marketer’s Secret Weapon. Here’s Why

Pini Yakuel October 22, 2020

2020 was a challenging year for science–too many people seemed to just ignore it. But science leads to progress. The good kind of progress. I’d even argue that you can’t have too much science.

Science is based on experimentation. To experiment, you need data. It works exactly the same when it comes to marketing.

Lucky for marketers, they are enjoying both unparalleled access to a wealth of performance data, and experimentation in marketing isn’t over-utilized. Yet.

It’s not a complete blue ocean, of course. But if properly leveraged, adopting experimentation as a strategic marketing initiative will likely lead to insights that may turn into significant competitive advantages.

Or, as a Product Manager here on Optimove put it recently: “if we are creating a hierarchy of marketing practices, the first should be the concept of letting customers plot their own journeys, the second, experimentation.” Bold statement. But just how true, I needed to find out.

What Research Says

According to research by Gartner “Marketing teams conduct experiments to test and improve the efficacy of new products, features, experiences and campaigns. These improvements allow marketers to innovate and stay ahead of the competition; Gartner’s 2018 Marketing Data and Analytics Survey found that the majority of marketers who prioritize testing and experimentation significantly outperform their peers.”

(Source: Gartner, Drive Marketing Innovation Through Experimentation, Joseph Enever, September 29, 2020)

With this sizeable difference in performance, I really hope experimentation is second in importance, if not first, for CRM marketers worldwide.

Why Experimentation is so Valuable

The reasons for why those who constantly experiment outperform their peers are straight forward:

1. Experimentation helps demystify or corroborate long-standing beliefs.

We all know the tiered trope of “this is how we’ve always done it, and it works. Why change it?” It is the number one excuse to avoid change and one of the most challenging phrases to refute, no matter the industry you are in. The one way around it, is data. And experimentation brings you the best data you could dream of.

Experimentation isn’t different than racing. Put two 100-meter sprinters head-to-head, let them race, and in the end, you will know who is better. But instead of holding only one race, have thousands of them, making the results statistically credible and undeniable.

CRM marketing teams that use experimentation as a foundational piece of their strategy end up turning into the backbone of their companies. See for yourself.

2. Experimentation reduces the risk of making bad decisions while increasing the probability of making good ones.

In business school they teach to always run sensitivity analyses in order to be ready for different scenarios, but those are only as valid as the spreadsheet on which they are calculated. Experimentation is your “live” sensitivity analysis. Whether you are looking to introduce a new product, try out a new marketing channel, or even launch a loyalty program, you are assuming that they will work, but are always at risk of overspending, overinvesting, or overestimating impact.

Running experiments helps you know whether your assumptions and hypotheses are valid or not. However, you must take into account that patience and understanding is crucial for insightful experiment results:

• You must wait for statistical credibility of results. Peek too early and you might jump to the wrong conclusions.
• You must be impartial. All outcomes are good outcomes, as even if you get a “bad result,” you can learn what to fix or stop before you overinvest.
• You must be curious. When results are in, don’t stay only at the high level. It is imperative to dig deeper. As the aforementioned Gartner research puts it “[f]or example, at an aggregate level, conversion may appear mediocre, but among a segment for which the campaign or segment was hyper-relevant, performance may have been excellent.”

3. Experimentation helps refine marketing efforts and KPIs in an agile manner

Marketing success is measured by a number of KPIs. Some might focus on a single “north star metric” while others will prefer a more democratized approach. No matter the style, you always need to continuously know if what you are doing is working. “Working” doesn’t mean whether a specific KPI is trending favorably, but rather whether it is doing so because of marketing efforts.

Experimentation lets you know how each marketing effort impacts that KPI allowing you to test, learn, finetune, and reiterate. Marketers who outperform peers, discover ways to do this at scale and in an agile manner, with little wasted opportunities.

Making Experimentation Your Secret Weapon

Marketers who experiment as a ‘way of life’, do so at all levels. They start at the single interaction level, comparing two or more variations using A/B/n tests, or measuring incremental behavioral changes using control groups. Then, they move on to test groups of campaigns as a whole, to measure the impact of larger-scale initiatives. And finally, they end up experimenting on complete marketing plans, comparing, for example, their standard strategy with a tweaked version.

This high-level tiered approach is what guides those marketers who constantly outperform their counterparts as they lean into marketing experimentation as their secret weapon.

Digital & Social Articles on Business 2 Community

(24)