The Comprehensive Guide to Creative Testing

The Comprehensive Guide to Creative Testing

Olaf Krysik

HK Digital Co-Founder · 04 Oct 2021

The Comprehensive Guide to Creative Testing

Taking a bit of a different approach this time around we are not going to go into the depths of Colombia to teach you about digital marketing. Instead, while still in Colombia I can place a strong emphasis on the nature of creative testing. It is a popular buzzword that gets thrown around a lot in the digital marketing space, specifically when it comes to channels like Facebook. Arguably, it makes a lot of sense when you advertise you need to build appealing creatives to beat through the clutter and make your ads stand out. But how do you know what type of creative strategy to adopt? How do you execute our creative tests? What lessons do you take away from your creative? This is the one thing that is often left untouched, or at least not spoken about in detail enough for a starter to get a good grasp on, and ultimately everyone has their own approach. In today’s article, we will cover the most important aspects of Creative Testing, which are:

  1. The Basics of Creative Testing; what is it?
  2. Creative Testing Frameworks, which one works for your business?
  3. The importance of a dedicated Creative Strategy,
  4. Building out your own approach

The Basics of Creative Testing — What is it?

Every piece of creative starts with an idea, a concept. At the helm of the creative process, there will be a designer/copywriter or another visionary who creates this initial concept and wants to see this executed. Eventually, when this creative is produced and presented it will go through “an initial internal review”, we could call this the first “creative test”. As with most things in our world, the definition of Creative testing depends on who you ask and at which point. To avoid any confusion, I want us to agree on single definition that we will utilize in this article to ensure that we are all on the same page, I define creative testing as:

“The deployment of creative to a wider external audience for the verification of its effectiveness on (paid) media platforms”

Because it may sound confusing at first, I want to expand on it a bit to ensure everything is clear. The main terms used in the definition are:

  • Wider external audience,
  • Any audience that expands beyond your own internal company and that has the potential size to be statistically relevant1
  • Verification of effectiveness,
  • Essentially, we outline all of the main KPIs of our Creative vs our benchmarks and compare it,
  • Media platforms,
  • Any channels that we utilize to run/test our creative, in most cases this will be paid channels,

With that being cleared up we now have a good understanding of what the basics of creative testing are.

Creative Testing Frameworks, which one works for your business?

Testing creative is a key component of most modern digital marketing strategies, and it is the role of a Growth marketer to help build a creative testing framework. Now most frameworks usually follow a single purpose approach or are basically just executed without too much thought. At HK Digital we utilize our own Testing Framework that we developed, but it is by no means a one-size-fits-all approach. Let’s get into the most popular methods used:

  1. Plug-and-Go,
  2. The Automated Algorithm,
  3. The 3-2-1 Approach,
  4. Feed the Beast™

Plug-and-Go Framework

Calling this a framework is probably a bit of a stretch but it is among the most common approaches that I see whenever I work with clients who have taken a first initial approach at Digital marketing. They will set-up a basic campaign, create an ad set and build one (maybe two Ads) and just let these run. This approach is super common, and it is obviously not the most ideal or optimal approach, but it can still work. It however, does not encompass a lot of testing opportunities and except for the fact that it is easy to setup and execute, it adds little additional value.

If you are reading this and recognize this as being your own setup, I suggest you set up a call with us to go through our growth audit – we can quickly help identify any such issues and propose alternatives. But as I alluded to earlier, this type of setup can still work.

The reason for this is that over time advertising platforms have become incredibly good at finding people to purchase your product/service with the creative you put-out. That is why they have this plug-and-go system, it is there to help out starters and encourage them to spend their advertising dollars on the platform rather than elsewhere.

The Automated Algorithm Framework

Coming in second is the Automated Algorithm which as the name so conveniently states just builds on the automated system that paid media platforms provide. It does however utilize more of the power that these platforms offer as opposed to the Plug-and-Go but it still relies heavily on the platform to do most of the heavy lifting.

When Auditing advertising accounts for clients, I will often come across a setup where clearly someone set out with a good idea in their head, implemented it to the best of their abilities and then let the system handle the matter itself. A recent example included an ecommerce website that was optimizing all of its traffic for email signups, having a variety of audiences in the same campaign and just a few variations of essentially the same creative on rotation. The fundamentals had been laid-out and the marketer in question just let the platform run with it. Technical optimization becomes a must in such a situation but not everyone is capable of doing an audit, that’s why at HK Digital we offer a Growth Audit service to help you get started.

An Automated Algorithm testing framework benefits from the intelligence of the platforms but it most likely does not have the most optimal setup for testing, as the effectiveness of the creative is dependent on the algorithm of itself, and there’s no dedicated background strategy supporting the creative tests.

The 3-2-1 Approach

When coming across this setup in an Ad Account I know that I am dealing with someone who has historically done advertising on paid media platforms with decent success. A stable methodology and creative framework, it is often no longer as relevant these days as systems have gotten better and a more fine-tuned approach is necessary, but it remains relevant.

It also utilizes the lessons from both a Plug-and-Go system and the Automated Algorithm framework at the same time, no longer solely relying on the platform but having a thought-out approach. The 3-2-1 system simply builds on the idea of deploying three Ads consecutively and having the platform depict the best performer. Ads are then slowly faced out as the best performer becomes apparent and the numbers are reduced from 3 into 2 and then into the single best performer. This approach is then utilized for every new creative concept that is produced consecutively and the methodology helps most businesses generate good results on Snapchat, Facebook and LinkedIn.

The truth of the matter though is that this approach is now outdated, and it has not evolved over the years, often not utilizing the latest advances in the platform’s technologies or even its capabilities. And that is why we can move on to our final Creative testing framework, “Feed the beast”.

Feed the Beast

At HK Digital we have developed and deployed our own unique creative testing framework that we have dubbed “Feed the beast”. The reasoning behind the name is quite simple, algorithms thrive on data, and the more data you give them the better they become at predicting what it is that you are looking for. But in order to power this algorithm it needs to be fed with data, it needs to be nourished so that it can be used to its fullest potential.

There are many different frameworks out there that almost exclusively rely on Algorithms to completely power their creative testing, while I can understand the effectiveness behind it – having worked closely with deep-learning algorithms while at RTB House, I am also painfully aware how inefficient they can be. Let me explain.

An algorithm is built upon a set of rules and targets, it keeps these in mind when finding who to show your Ads to. We set up certain rules to constrain the algorithm and steer it in the right direction – if we however let it run rampant it can get lost. Lost, in the sense that it will give you the best results you will have ever seen, but you will not know why. And that becomes a problem when you want to scale-up your advertising over time, as you will have no idea why certain practices worked and others did not – you are left solely to the mercy of the algorithm.

With Feed the Beast we punch through this, by setting up a clear direction for the algorithm to take so that we can understand its movement, but not mapping out its actual path. This translates to us having an always-on-performance environment in which we are benchmarking our results and feeding the beast with consistent data regarding what we want to achieve. On the other hand, we have a testing environment where we continuously introduce new creative concepts that we are testing against the already existing performance environment. By introducing this additional layer, we are ensuring that we are always telling the platform what it is that we are striving for, and we have a consistent supply of data to help it achieve its goals.

For a full breakdown of the framework, you should read our Feed the Beast case study.

The Importance of a Dedicated Creative Strategy

With your creative testing framework built and ready for use, you still require the actual creative production, which if you are a small business is often conducted in-house by a small team of designers (or even a single person!). From the get-go I want to emphasize that there is nothing wrong with this approach, but that it can certainly be improved upon.

For our clients, we build a creative strategy from scratch that fits the brand and the guidelines that we are given. In some cases though, a client requires a brand pivot before they can put a strong emphasis on their digital marketing. This can happen at any lifecycle of a business and is common practice, for a fully-fledged and redefined brand strategy we partner with Mila Branders to help deliver a full hands-on brand and creative strategy.

The reason that you need a dedicated Creative Strategy for your creative testing is simply put that you need to have a goal or objective in your mind for the creative that you are producing. Just having a framework does not suffice, after all, who are you targeting, why are you targeting these individuals? What type of creative are you going to use? All these are important questions that should be part of any creative testing framework. But a dedicated creative strategy and what it requires needs a full article of its own.

Building out your own approach

Whether you have read or just skimmed through this article, you will without a doubt have noticed that there’s a lot of options out there – and even though we have our own framework that we use within HK Digital for our clients, this is not always the best solution. Understanding your own business and the situation it is in is crucial in determining what creative testing framework you are going to adopt. Resourcing is a crucial and key aspect here, and not just in terms of the media budget that you have for advertising but also in terms of creative production and output. The heftier the framework the more resources it demands, both in terms of budget and creative production.

Does that mean that if you have a small budget you cannot conduct proper testing? Absolutely not. But it does mean that you will have to allow either more time for your tests to conclude or return meaningful data or significantly limit the number of tests that you are running consecutively. In my opinion that last part is the least problematic, in fact if you check out my article on Growth marketing lessons from the depths of the Colombian Jungle you will find that I argue that overexposure can be detrimental to your marketing efforts.

When building out your own approach, start with the basics, your budget. Work your way down (or up) from there by pinpointing your key KPIs – like your CPM, CPC, CPLs, Conversion costs etc. And then decide how much you can effectively test – this will trickle over into the rest of your creative testing framework and allow you to test effectively. If you need a hand in getting started, you can always reach out to me or request a Growth Audit.

Summary

When deciding on your Creative Testing you should focus around understanding the necessities of your business first. Creative Testing is a process that is first and foremost driven by your business and its needs, you need to put that on the first spot. Testing just for the sake of testing is a killer for your business, so consider it carefully.

Select a framework that works for you, but ensure that you are using the platforms to your advantage ensuring that your creative testing framework is up-to-date and uses the power of data and creative to the maximum.

If you are feeling overwhelmed and lost, do not hesitate to reach out, we will be happy to help you out!


  1. Statistical relevance across a wide population is often unattainable within Creative Testing, but the rule of thumb is the larger the audience the better.

The Comprehensive Guide to Creative Testing

WRITTEN BY

Olaf Krysik
HK Digital Co-Founder

Growth Hacker with over 7 years of experience in helping companies achieve their growth goals.