Getting the Basics Right: Test, Test, and Test!

Probably the most underutilized aspect of direct marketing is in testing all the elements of a direct marketing campaign.

It’s surprising because through testing different media sources, lists, creative executions, offers and call to actions you can systematically improve your response to find that sweet spot to deliver the best cost of sale and profit.

Testing provides you with value information or performance learnings to help you make informed decisions prior to rolling out your campaign. Testing takes the guess work out of your campaigns, so why wouldn’t you!

To get the best ROI from your campaigns you need to optimise the performance of your campaigns to understand which elements of the campaign can be improved – media selection, list selection, creative execution, offers and call-to-action.

You can only make informed decisions regarding any of these campaign elements provided you measure their performance.

By tracking all of these elements you’ll be able to identify which areas need to be optimised to increase response, reduce cost per response or increase sales conversion rates to deliver a better cost of sales and more profit.

When testing different elements within a campaign you need to consider a few basics:

  1. Campaign ReportingYou need to create a campaign report that covers all the metrics to track results. You can’t expect what you don’t inspect, and you can’t inspect what you don’t measure and track!
  2. Split Test or Cell TestThere are a number of ways you can test different elements within your campaign. For example, you can either Split Test (often called an A/B test) by having two separate communications go out with different offers to see which offer gets you more response and a better cost of sale.

    Or, you can have a Cell Test where you have a number of different offers or combinations of lists, offers and call to actions.

  3. Test Group SizeWhen testing any element within your campaign you need to ensure that the results you achieve through your test cells are statistically valid. The size of your test cells are really going to be determined by the size of your universe. If you have a smaller universe you’ll need to be careful that the sizes of your test cells aren’t too small to render the test outcomes inconclusive.

    With a much larger universe to play with, you can have the luxury of fine tuning the campaign first to find your “Control” piece (what I call the Sweet Spot). With a larger universe to play with you can select up to 10% of the universe for tests to find that sweet spot, before rolling out to the rest of your universe.

    Generally speaking, in the case of direct mail for instance, I wouldn’t test anything with less than 5,000 records. Considering typical response rates anything less wouldn’t provide enough responders to provide any real performance learnings.

    Test sizes for telemarketing can be much less, and I find that when testing a script 250 presentations is a good number to get good learnings. And, do the test with just a few good callers so there is consistency in the delivery of the script.

  4. Random SelectionAnother important element to consider is how you select the records for your cells: To ensure your testing as scientifically as possible, be sure to use a random selection method when selecting the number of records for each test cell.

    For example, if you want to test different offers in a 5,000 cell head to head test, don’t just select the first 5,000 records for Cell 1 and then the next 5,000 records for Cell 2: Be sure to randomly select your records from alternative rows within your database.

    What this means is that you’d start with Record 1 for Cell 1 and then select every 8th record thereafter until you reached your sample size for Cell 1. For Cell 2, you’d use the same process but would start with Record 2.

    Selecting your records for the test cells this way will ensure that you are taking a random selection throughout your entire database, which provides for a representative sample of the whole file.

  5. Campaign CodesWhenever you test any element within your campaigns, be sure to assign a separate campaign code every time. You’ll need to have a separate campaign code for each test to make it easier for you to compare the results between each of the test cells. You want to be able to compare each cell on a head-to-head basis.

    Comparing the results this way will usually take any emotional bias out of your consideration, and sometimes you’ll find some interesting learnings in ‘losing’ cell that you might be able to test again against your ‘Control’.

  6. Go with the Winner!Sometimes, you’ll find that the results of your test surprise you. The ‘winning’ cell isn’t the one you or the team thought would be the winner. It’s happened to me many times. Early on in my career, I wouldn’t trust the results and would try again with what I thought was the best cell as my Control.

    And guess what? I was wrong!

    Be careful you don’t fall into this trap. If you’ve set up your test cells correctly, then you should trust the results of the outcome of the test. Basically, the winner is your customers telling you what the right combinations of medium, lists, copy, creative execution, offers and call-to-action they prefer.

It’s not about you, or what you think should be the winner in a test. Let the numbers make the decision for you!

This entry was posted in About us. Bookmark the permalink.

Leave a Reply

Your email address will not be published. Required fields are marked *