By now, everyone running a marketing campaign or promotion has probably thought about A/B testing certain elements. However, I bet that a large majority of them have not actually done it. The disconnect is actually pretty straightforward — it's easy to overthink.
From the mind of a continual tester, here are the basics:
Keep it Simple — Yes, we could get into multivariate testing but I am guessing that the majority of us do not even need to get that deep. The trick is to make the test simple by testing an image, an offer, a color, or a size. As long as you don't confuse yourself, you should find success.
Apples to Apples — Whatever your test, make sure the variables are equal. This is where you can easily get too complicated and wind up shooting yourself in the foot. You cannot compare a larger postcard to a smaller one if the creative isn't exactly the same. And you can't compare two promotions for two different products. Keep this in mind for more accurate reporting.
Measure the Results Numerically — Reporting in marketing is key. Take the time to understand how you are going to find the winner of the test, whether it's through a website visit, a sales conversion, or even a phone call. As the results come in, document them, and do not declare a winner off a thought or inclination. I cannot tell you how many times I have heard "Well, I think people really enjoyed the B creative better." Report on the numbers that support your claims.
Make it Count — Don't go through all of this work to gain a perspective on a test that doesn’t really matter to your bottom line. Testing two different phone numbers, unless your business depends on where a person calls, seems useless to me. Produce a test that will give you a better insight into your customers for future advertisements. You want to complete your test with a procedure for the future, not just another question.
Do it Again, but Differently — Ongoing testing is vital to growing your business. Once a test is complete, you should have a new procedure for that type of advertisement. Now test within that new procedure. If you found that customers react better to images of animals, find out which type of animals. Once you think you're done, go back to the beginning and retest.
By following these simple ideas, your testing should become more accurate, easy to understand and in the end, maximize your marketing dollars while boosting your revenue.