{"id":81168,"date":"2017-01-27T10:23:37","date_gmt":"2017-01-27T18:23:37","guid":{"rendered":"https:\/\/thegood.com\/?post_type=insights&p=81168"},"modified":"2023-10-09T07:29:03","modified_gmt":"2023-10-09T14:29:03","slug":"ab-multivariate-testing","status":"publish","type":"insights","link":"https:\/\/thegood.com\/insights\/ab-multivariate-testing\/","title":{"rendered":"What Others Fail to Teach About A\/B and Multivariate Testing"},"content":{"rendered":"\n

Do you believe everything you read online?<\/p>\n\n\n\n

No?<\/p>\n\n\n\n

Then why do you keep getting distracted by those messages promising a revolutionary, sure-fired way to pole vault your conversion rate to the top of your industry?<\/p>\n\n\n\n

Don\u2019t worry. You\u2019re not alone. Flavor-of-the-week conversion advice gets more websites stuck in the digital mud than any other distraction we know of \u2013 including social media and mirror image bias (thinking the customer looks just like you).<\/p>\n\n\n\n

Let\u2019s take a look at a straight-line path to accomplishing your goals \u2013 something scientifically proven and confirmed by the most successful companies in the world to be absolutely trustworthy.<\/p>\n\n\n\n

After all, the opinions we care about most come from the customer, and the results we care about most are always aimed at well-defined goals. Let the sellers of new ideas keep repackaging and pushing their wares.<\/p>\n\n\n\n

We\u2019ll stick with the narrow way. Nothing complicated. Nothing theoretical. Just clear-cut paths to carefully crafted outcomes.<\/p>\n\n\n\n

Period.<\/p>\n\n\n\n

Let\u2019s take a detailed look at conversion optimization testing methodology: Split, A\/B, and Multivariate.<\/p>\n\n\n\n

\"ab
Which do you like better\u2026 A or B?<\/em><\/figcaption><\/figure><\/div>\n\n\n\n

A\/B and Multivariate Testing – Defining Terms<\/h2>\n\n\n\n

First, let\u2019s get at the differences between the primary methods of conversion optimization testing. Then we\u2019ll talk about when and why you should use one or the other.<\/p>\n\n\n\n

A\/B testing<\/h3>\n\n\n\n

You have a decision to make. Maybe your current call to action (CTA) copy for the buy button is \u201cBuy Now\u201d, but your conversion rate isn\u2019t where you want it to be, so you wonder whether changing the copy to \u201cPlace My Order\u201d would make a difference.<\/p>\n\n\n\n

You set up an A\/B test<\/a> to send half your traffic to the Buy Now version and the other half to the Place My Order version. You wait long enough to get a good sample, compare the results, and choose the winner.<\/p>\n\n\n\n

That\u2019s the way many, maybe most, marketers use A\/B testing.<\/p>\n\n\n\n

It\u2019s wrong.<\/p>\n\n\n\n

It\u2019s not the most efficient use of your time, nor will it deliver the best results.<\/p>\n\n\n\n

A better project for A\/B testing would test two radically different site designs. A\/B testing can help you quickly narrow down the candidates from a global perspective.<\/em><\/p>\n\n\n\n

Moreover, A\/B testing isn\u2019t limited to two variations. You could put forth five (let\u2019s say) different website layouts or design perspectives and test them all against one another. That\u2019s still A\/B testing, since you\u2019re testing one thing (an entire page) against gross revisions of the page.<\/p>\n\n\n\n

Key point:<\/strong> Use A\/B testing for global changes, not for small changes.<\/p>\n\n\nUse A\/B testing for global changes, not for small changes. <\/a><\/span>Share on X<\/a><\/span>\n\n\n

Multivariate testing<\/h3>\n\n\n\n

Testing takes time and costs money. The sooner you find better solutions to your current problems, the sooner you can begin benefiting from those solutions.<\/p>\n\n\n\n

Once A\/B testing has determined the overall layout that works best to help move your visitors further down the sales path you\u2019ve established, you\u2018ll want to switch to multivariate testing (MVT) to get your conversion rate spiking as quickly as possible.<\/p>\n\n\n\n

Rather than test only one change at a time, though, with multivariate testing you\u2019ll throw several variables into the mix and test them all at once. That\u2019s why MVT provides a time-to-solutions advantage. It can quickly help you determine the best combination of potential changes.<\/p>\n\n\n\n

You may want to change the color of the button and the size of the button, in addition to the CTA copy on the button. That will give you eight variations (2 x 2 x 2) to test. You\u2019ll need to run the MVT longer than the A\/B test, but the results will give you considerably more insight.<\/p>\n\n\n\n

Once again, you should use A\/B testing to compare one page against another. You should use multivariate testing to compare changes to multiple elements on a certain page.<\/p>\n\n\n\n

Key point:<\/strong> A\/B testing takes care of selecting the big picture. MVT refines the picture you choose.<\/p>\n\n\nA\/B testing takes care of selecting the big picture. MVT refines the picture you choose. <\/a><\/span>Share on X<\/a><\/span>\n\n\n

Split testing isn\u2019t what it\u2019s often made out to be<\/h3>\n\n\n\n

Promise us you won\u2019t use this information to start arguments at the water cooler. This is for those who like to be exact. It won\u2019t matter to anyone else. If you introduce the concept to your team at all\u2026 be kind, and expect to get challenged.<\/p>\n\n\n\n

You\u2019ll often hear A\/B and multivariate testing referred to as \u201csplit testing.\u201d You\u2019ll read it in articles, hear it from the podium, even see it in white papers.<\/p>\n\n\n\n

But split testing is neither A\/B nor multivariate. Split testing, in its pure form, is the process of using a different URL for different versions of the test. That\u2019s it.<\/p>\n\n\n\n

It\u2019s really \u201cSplit URL Testing.\u201d<\/p>\n\n\n\n

Split testing can be used in either form of conversion optimization testing, but split testing isn\u2019t either form. Of course, that\u2019s not the prevalent current usage and words do change meaning over time\u2026 so few people care.<\/p>\n\n\n\n

No argument. But don\u2019t let the potential benefit of split URL testing disappear along with the strictness of the meaning. Along with the technical reasons for using different URLs for each version of your page, there\u2019s another component often overlooked: the URL can affect click-through rate.<\/p>\n\n\n\n

For instance, would someone be more likely to click on \u201cyourcompany.com\/buynow\u201d or \u201cyourcompany.com\/yes\u201d? Would it make a difference at all? You\u2019ll never know until you test. You can use split URL testing to compare URLs.<\/p>\n\n\n\n

Resources:<\/p>\n\n\n\n