{"id":97911,"date":"2022-01-27T08:27:41","date_gmt":"2022-01-27T16:27:41","guid":{"rendered":"https:\/\/thegood.com\/?post_type=insights&p=97911"},"modified":"2025-05-21T09:40:34","modified_gmt":"2025-05-21T16:40:34","slug":"multivariate-testing","status":"publish","type":"insights","link":"https:\/\/thegood.com\/insights\/multivariate-testing\/","title":{"rendered":"How Ecommerce Brands Can Run Successful Multivariate Testing"},"content":{"rendered":"\n

Trial and error is often the route to success. <\/p>\n\n\n\n

When you figure out what works and what doesn\u2019t, you can do more of the good stuff (and who doesn\u2019t like the good stuff?!). <\/p>\n\n\n\n

Luckily, there are plenty of ways you can test your ecommerce website. After all, there are so many elements that come together, from your product pages and everything that populates them to your marketing efforts, checkout process, and post-purchase campaigns.<\/p>\n\n\n\n

Multivariate testing allows you to test many different possible combinations, which can be a great opportunity to quickly learn what site features are getting the most meaningful results. <\/p>\n\n\n\n

What is multivariate testing in ecommerce? <\/strong><\/h2>\n\n\n\n

Multivariate testing is the act of testing different combinations of elements on your ecommerce website to find out which converts best.\u00a0<\/p>\n\n\n\n

For example, you might serve consumers several different versions of the same landing page with a few differences. One version might have three static product photos, another might have a mix of static photos and a handful of user-generated photos, and yet another might have user-generated photos and a video. <\/p>\n\n\n\n

If you\u2019ve got a good amount of traffic, you can quickly start to see which page (and which combination of elements) resonate the best with your audience. If the page with the UGC and video performs better than the photos on the other web pages you\u2019re testing, you can safely assume that\u2019s the option shoppers prefer or are more inclined to click. <\/p>\n\n\n\n

An important starting point in multivariate testing is formulating a hypothesis based on cold, hard data. For example, let\u2019s say you run user testing and see that the majority of users are clicking out of your checkout after 2 minutes of trying to fill out the checkout form. Then using Google Analytics data<\/a> for your store and benchmarking stats from a reliable research company, you discern that your cart abandonment rate is higher than it should be.\u00a0<\/p>\n\n\n\n

You then form a hypothesis: \u201cThe checkout form is too long\/complex. By shortening the form and adding inline validation, we will deliver a better customer experience and better form completion rate.\u201d <\/p>\n\n\n\n

This is the perfect opportunity for multivariate testing. You can create different combinations of form elements, lengths, or instructions, and serve them to different audience segments to see which delivers the best completion rate.<\/p>\n\n\n\n

Multivariate testing vs A\/B testing: what\u2019s the difference?  <\/strong><\/h3>\n\n\n\n

Multivariate testing is not to be confused with A\/B testing<\/a> (although it can be a bit murky).\u00a0<\/p>\n\n\n\n

A\/B testing focuses on one element and tests multiple variations of that specific element (e.g. testing the headline on a landing page). <\/p>\n\n\n\n

Multivariate testing is a way to try out different combinations of multiple elements all at the same time. Say, for example, you want to test a video, an image, and a combination of the two to see which variation gets the best results. <\/p>\n\n\n\n

The pages you create might look something like this:<\/p>\n\n\n\n

  • Control: your current page design with no changes<\/li>
  • Combination 1: just an image<\/li>
  • Combination 2: just a video<\/li>
  • Combination 3: an image and a video<\/li><\/ul>\n\n\n\n
    \"ab<\/figure><\/div>\n\n\n\n

    This way you can isolate which combinations of elements work best together and create the biggest uplift. <\/p>\n\n\n\n

    You might also have come across the term split testing, which falls under the umbrella term of A\/B testing. In this method, you essentially send traffic to two or more different URLs to see which performs better – think ad traffic seeing the same ad, but being sent to different pages once they click on the CTA. One group might be sent to a product page and the other might be sent to a category page so you can see which journey reaps the biggest rewards. <\/p>\n\n\n\n

    You can run both A\/B testing<\/a> and multivariate testing in tandem. Where A\/B testing identifies the winning elements in an X vs Y vs Z (e.g. video vs image vs text) formula, multivariate testing is more about how combinations of elements work together in an X1 vs X2 vs Y1 vs Y2 formula (e.g. video + image vs video + text vs text + video + image vs just video).<\/p>\n\n\n\n

    The benefits (and downfalls) of multivariate testing <\/strong><\/h2>\n\n\n\n

    There is a time and place for every type of ecommerce testing. <\/p>\n\n\n\n

    One of the most crucial upsides to A\/B testing<\/a> is that you can do it with a slightly smaller audience than multivariate testing. With the latter, you\u2019re likely running a larger number of variants and combinations at once which means you\u2019ll need a bigger sample audience to get significant results.\u00a0<\/p>\n\n\n\n

    Even so, multivariate testing comes with its pros. This method of testing tells you about the interactions between a number of elements on a single page and also shows you what\/where different page elements have the most impact all at the same time. <\/p>\n\n\n\n

    A\/B testing may get you the best headline, but multivariate testing will tell you the best combination of headline + hero image + call to action. <\/p>\n\n\n\n