In this article, we\u2019ll cover:<\/p>\n\n\n\n
- \n
- Why A\/B testing<\/a> velocity is important<\/li>\n\n\n\n
- Why we prefer high-velocity testing over isolation<\/li>\n\n\n\n
- How to run multiple A\/B tests the right way<\/li>\n<\/ul>\n\n\n\n
Why Is A\/B Testing Velocity Important?<\/h2>\n\n\n\n
A\/B testing is one of the most efficient methods for increasing revenue and optimizing the customer experience on your ecommerce site. If you want to get the most out of your testing efforts, you\u2019ll need to run multiple A\/B tests at once to get quicker results.<\/p>\n\n\n\n
As the Jeff Bezos adage goes, \u201cOur success at Amazon is a function of how many experiments we do per year, per month, per week, per day.\u201d Without velocity, any testing program will be low impact and may not be worth the investment.<\/p>\n\n\n\n
The more tests you can perform in a certain amount of time, the more efficient you\u2019ll be in optimizing your site for conversions.<\/p>\n\n\n\n
Why We Prefer High-Velocity Testing Over Isolation<\/h2>\n\n\n\n
High-velocity testing aligns with the way the site actually runs\u2014dynamically, with small, iterative changes over time that all support the long-term key performance indicators (KPIs) and strategies. Customer expectations<\/a> evolve with the times, so your site should too.<\/p>\n\n\n\n
Testing in isolation means missing opportunities to understand how your changes will react to each other over time, sending you down a one-way path with each test\u2019s results, limited by the road set out before them.<\/p>\n\n\n\n
Obsessing over changing one detail at a time in isolation could lead to long-term stagnation and a site that just can\u2019t keep up with customer expectations<\/p>\n\n\nCustomer expectations evolve with the times, so your site should too <\/a><\/span>Share on X<\/a><\/span>\n\n\n
Note:<\/strong> You may be wondering if multivariate testing is the best solution to your isolation dilemma. However, if you\u2019re asking about high-velocity vs. isolation testing, you likely don\u2019t have enough site traffic to run effective multivariate testing with any significant velocity due to the traffic required.<\/p>\n\n\n\n
How to Run Multiple A\/B Tests the Right Way<\/h2>\n\n\n\n
Test all of the key pages of your site:<\/a> <\/strong>Most A\/B tests running on ecommerce sites are testing one (or all) of the following pages:<\/p>\n\n\n\n
![\"key](\"https:\/\/thegood.com\/wp-content\/uploads\/ecommerce-pages-1024x412.png\")
<\/figure>\n\n\n\nWhen you\u2019re organizing your testing program, make sure you\u2019re not focusing all your ideas on one page of your site. When operating at high velocity, you should be testing several pages at any given time so you\u2019re not neglecting any aspect of the customer experience.<\/p>\n\n\n\n
Isolate tests by page:<\/strong> Try thinking about tests in terms of where they touch users in the funnel<\/a>, isolating page-by-page. Simultaneously testing multiple elements on any one page muddies the waters, while testing on funnel stages will help you understand what\u2019s keeping folks interested in learning more about your product<\/p>\n\n\n\n
Note:<\/strong> To test on funnel goals, the conversion rate shouldn\u2019t be the only metric you\u2019re monitoring: although a transaction may be the ultimate goal, a micro-conversion of getting a user to the next stage in the funnel would also be considered a win in this scenario. A few metrics we use to measure the success of a test include:<\/p>\n\n\n\n
- \n
- Click-through-rate (CTR)<\/li>\n\n\n\n
- Revenue per visitor<\/li>\n\n\n\n
- Field completions<\/li>\n\n\n\n
- Average session duration<\/li>\n\n\n\n
- Bounce rate<\/li>\n<\/ul>\n\n\n\n
Stagger your test launches:<\/strong> Track launch dates and stagger them so that if results from one previously running test change drastically with the launch of another, you\u2019re more likely to notice the impact.<\/p>\n\n\n\n
Rely on statistical significance instead of time since launch:<\/strong> It makes sense to set time frames for your tests, so that you have a consistent schedule of tests occurring. The problem with this practice is that it can compromise the impact of your results. If you don\u2019t let a test run to the point that it reaches statistical significance, the results may not be a true indicator of the success of a test.<\/p>\n\n\n\n
Separate site-wide navigational tests:<\/strong> If you\u2019re conducting an expansive test that impacts the navigational hierarchy of your entire site, consider that a top-of-the-funnel test and isolate navigational tests from each other. Even when operating at high velocity, running numerous site-wide tests at once can compromise clean testing results.<\/p>\n\n\n\n
Prioritize your tests:<\/strong> There are a variety of effective options to prioritize tests for your team, one of the most common being the PIE framework<\/a>.<\/p>\n\n\n\n
- \n
- The PIE framework<\/strong> is a popular method of prioritization that ranks tests on three criteria: potential, importance, effort. Calculate the average score for each of those criteria to generate the PIE score, which will help you rank and determine the priority level of each test.\n
- \n
- Potential: How much potential for improvement does the page you\u2019re testing have? Take a look at your site analytics and heatmap data to determine which pages have the most (or least) potential for improvement.<\/li>\n\n\n\n
- Importance: How would you rank the importance of the page you\u2019re testing? Think of it this way: Users on your FAQ page may not be as valuable as the users on your checkout page.<\/li>\n\n\n\n
- Ease: How difficult will it be to implement this test into the page? Are there technical implications that may make your test more difficult to implement than others?<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n\n\n\n
![\"PIE](\"https:\/\/thegood.com\/wp-content\/uploads\/pie-framework-1024x490.png\")
Prioritize your tests based on the PIE Score<\/figcaption><\/figure>\n\n\n\n <\/p>\n\n\n\n
Learn from Your A\/B Tests<\/h2>\n\n\n\n
Determining the appropriate testing velocity for your team is vital to running successful and productive A\/B tests. Running too few tests in \u201cperfect\u201d isolation means your experiments will be stagnant and inefficient; run too many at once and it may be difficult to determine the true impact of each test.<\/p>\n\n\n\n
Here\u2019s the deal: It\u2019s impossible for us to tell you what the perfect velocity of testing is for your team. It will vary from team to team. Don\u2019t start comparing yourself to other people in your industry; just start generating ideas based off of the data and insight that you have available, and test those ideas at your own pace. Eventually, you\u2019ll find a rhythm that works and run with it.<\/p>\n\n\n\n
Remember that there is no perfect testing scenario; just follow the KPIs and the results you get from each test. Let the data guide the way\u2014keeping in mind what you know about your audience\u2014and you\u2019ll be well on your way to designing winning A\/B tests<\/a>.<\/p>\n\n\n\n
- The PIE framework<\/strong> is a popular method of prioritization that ranks tests on three criteria: potential, importance, effort. Calculate the average score for each of those criteria to generate the PIE score, which will help you rank and determine the priority level of each test.\n