![\"model](\"https:\/\/thegood.com\/wp-content\/uploads\/image3-42-1024x406.png\")
<\/figure>\n\n\n\nThe “test everything” mantra is sung far and wide by experimentation experts, but that approach simply won’t work for most companies.<\/p>\n\n\n\n
As companies strive to optimize user experiences, they are encountering pitfalls inherent to A\/B testing<\/a>. Instead of continuing deeper into the trough of disillusionment, I want to share some alternatives that promise faster, cheaper, and more satisfying outcomes.<\/p>\n\n\n\n The popularity of search terms like \u201cdata-driven\u201d are at an all-time high.<\/p>\n\n\n\n If your media consumption looks anything like mine, you\u2019ve probably read countless articles about all the ways in which tech leaders leverage data to make informed decisions.<\/p>\n\n\n\n With the advent of analytics platforms, tracking pixels, and attribution modeling, it seems like everyone is an analyst. Or at least they play one in meetings.<\/p>\n\n\n\n As Stanford\u2019s Ramesh Johari notes, \u201c100 years ago, data was expensive and slow and practitioners were trained statisticians. Today, data is cheap and real-time and everyone is a practitioner.\u201d<\/p>\n\n\n\n With the increased access to data<\/a>, leaders are antsy to put that data to work. We regularly speak with leaders who want to transform their company\u2019s relationship to their data: they run heat mapping tools on their site, have numerous analytics tools at their disposal, and their teams have dabbled in A\/B testing<\/a>. But many still struggle to make sense of the abundance of data and choose the right path forward. They struggle to move from data-informed to data-backed.<\/p>\n\n\n\n Though the language is similar, we make a crucial distinction between data-informed and data-backed.<\/p>\n\n\n\n Many organizations aspire to be data-backed, but they often fall short by primarily relying on generative research methods.<\/p>\n\n\n\n Generative methods are great for understanding what\u2019s happening on a website and forming hypotheses about what would work better.<\/p>\n\n\n\n Generative research methods include things like:<\/p>\n\n\n\n Alternatively, evaluative research can be used to substantiate ideas with evidence. Evaluative research methods include things like:<\/p>\n\n\n\n When we ask so-called \u201cdata-driven” companies what data they use, they typically list a number of generative research methods. Only about 25% of them list any evaluative research methods and rarely do they go beyond A\/B testing<\/a>.<\/p>\n\n\n\n Generative research, those methods on the left, are great for understanding what\u2019s happening on a website and forming hypotheses about what would work better.<\/p>\n\n\n\n But in order to move forward with confidence<\/strong> that your solutions will actually work, you need the methods on the right. Evaluative research is how we move from data-informed to data-backed.<\/p>\n\n\n\nThe State of Data-Driven<\/h2>\n\n\n\n
![\"graph](\"https:\/\/thegood.com\/wp-content\/uploads\/image2-51-1024x333.png\")
<\/figure>\n\n\n\nData-Informed vs. Data-Backed<\/h2>\n\n\n\n
![\"graph](\"https:\/\/thegood.com\/wp-content\/uploads\/image7-29.png\")
<\/figure>\n\n\n\n\n
\n
![\"infographic](\"https:\/\/thegood.com\/wp-content\/uploads\/image1-59-1024x447.png\")
<\/figure>\n\n\n\n