“Over the past decade, the power of A/B testing has become an open secret of high-stakes web development. It’s now the standard (but seldom advertised) means through whichSilicon Valleyimproves its online products. Using A/B, new ideas can be essentially focus-group tested in real time: Without being told, a fraction of users are diverted to a slightly different version of a given web page and their behavior compared against the mass of users on the standard site. If the new version proves superior—gaining more clicks, longer visits, more purchases—it will displace the original; if the new version is inferior, it’s quietly phased out without most users ever seeing it. A/B allows seemingly subjective questions of design—color, layout, image selection, text—to become incontrovertible matters of data-driven social science.”
According to Brian, “In Amazon’s early days, developer Greg Linden came up with the idea of giving personalized ‘impulse buy’ recommendations to customers as they checked out, based on what was in their shopping cart. He made a demo for the new feature but was shot down.Lindenbristled at the thought that the idea might not even be tested. ‘I was told I was forbidden to work on this any further. It should have stopped there.’”
Brian goes on and says that “Instead Linden worked up an A/B test. It showed that Amazon stood to gain so much revenue from the feature that all arguments against it were instantly rendered null by the data. ‘I do know that in some organizations, challenging an SVP would be a fatal mistake, right or wrong,’Lindenwrote in a blog post on the subject. But once he’d done an objective test, putting the idea in front of real customers, the higher-ups had to bend. Amazon’s culture wouldn’t allow otherwise.”
Putting ideas in front of real customers is the best way to find out what your customers’ desire and how much they will pay for it. A/B testing is an important step in determining, developing and delivering the customer experience. It is a effective way to improve the real world effectiveness of your experience design and ensure your customers are delighted.