March 8, 2016

Let’s Talk About:

Split Testing for Big Wins

You’ve likely heard someone promote split testing as a panacea for improving conversions -- “change a button color,” they say, “and your conversions will magically skyrocket!”  Sometimes changing buttons DOES lead to big impacts -- but often, tests require more planning, intentionality, and resources.

When split testing, one-size-does-not-fit-all.  Your approach depends on your traffic volume, conversion rates, and ability to quickly design and develop.

If your site doesn’t get massive traffic, split testing should still serve as a key component of your strategy.  Rather than making small tweaks, though, consider aiming for home runs -- drastically different experiences than what you currently have.  When launching these, fight the urge to throw away your old version and to only start iterating only on the new.  Here’s one way we test two entirely different applications.

If you’re fortunate enough to have high traffic, test early and often: some of our biggest wins have come from tests that were almost afterthoughts.  In testing every low-barrier idea you can, you’ll increase your chances that some will have big impacts and useful learnings.

For more, check out the Wildlife Conservation Society zoos and aquarium sites -- some of our most-tested projects.

WCS Zoos & Aquarium Sites

Community Thoughts on Split Testing for Big Wins...

  1. 1. 7 Conventional Landing Page Design Tactics You Should Still Test “It’s far too easy to defer to best practices instead of discovering what makes your unique audience click. Proven landing page techniques are commonly practiced for a reason, but what works for 90% of websites won’t automatically work for yours.”
  2. 2. Why most A/B tests give you bullshit results “As most who have run the experiment have discovered, while button color is an excellent way to describe A/B testing, it’s rarely a meaningful way to improve your product.”
  3. 3. Optimize your Site the 'Hotjar Way' How do you decide what to test? Try matching visitor drivers, minimizing barriers, and amplifying hooks. It outlines a plan for measuring and understanding each aspect as a first step.
  4. 4. 6 Ways to Generate A/B Test Hypotheses Consider employing 6 psychological heuristics in your testing practice: scarcity, recognition, naive diversification, anchoring+framing, fluency, and social proof.
  5. 5. Why Your Testing Strategy Shouldn’t Look Like LinkedIn’s “You can never be certain if a test will win or not, but you can improve the chances with great research, a solid ideation methodology, and creating a collaborative environment with others to generate ideas.”

Related Articles

How Long to Run A/B Tests: the Known and Unknown

How to Visualize a Series of A/B Tests

Washington DC Metro
105 West Broad Street
4th Floor
Falls Church, VA 22046

Durham, NC
309 E. Chapel Hill Street
4th Floor
Durham, NC 27701

Boulder, CO
1915 Broadway
Boulder, CO 80302