Independent Tests Yield Same Result

I have analyzed two A/B testing case studies of Visual Website Optimizer users and have concluded that the sure fire way of catching visitors attention is having human photos on a website.

What’s interesting is that even though both tests were independent, they (surprisingly) tested the same aspect and arrived at the same result. This gives me confidence that there is something special about having human photos on a website which increases conversion rate instantly. Let’s have a look at the two studies.

Case Study #1

Medalia Art sell Brazilian and Caribbean art online and they recently got pleasantly surprised when photos of artists on their homepage increased visitor engagement by 90%. This wasn’t their first A/B test as they keep playing with their homepage. In the past, they successfully reduced bounce rate by 20% by testing the placement of their promotional message.

As can be expected from an online art shop, they showcase paintings from three famous artists on their homepage. This time, for no good reason they decided to see what happens if paintings are replaced by the artists’ photos. I tried to probe Medalia Art for reasons why they came up with this idea but they said it was a completely random choice to test paintings v/s photos.

The goal for this A/B test was to increase visitor engagement (defined as a click on any link on the homepage; it is the inverse of the bounce rate). They didn’t use sales as a conversion rate as many of their sales happen after interaction on the telephone (and a lot of haggling) with the customer.

Here are the screenshots of the control (paintings) and the winning version (photos).

Control (Paintings)

Variation (Photos)

Test results are as follows:

Clearly, version with artists’ photos had a much better conversion rate that the version with artists’ paintings. In fact, the winning variation almost doubled the conversion rate on the homepage.

Case Study #2

EmptyMind is a web analytics and optimization blog by Jason Thompson. There is a large phone icon in the blog’s sidebar which links to his contact us page. Unlike Medalia Art who did the test just to check what happens, Jason had a reason for running a phone icon v/s his photograph test on the site.

While attending a summit, he got several comments from other people that he didn’t look like his (then) Twitter icon. So he thought it would be best to have a single photo representing him across all the social network sites he uses. Testing the photo on the blog was just an extension of that thought.

The goal for the test was visits to Jason’s contact us page and here are the variations and their performance:

Control	Variation
Conversion Rate: 3.7%	Conversion Rate: 5.5%
Increase in Conversion Rate: +48%

Here is what Jason said about the test results:

People want to connect with other people emotionally, the photo makes that emotional connection so much easier and as the test is proving, drives people to the contact form more than a nondescript icon.

The Key Lesson

Both case studies independently arrived at the same (statistically significant) result that human photo(s) increased conversion rate. This proves that there is certainly something magical about human photos.

I am going to try this test on Visual Website Optimizer homepage (perhaps using customer photos for testimonials).  If it is relevant for your website, I recommend you try an A/B test to see if a human photo can do wonders for your business too.

If you try out a test involving human photos and get positive results, then we may have discovered a great formula for increasing conversion rates!

Case Study

This post is about a recent A/B split testing case study where a redesign of a sales page resulted in 20% increase in sales. AquaSoft is a leading software company in the area of digital photo presentation. They have a complete portfolio of (desktop) software products related to photos – slideshow creation, desktop publishing, photo books, etc. Their software products can be downloaded for a free limited duration trial, following which a customer pays for the full version.

AquaSoft undertake more than 10 A/B tests per month using Visual Website Optimizer, an online A/B testing tool. They recently finished a test in which they tested their existing sales page against a new redesigned sales page. Note that this page wasn’t the free trial download page; rather it was a sales page where customers visit to actually buy the software.

Here is the control version of the page (click to view full version):

And here is the (final) redesigned version (click to view full version):

What’s different?

The pages are in German language, so let me point out a few key differences between the new and the control version:

• The new design gave a modern, fresher vibe on the page and had better contrast.
• The old design had two calls to action per product – one for instant download and other for purchasing software in a shrink wrapped box. The new design reduced the choice to primarily one option (instant download) and the call-to-action was also transformed into a snazzy button (as compared to a simple text link in old design).
• The new design was optimized to build trust in the customer. The sidebar in new design contained different kinds of guarantees:
  • Free lifetime updates and bonus software
  • Lifetime guarantee on the software
  • 24h/365 days support
  • Free customer login id
• The new design also incorporated a seal made out of their logo to provide authenticity to the guarantees

A key point to note is that AquaSoft did not change their product features neither their guarantee policy. The details were simply not present (or highlighted) on the old sales page.

Two phase testing

AquaSoft carried the A/B test in two phases. In the first phase, old design was pitted against a variation of the new design. Improvements started with about 50% but settled on about 35-40% after some time. They were very happy with the result but craved for more details because in this test they were measuring clicks into the shopping cart, not the actual sales.

So they measured the sales which showed an increase of 17.7% of actual sales. It also showed that the bounce rate within the shopping cart was somewhat higher in the winning combination but the overall increased click rate outtakes the bounce rate.

In the second phase, AquaSoft refined the winning variation by doing minor changes such as reducing whitespace and placing software names above their respective box shots. This resulted in a further increase of 10% in clicks on shopping cart button and the overall increase in sales settled to about 20% (up from 17.7% in previous phase). All results were statistically significant, so the 20% increase in sales is real.

What can we learn form this?

This case study has several lessons for all of us who are serious about optimizing online sales:

1. Design matters! If your sales page has a boring design, expect boring sales.
2. A/B split testing is totally worth it! AquaSoft invests time, money and effort into doing as many as 10 split tests per month. And that is because they understand what impact a successful A/B test can have on the bottom-line.
3. Some customers do abandon the checkout process! The next logical step for AquaSoft would be to optimize their checkout process but their payment system is integrated into a third party so it won’t be easy to set up a split test there.
4. A/B testing is best done in phases. Small improvements in multiple phases ultimately add up to give a significant boost to conversions.

AquaSoft summarized the key take-home lesson from this A/B test:

“Don’t think your website is good as it is. Always test, always improve. It’s a slow process but worth doing. Our concrete test results show that a clear modern design improves sales. Especially in the sales process, trust building is very important.”

For AquaSoft, A/B testing required one-time investment of design resources but the 20% increase in sales will hopefully continue for months to come. If AquaSoft can manage to get such a fantastic ROI, so can you. Think about it – is your sales page optimally designed? What elements can you test on it?

Beta Invite

To try out Visual Website Optimizer enter invite code ‘carsonified‘ when signing up.