Article 63

How to do A/B Testing in WordPress

By Ryan Carson @ryancarson

27 July 2009 | Category: Design

We recently posted about how 37signals increased their sign-ups by 200% with A/B Testing.

A lot of you are using WordPress, which makes A/B testing pretty tricky, so we've recorded a screencast tutorial to show you how to do it.

Editor's note: We'll be covering topics like A/B testing and marketing your web app at The Future of Web Apps London.

If you’re new to A/B testing, here are a few resources to get you started:

An Introduction to A/B Testing by Jesse Farmer
Wikipedia definition
Google Website Optimizer video tutorials
Advanced Website Optimizer Tricks
Official Google Website Optimizer blog
Google Website Optimizer Tricks

The Video

Please note, if your goal page is on a different domain or sub-domain, you need to make small modifications to your tracking code.

Follow @thinkvitamin on Twitter Please check out Treehouse

Other Posts You Might Find Interesting

Sorry - No Related Posts Found

Comments

http://www.rossbearman.co.uk Ross Bearman

Good screencast, I’ve just been setting up this system for a client application I’m currently building around WordPress.

Amazing how easy it is to get addicted to A/B testing!
Ryan Carson

I know – it makes you want to start testing everything :)
http://johndturner.com John Turner

Google Optimizer is cool, I didn’t even know it existed.

By the way , what screencast software do you use?
http://unmatchedstyle.com Gene Crawford

Great overview on A/B testing Ryan. Was just starting to get into this for ourselves and have had some clients asking about it, so your video is right on time. Thanks for pulling this together!
Ryan Carson

Hey John – I used iShowU – a great little app for the Mac. For the video of me talking, I just used PhotoBooth.
http://www.jeromecastaneda.com/ Jerome

thanks for the great lesson, Ryan.

wordpress is really cool for non-techie like me.
Benjamin Alijagić

Great video! Please more testing tutorials!
http://www.obliviousinvestor.com/ Mike P

Thanks for this video! I’ve been trying to figure out how to do this for some time.

Additional question: What if my goal is getting them to click an outbound link? (I’m an author marketing my books on Amazon.)

Since I can’t put tracking code on an Amazon page, would it work to make a page on my site that just redirects to the Amazon page, but also includes the optimizer javascript?

Something like: mysite.com/book.html (and the file would consist of the optimizer javascript, followed by the redirect javascript)?
http://carsonified.com Ryan Carson

That should work fine, however, I’ve never done it.

Also, you can connect Google Analytics with Google Website Optimizer, so you can create Goals and measure those. You’d need to get Amazon to send buyers back to a thankyou page on your site though, and I don’t know if that’s possible.
http://www.obliviousinvestor.com/ Mike P

Cool. Thanks for the reply–I’ll give it a go tonight. :)

(As to getting Amazon to send people back to my site, I can’t imagine how I’d do that. They’re not the most responsive to authors/publishers/vendors. They know they’re holding all the cards, so to speak.)
http://www.seitengestalter.de/neustart/2009/07/28/neue-links-97/ Neue Links

[...] Carsonified » How to do A/B Testing in WordPress – [...]
http://www.bronsonharrington.com Bronson

This stuff is priceless, many thanks for the great tips.

Stumbled!
http://brendanwenzel.net Brendan Wenzel

Thank you so much for sharing this. I have been trying to figure this out for the past few days and this helped me tremendously.
http://www.ouvyt.com/weblog/2009/07/28/daily-delicious-185/ OUVYT » Blog Archive » Daily Del.icio.us

[...] Carsonified » How to do A/B Testing in WordPress [...]
http://www.kygeek.com/daily-links/daily-links-for-tuesday-july-28th-2009 Daily Links for Tuesday, July 28th, 2009

[...] How to do A/B Testing in WordPress | Carsonified [...]
http://blissofbeing.com/blog/2009/07/28/waynes-weekly-notable-links-6/ Waynes Weekly Notable Links | The Bliss of Being

[...] Shared Carsonified » How to do A/B Testing in WordPress [...]
http://michaelkjeldsen.com/ Michael Kjeldsen

Hi Ryan,

thank you so much for this video, it made me go all-in on testing in WP.

Unfortunately WP (TinyMCE?) keeps removing from the code, even when I have disabled the visual editor. I have to manually edit the post in the database (phpmyadmin) to be able to add the tag.

Did any of you experience this, and i that case: what did you do to prevent it?
http://michaelkjeldsen.com/ Michael Kjeldsen

“… removing from the code” (never mind the gaps in the tag – the comment system removed the tag entirely in the above comment :-)
http://michaelkjeldsen.com/ Michael Kjeldsen

Oh my, your system is teasing med.

In the above two post, guess where to insert [/noscript] :-)
http://designtowp.com Haris

Right post as right time (atleast for me).

Thanks for the super awesome screencast Ryan.

With the information provided by your blog post and google love, I can easily checkout what variables work and what doesn’t work for my business. :)
http://www.bworx.be/blog/2009/07/30/my-del-icio-us-bookmarks-for-july-28th-through-july-30th/ My del.icio.us bookmarks for July 28th through July 30th | B-WORX blog

[...] Carsonified » How to do A/B Testing in WordPress – A/B Testing in WordPress [...]
http://www.seohats.com Alex Juel

Wow, this was a great post! I’ve actually been meaning to toy around with Google Website Optimizer after I had signed up for it about 3 months ago. I started watching this video and decided that now is as good a time as any and set up a heading test, so thank you very much for the push :)

I had one question. Once it’s set up, can we re-enable the visual editor. Was that only so that we could get the full code instead of what WordPress shows in the HTML editor?
http://www.lillbra.se/2009/07/links-for-2009-07-30/ lillbra » Blog Archive » links for 2009-07-30

[...] How to do A/B Testing in WordPress – Carsonified Allt du behöver för att komma igång med a/b-testning i WP. (tags: abtesting wordpress screencast) [...]
http://www.wiredvision.com Keith McLaughlin

Thanks for the video.

I hadn’t heard of the Website Optimizer service before. Seeing as I also use WordPress, I think I’ll include a few experiments with my planned redesign.

Thanks again =)
http://www.dominixz.com/blog DominixZ

Great Video !!

I know Website Optimizer for a long time but I never know how to use it. Today I know because of you Ryan. Thanks a lot.
http://inspectelement.com/articles/links-of-the-week-8/ Links of the Week 8 – Inspect Element

[...] How to do A/B Testing in WordPress [...]
http://bloguismo.com/tutoriales/como-hacer-ab-testing-con-wordpress/ Cómo hacer A/B Testing con Wordpress | Bloguismo

[...] a Aleyda que me pasó el enlace hace unos días llegué a un interesante artículo en inglés sobre A/B testing en WordPress y me planteé grabarlo por mi cuenta en [...]
http://www.silverspider.com/2009/weekly-digest-for-august-3rd/ Weekly Digest for August 3rd » Alex Jones

[...] A collection of my actions and interactions from around the Net over the last week. Shared How to do A/B Testing in WordPress [...]
Gavin Steele

That was a great watch! Learnt a lot, thanks.
Are there any security risks?
What security do most people run on WP?
http://www.socialmallard.com/marketing/ab-testing-with-wordpress-ht-carsonified/ A/B Testing with Wordpress (h/t Carsonified) — Social Mallard

[...] goes with this great video on how to do A/B Testing using WordPress from the team at Carsonified. Check out the original post for some great links and more [...]
http://digitalgilbert.com/2009/08/daily-digital-recap-for-august-6th/ Daily Digital Recap for August 6th | Digital Gilbert

[...] Kevin Gilbert shared Carsonified » How to do A/B Testing in WordPress [...]
http://plano.maidbrigade.com Angela McCall

I agree, Ross, A/B Testing is a necessary function but I have wrestled with the questions that this article has answered. Our clients should appreciate the value of this tool.
http://tarrask.com/blog/2009/08/08/links-de-agosto-2009-o-que-e-necessario-para-ter-um-blog-wordpress-bem-bacana/ Links de agosto 2009: O que é necessário para ter um blog wordpress bem bacana. | the worst kind of thief

[...] Como fazer testes A/B usando o wordpress. É melhor o título assim ou assado? E o layout? E nao sei o quê? Testa, ué. [...]
Stergios

I enjoyed this screencast very much. I found it very educating. I actually had not used Google Website Optimizer but now i am going to use it in future projects. Thank you Ryan. By the way, the music heard at the end of screencast sounds like Stray Cats.
http://www.maestrosdelweb.com/editorial/pruebas-ab-con-wordpress/ Pruebas A/B con Wordpress

[...] mientras escribía dicho artículo vi un video que hizo Ryan Carson explicando como se podía hacer para WP, la herramienta favorita de blogging, así que me pareció [...]
http://www.seoptimise.com/blog/2009/08/30-simple-ways-of-improving-bounce-rate-and-conversion-rate.html 30 Simple Ways of Improving Bounce Rate and Conversion Rate | SEOptimise

[...] of a landing page using Google Website Optimizer or A/Bingo no matter whether your landing page is just a blog post and the call to action only a “follow me on Twitter” [...]
http://briancray.com/2009/08/04/ultimate-ab-testing-resources/ Ultimate A/B Testing resources and how I used them

[...] How to do A/B Testing in WordPress on Think Vitamin by Ryan Carson; A screencast for using Google Website Optimizer specifically on WordPress [...]
http://www.freshtilledsoil.com/ab-testing-with-google-website-optimizer-and-wordpress/ AB Testing with Google Website Optimizer and WordPress | Fresh Tilled Soil

[...] How to do A/B Testing in WordPress RelatedSite testing on a budgetDefining and Developing a Web Prototype Design: Part 2Great Examples of Using WordPress as a CMSWordpress essentials: SEO ToolboxDefining and Developing a Web Prototype Design: Part 1 [...]
http://aparelhosdentarios.com Joana p.

I just love to split test my site’s pages. But having a rough time with some pages in wordpress that require forms.

But thanks anyway for the insights.

Cheers

Joana
http://koszonjukemese.wordpress.com/2009/10/14/links-for-2009-10-14/ links for 2009-10-14 « Köszönjük, Emese!

[...] Carsonified » How to do A/B Testing in WordPress A lot of you are using WordPress, which makes A/B testing pretty tricky, so we’ve recorded a screencast tutorial to show you how to do it. (tags: wordpress testing usability a/b analytics optimization) [...]
http://www.leemunroe.com/ab-testing/ Perfecting User Experience Design with A/B Testing – Freelance Web Design Belfast Northern Ireland – Lee Munroe

[...] Carsonified: How to do A/B testing in WordPress [...]
Angelo Cioffari

nice one ryan! thanks for sharing…
http://www.carmasnack.com Carmen

Great article! I love your video on A/B split testing.
http://garyjones.co.uk Gary

That’s a really useful video, not just on how to do A/B testing in WordPress, but A/B testing with Google Website Optimizer generally! Thanks.
http://www.paulolyslager.com/user-testing-thirteen-ways-selling-book User testing and thirteen ways of selling a book – paul olyslager

[...] and you’re using WordPress for CMS, I can highly recommend the video of Ryan Carson on Carsonified. Using Google’s Website Optimizer, Ryan explains step by step how to conduct a simple [...]
http://terrybritton.com Terry Britton

Kim Roach has been pushing all of us at her membership site to do testing, but I have opted to use WordPress, so it seemed insurmountable. TILL NOW!!

I’ve shared this with Kim already, and will be doing so in a post to hers and Chris Farrell’s forums next. Thanks so much!

Terry Britton
http://www.professional-copywriter.co.uk Professional Copywriter

I love it Ryan!!!! A/B testing is an obsession of mine! and to have a WP plugin as well makes life so easy!
http://designshack.co.uk/articles/accessibility/the-importance-of-ab-design-testing The Importance of a/b Design Testing | Design Shack

[...] How to do A/B Testing in WordPress (Carsonified) [...]
http://davyia.dk/ David Yiadom

Hi Ryan. Thanks for the video… you make it look so simple. And once trying it once or twice, it really is simple.

I was wondering if it’s possible to do a test on a page that has multiple conversion goals. Like a page with product a, b, and c?
jared

GREAT tutorial!! Thanks.

I’m having trouble getting this to work for a home page with an Aweber opt-in form. I’m basically enclosing the entire page in the control script and loading variations of video, text, and an aweber form. For some reason the forms are showing up in IE. When I preview them they work fine but once live the forms don’t show up at all.

I realize this is a pretty vague problem and a long shot anyone has had this some issue, but just thought I’d see if anyone had and has a suggestion. I’m basically just wanting to test various video/text/image combinations on a wordpress home page to test conversions and figured this would work.
http://www.claususonline.com Jorge R.

Hi there Ryan,
Great tutorial. I had tried before to do A/B testing in WordPress, but never succeeded at it. I guess it was because I didn’t consider multivariate testing the way you show it on your tutorial.
I’m no techie, so I guess I better start testing simple things and then move on from there. Like using aweber forms and stuff like that.
Thnx for the light!!
http://www.itsdigitalmarketing.co.uk/ Gary Robinson

Bit late to the party on this article, but thank you very much for the useful video. I’m just starting on A/B testing and was looking for ideas on how to test on a WordPress blog. Your video answered that perfectly. Thank you
http://joeyreed.com Joey Reed

Before watching your video I had a tough time wrapping my head around how multivariate testing would work with WP. Now I see that it’s not so tough. Thanks for the run-through, Ryan!
http://3store.uk.com 3 Store UK

Can the goal be an event in Google Analytics. I too run an affiliate site where multivariant testing will be invaluable
ldexterldesign

So which did get more clicks out of interest?
http://twitter.com/szodaviz szodaviz

Új weboldalunkat a felhasználók igényei szerint kívánjuk alakítani. Ehhez fontos tudni, mik működtek jól eddig, és min kell változtatni. Az A/B teszt jó eszköz lesz.
Bupham

Thank you.
http://pulse.yahoo.com/_UGCPDB6GMOXXI5CBMHCBXHAJDM Halley

So I followed you directions exactly(except that google now seems to combine the control and tracking scripts into one), but when I validate pages I get these:

Control and Tracking script-
Expected to find: function utmx_secti
Found on line 284: /*<![CDATA[*/functi

Page sections-
{ – Unexpected or missing characters – Line: 284 Expected: utmx_section Found: /*<![CDATA[*

Any ideas on how to fix this?
Thanks
http://carsonified.com Ryan Carson

Hi Halley – I’d recommend switching to http://optimizely.com now – much
easier than this method I used to use
Pieter

Hi,
Great tut, thanks!
One question: is it also possible to do this with images or widgets?
http://www.jamiemchale.co.uk jamiemchale

Just a quick tip:

In the video you disabled the visual editor. You can use WordPress shortcodes to insert the section into your blog post using the visual editor. Just add a couple of lines to your functions.php in your theme:

add_shortcode( ‘gwo’, ‘gwo_shortcode’ );

function gwo_shortcode( $atts, $content = null ) {

$t = “utmx_section(“”
.esc_attr($atts[section])
.”")”
.$content
.”";

return $t;
}
http://carsonified.com Ryan Carson

Great tip! Thanks Jamie
http://lastdropofink.co.uk/ Matthew Ogborne

Just to say thanks, I had this on my todo list for ages and just never got around to it

Topics

Learn with Treehouse

How to do A/B Testing in WordPress

By Ryan Carson @ryancarson

27 July 2009 | Category: Design

The Video

Other Posts You Might Find Interesting

Comments

Think Vitamin Radio
Episode #34: Amazon Fire and Responsive Roundtable

Advisory Board

Aarron Walter

Ethan Marcotte

Molly Holzschlag

Paul Boag

Jeff Veen

Ryan Carson

Topics

Learn with Treehouse

How to do A/B Testing in WordPress

By Ryan Carson @ryancarson

27 July 2009 | Category: Design

The Video

Other Posts You Might Find Interesting

Comments

Think Vitamin RadioEpisode #34: Amazon Fire and Responsive Roundtable

Advisory Board

Aarron Walter

Ethan Marcotte

Molly Holzschlag

Paul Boag

Jeff Veen

Ryan Carson

Think Vitamin Radio
Episode #34: Amazon Fire and Responsive Roundtable