One of the things on my list is to try calls to action which are not Sign Up, precisely because folks think that that involves either making a commitment or paying money. (I get at least a half dozen emails a month asking me to cancel a free trial, from folks who grew up in a generation where anyone giving you something free is going to send you a bill 30 days later.)
"Sign Up" solves a problem for the site owner. Write something which solves a problem for the customer.
I'm going to try things like, e.g., Start Making Bingo Cards.
I actually had a few dozen buttons made in a particular design to test different calls to action against each other earlier this year, but the design itself flopped -- hugely -- against my pre-existing buttons, so much so that I didn't think color or copy changes on it would recover the loss. Curses on having my testing bandwidth limited by graphical expertise...
Though actual result will depend on A/B test, my experience suggests that this could be a double-edged sword. The call to action on my homepage said 'Start Optimizing Your Website Now' and some of the savvy users complained they couldn't find how to sign up. Some users simply wanted to sign up and my call to action looked like they had to go through additional pages before they get what they were looking for.
You could easily put a small "sign up" link in the header and/or footer that are not part of your "pitch" to the customers. Most "savvy" users I know are likely to search the page for something like "sign up" if they're looking to the point.
I think it depends of your target audience , people doing A/B testing are tech savvy enought to signup and spot a trial from a free offer, so it may not make much difference for them as they know what they are looking for, but if you are selling an ebook or a Bingo software, chances are your audience isn't tech savvy, so you can really tailor the message the way they understand it. So I'll say talk geek language to geek audience, they will get it, and avoid talking Geek language to non Geek audience, that's probably one of the main problems, geeks tend to talk geek language to non geek audience for non geek products. I think trial period or demo...etc means something for a geek, but probably not much for a non Geek, at the other hand, a non geek understand: free for 3 months, or try it for free and you pay nothing...I'll start some experiments for my eBook and report.
You can use a different version for people who uses Chrome/Iron (I think most chrome users are tech savvy). For other browsers you can use some other filter mechanism (like referrer).
If Chrome users are like me, a lot of Chrome users were set up by other Chrome users who fixed their computer, deleted the IE shortcuts from Desktop and Quick Launch bars, and renamed the Chrome shortcuts to "Internet".
I'm personally far more likely to click something like "Start Making Bingo Cards" -vs- "Sign Up Now" because it feels more like a call to action, not a call to fill some stuff out, then do some action.
I've found "get started now" to be quite effective. One small test showed an increase of conversions from 12/126 to 29/127, so up 140% (the loser there was "Create your free account now", the winner was "get started now").
I'm running a test right now, with "try it for free" currently at 262 out of 792 clicking through, and "get started now" at 269/778 (so up 4.5%). It's not yet conclusive, but at no point in the test has "try it for free" outpaced "get started now".
If my results ever show themselves to be "conclusive", I'll post about them at abtests.com. I'd encourage you to try "get started now", though, to see what you find.
"There is no reason why 'It’s free' should work better than 'Sign up for free'"
Actually I disagree. The phrase "sign up for free" contains some hint of a potential "but", similar to "free to join" or "download trial for free", or other carefully worded phrases that litter anything-but-free internet offers.
Having said that, a hint of a potential "but" (i.e. trial-ware) is good at times so that the users don't take the free part for granted and they are aware of an upgrade option.
Effectiveness of this kind of test should not just be done at the spot where you make the change but end-to-end.
It's very well possible that by placing the 'it's free!' there the signups shift to a much larger number of people that will never convert to paying members (for a freemium site).
If you only test at the point of making the change it is very well possible to actually increase free sign-ups but to decrease your conversion rate to paying members.
So, this test is about an intermediary goal, the real conversion is the rate between new visitors and paying members.
I sure hope that your overall conversion rate to paying members shows a similar increase, but make sure that really is the case.
Actually, spif mentions the following in the post:
>To be honest we aren’t sure yet [of what impact this makes to the actual signups]. We wonder how these tests measure up to the goal funnels in Google Analytics and compared to actual conversions into paying customers
This went live before we launched our premium account so don't have those stats to track.
We measure 3 related goals in GA, the homepage -> signup form; homepage -> signed up user; and new visit -> paying user (using the goals but also e-commerce values).
Looking back and learning from this exercise, in the next tests that we're doing we'll be adding different UTM campaigns to the different variations (in VWO) and be able to track conversions to paying customers further down the track. I cannot stress enough how much valuable information this gives.
Although the two words weren't revolutionary we were quite surprised by the increase. Mind you we had about 6 other variations as well, including "It's free up to 250 contacts" and an orange version of the "Signup now!" button - both we thought would have higher conversion.
Right now it's about 1% of our raw unique daily visitors.
The tricky thing is to drill down and see which users these are and where are they coming from and how are they using Soocial. Any tips are welcome! ;-)
Yeah, true. We are looking into l10n and I18n. Right now just under 50% of our users are from the US. There is a seeming correlation between high conversion rates and english speaking countries and this would confirm your point.
In order of highest conversion rates are:
New Zealand (1.16%), United States (0.98%), Brazil, Australia, Ireland , UK (all around (0.79%), Singapore, South Africa (around 0.6%) then it levels out.
Now if only you could modify the video on your site or post a blog post to explain... how does my site get loaded into your 'visual editor?' Does all my css stay there? etc. The video makes it look like the site has magically been loaded into the editor and everything's intact :P
To me, this is the #1. I would go so far as to place that 'first_time_ab.php' box directly at the top of the page. Kind of an 'enter your webpage and get started NOW' type call to action.
After seeing this link, it's totally clear to me and i will definitely sign up when we're ready.
Thanks for your feedback. We will definitely A/B test your suggestion. In fact, had been meaning to test the demo with the input box for a while now.
I'm trying to do a test, where one variation has this text box and nothing else (kind of like Google homepage). My only worry is that it may spook some less technically savvy people into thinking we have hacked their website!
I agree strongly. If I can paste my url into a demo site, and and get a taste for how easy this would be for me to a/b test my next question is what do I have to do to make this happen? A blurb telling me all it takes is copying and pasting one line of code takes care of that. Maybe show a video showing how easy it is - how long would it take? under a minute?
EDIT: Just confirm, I think I was incorrect in the message below.
I'm not completely sure I'm correct (please correct me if I'm not!) here but as I understand it the article does not support the claim in the headline. The headline claims that signups increases by 28% in the changed version and that this was all attributable to the change.
It's the second bit that isn't supported, they say that the result was statistical significant but what I understand them as saying was that it was statistically significant that the new variation was better than the control. But it could be better by an amount more or less than 28%, all we know from that is it's almost certainly (95%) at least a little better. We would need to know the number of trails to be able to get a certainty for the amount of improvement.
Could someone with a slightly better understanding of statistics chip in maybe? I could use some more information in my own A/B tests, sometimes I know a change is going to be a pain to maintain so I want to know not just if it is better, but by how much.
Since I am the one who wrote the case study, perhaps I can explain. Loosely saying, what the article actually means is that the observed difference of 28% or more may actually be true in 95% of cases. However, there is a 5% chance that that difference is a lesser than 28%.
If you have any specific question, please feel free to ask.
How are you using to arrive at that conclusion though? When I do a A/B testing I (or rather, the software I use) use the chi-squared test to give a confidence value expressed as a percentage. When I get over 95% I know I have statistical significance and I end the test. At that point I also have an "improved by" number but I would have to collect much more data for that to be statistically significant as far as I understand.
I'm not an expert in statistics, but I'm careful about what I do, and I know the limitations of my knowledge.
However ...
I believe it is flawed methodolgy to run a test until you get a significant result. I'm pretty sure I read something lunk to from HN that discussed this at some length. It's possible - if you simply run your test until you get significance and then stop early - that you will get significance because of random fluctuations in the middle of your trial and stop early when you shouldn't.
As I recall, you should decide on the length of your trial at the beginning, then run your stats at the end.
I'll try to find the article in question, but my Google-Fu is pretty poor today for some reason.
HN won't let me reply to your answer below (presumably to stop me from flaming you :p). But thanks, I'll read that blog post later but it looks like it answers my questions plus a bit more.
Seems my understanding was even less than I thought :)
you are right - without the sample sizes (for before and after) there's no way to know whether this measured increase of 28% is statistically significant or not.
The results could really benefit from some inferential statistics. The percentages, while interesting aren't particularly meaningful on their own. I realize you may not want to share how large your sample sizes are, but something like a Pearson's chi square test would be very useful here.
I read somewhere saying that "<strike>2.99</strike> Free!" works better that just saying that "Free!" because it's hard for users to understand the real value of the latter.
At least in Europe, I'm fairly sure you'd run afoul of unfair competition laws with that - you're not allowed to run "infinite sales". If you show a non-sale price, you must have charged that much prior to the sale, and your sale must have an end date.
Just adding: I believe the end date is a pro-consumer thing and for me it's really essential because I like to make good decisions and that offers me a possible deadline days away instead of now or never.
Not how. Who: competitors. And yes, this sort of stuff is enforced in practice. Initially you might only get a cease & desist notice, but you'll probably be reported to the authorities if you keep doing it, or you're doing it especially egregiously.
I agree. When somebody uses this sort of second order percentage, it is confusing.
When a car's top speed increases from 105mph to 120mph, after you modified the engine, you say it increased 15mph ... or, perhaps you could say it increased 14.3%.
Using this paradigm, when a conversion rate increases from 14.5% to 18.6%, you would say it increased by 4.1% ... or, perhaps you could say it increased 28.28%
Woops! We have a problem. We just expressed the same increase, in the same units, using two very different numbers.
This is obviously a problem. The reader doesn't know what you're talking about.
It's perfectly valid to say the car increased by 15mph, so it must be perfectly valid to say the conversion rate increased by 4.1%. In fact, I think that when most people read something like this, this is what they assume, that the delta between the old conversion and the new conversion is that percentage. At first blush, saying the conversion rate increased by 28.28% should be OK, but it's not OK, because the whole point is to convey information, and you have failed to do so. Now you have to use context, to figure out what the hell they meant. This is unacceptable, especially when talking about mathematics, we should not need context to sort out what's going on.
What's the answer? I don't know. But there's definitely a problem here.
>Using this paradigm, when a conversion rate increases from 14.5% to 18.6%, you would say it increased by 4.1%
No you cannot say it increased by 4.1% and that's because you are talking about per cent. I don't know why you would ever use 4.1% in this context. It is simply wrong.
May be you can say 4.1 percentage points but I don't see a need for it when you can say (and be right) that it increased by 28%.
To be sure, the fact that we are talking about percent changes something, which is what I spent several paragraphs exploring.
To say that subtraction of percentages is against the rules, (while perfectly valid and often used in other units) but percentages of percentages is not against the rules, requires a bit more than "It is simply wrong", don't you think?
Sure, you're "right", when you say it increased by 28%. The point is, perhaps you shouldn't be "right", and what is "right", anyway, if nobody knows what the hell you're really saying?
We see this when talking about tax changes. If somebody wants to minimize the sticker shock of a tax increase, they use the delta. So, if the rate rises from 3% to 4.5%, they say they increased the rate 1.5%. If you want to maximize the sticker shock, you say that the tax rate was increased 50%. Either side can say they are doing the math "right".
Those who care about actually conveying the information, rather than produce sticker shock of either sort, always have to spell it out carefully, by actually saying, "the tax rose from a rate of 3% to a rate of 4.5%" This is a bit longer, but it's necessary, given this weird unit.
Why not? It makes perfect sense and conveys the relevant information. You should talk about percent changes whenever the important bit is the ratio between the new and old value.
