I used to be interested in this type of service, but then I got an SSD. With an SSD it takes about 5 seconds to boot windows through parallels, and now cross browser testing is no longer a chore.
I am willing to admit that I pirate windows for the sole purpose of supporting IE with my webapps. It would be too expensive to do it legitimately, so my solution is not for everyone.
I have found multiple IE just doesn't work the same a real IE install, and it is hard to duplicate the same bugs as users see. As you can see the top rated answer is "don't do it, use VMs".
If they are the same ones we tried, they are a giant pain. Lots of screwing around to get everything downloaded and set up right for multiple version of IE. Then they expire every few months, requiring regular doses of rigmarole.
And when you finally have them set up, you know what prize you win? You get to debug some crazy-making IE issue. As far as I'm concerned, free isn't enough: they should come with masseuses and bottles of good scotch in compensation.
Yes, I do. Which is part of the reason I pirate windows, because it is the only way I know how to get an IE6 install.
It is reasonably fast to setup, because once you install XP, you can copy the VM, and install IE 7, copy the VM and install IE 8. I then do the same thing for Vista/Windows 7.
With 8 Gb of ram, I have no problem running 3 Xps, and 2 Windows 7, at the same time.
"With 8 Gb of ram, I have no problem running 3 Xps, and 2 Windows 7, at the same time."
That's good to know. I'm in the market for a laptop to replace my desktop, and I want enough RAM to run multiple VMs like that. Was thinking I'd need to find one with 12 or 16GB, which rules out MacBooks (unless I get an expensive 3rd party 2x8GB RAM upgrade). But maybe not.
I configure XP to have 512MB of ram, and Windows 7 to have 1 GB. This works fine if all you are running is the OS and a browser. I will re-iterate that an SSD is the important factor because it cuts the VM startup time by 75% in my experience.
I used to buy Windows licenses, even multiple use family packs. And I would install them in Parallels on my Mac for testing IE. That works, until your Windows keys start to expire. I guess I installed them too many times. Now my keys no longer work.
So now I pirate copies of Windows that I have paid for, because it's too much hassle to work with the legitimate versions.
Browserstack is great, but it's still catching up to Sauce Scout (disclaimer: I work at Sauce Labs). It needs better support for testing firewalled or localhost sites, and it needs security (it re-uses VMs)
Awesome work, guys! I know first hand how hard it is to maintain a product like this :)
A bit off-topic, but i find the Plans & Pricing page to be a very good example of how to design for an attention-lacking crowd, i.e. most web users. In particular, the team-size icons is a rare example of using eye-candy to succinctly help in clarifying for whom each package is tailored. Or maybe i am just easily excited today!
If anything, i would invert the per-person price with the overall price in order to give emphasis to "more people is actually cheaper". There is something in descending prices that makes people feel good.
This looks interesting - we've been pretty happy with crossbrowsertesting.com until now, which is also worth a look.
Here's what I'd love next - we'd write a bunch of QUnit javascript tests, put them behind a private url on our staging server, and then be able to run those tests from the command-line on multiple browsers (through BrowserStack, say), and spit back the output in a form that Jenkins understands. I'd certainly pay $20/month for that, and maybe more if it was great.
This looks cool, but I'm not sure about the long-term value.
Looking at their browser list, I could test all Firefox, Opera, and Chrome versions, one version of IE, and the Windows versions of Safari on one Windows 7 box.
I could add to that three VMs for older versions of IE, and a Mac (maybe with a VM) for Mac Safari versions. If the main Win7 machine is also a VM, I could do this all on the Mac.
So, that's a total of 5-6 OS licenses, which would cost $750-$1000.
At $19/month for a single user, BrowserStack becomes more expensive after 2-3 years. At the small team rate of $68/year it becomes more expensive after one year. (I could host VMs on a server and have my testers RDP into it; they can't use the VMs simultaneously but with 4 testers and 5-6 VMs that's not a problem.)
To me, the pricing is right on the edge, which forces me to evaluate the effort of maintaining local VMs and browsers (an invisible cost is most organizations) vs the potential risks of running my in-development software and providing access to my development environments to a third-party over the internet. If the pricing was lower and clearly less expensive than maintaining my own VMs, it'd be a slam-dunk decision and the other factors wouldn't even be considered. (I'm thinking about this from a "how does a developer sell this to corporate management" perspective.)
Installing and configuring VMs up to the point where you can do good testing with them is a painful and tedious work. Why waste the time of expensive professionals on something if you can fix the problem for $70-$150 a month?
$70 bucks a month is only real money if you're a broke startup. Otherwise it's peanuts.
And don't forget the maintenance you have to do on those machines. You're testing is 5 minutes under way and BAM Windows updates wants your attention for a reboot.
Just tried this on my in-development web app. The good news is that it works like a charm. The bad news is that the browsers keep prompting to save the password of my test account. I wish they'd turned of password manager for all of the browsers.
I also feel a little weird typing in my test account password into their site.
Bravo BrowserStack team! I am using this to test IE7 on our staging environment. Your signup process was great, I was able to jump-in and start testing immediately. I think you could add better communication and assurances about security and private information. Keep up the good work.
An API would be great for us since it would let us to run our tests against all web browsers by making some http requests. Wouldn't it be revolutionary?
As I just mentioned elsewhere this thread already, browserling is rolling out exactly this service really soon! Here's a sneak peak: http://browserling.com:9088/
I typed "neato!" into their search bar; they promptly logged off.