Oki-doki. It's a good start, but you need to take it a notch up to make it actually useful.
Specifically, let say I am on a receiving end of reviews made by some random people. I don't really care what they have to say unless it's credible or representative.
Let's look at latter, as it's an easier one. The way to ensure a review is representative, is to ask other people to look at the review and say if they agree with it or not. Once it gets 10-20-30 meta up/down votes, it'd be more or less possible to say if the review is meaningful or if it was a brain fart.
Make the meta reviewing process mandatory. You want to get your site reviewed, pull a "meta" duty on a dozen of past reviews and (perhaps) write a review. When selecting a review for meta, do it randomly, but favor those with low vote count and controversial ones (over those voted way up or way down).
This would be an MVP of your idea. Without this, it's nice, but it's generally useless.
(edit) Oh, and perhaps simplify reviewer's life a bit and have 3 sections - "what's good", "what's bad", "free-form comments".
"This would be an MVP of your idea. Without this, it's nice, but it's generally useless."
But you've pointed out at least 5 new features. Let's say they add all of them and it fails.. Could they have learned that with only 4? 3? 1, new features?
To me, the MVP of this idea is that someone can review a site and have their site reviewed. Everything else is just a feature. Not to say they wouldn't greatly enhance the product, but MVP literally means minimum viable product; not minimum awesome product.
Thanks, that was exactly the idea -- solicit feedback on something that works but hasn't too much 'chrome'. I received more than 200 really useful suggestions (and bug reports) and the tool will definitely evolve. It already is.
I need a terms and conditions link. I want to submit my company's website but I want an assurance that the feedback won't be visible by the general public; i.e. through a google search for "example.com review" or something. and Ts and Cs page or data policy would clear that up. As it stands, I'm reluctant to use this service.
If you work for a company that requires secrecy, you should probably go with a professional evaluation anyway (via consultants and user testing).
I think this site was meant for quick 1 or 2 sentence feedback from the general public for starter/personal sites and the feedback examples on the front page of their site clearly shows this.
What prevents anyone from submitting your website on your behalf? Do they use some kind of "put this HTML file in your root folder" or something similar?
Oh, nothing at all. I'm just saying that the reason I haven't submitted it is because there's no indication of what will happen when I do. I just don't want "example.com uses criticue" plastered on their twitter/facebook/etc, and there's nothing reassuring me that it's semi-private.
I also wasn't specific enough; I'm talking about the website of the company that I work for, not one I own. So the reason I'm reticent about using the service is that I'm merely casually interested in what people have to say, rather than submitting the site in an official capacity.
I don't agree with your reason but am totally behind you when you say there should be a t&c page. I need to know how my user will be treated and if the founders care enough about my placing their trust in me etc.
Loved the concept. And kudos for a frictionless onboarding process. Just submitted 3 reviews. Some suggestions:
- I forgot to submit the "star" ratings. Just entered the comment. May be you can have 1. give rating, 2. provide specific feedback (I would not use the word "comment")
- It would be nice to see the reviews I submitted and edit if needed
- Also, I'm not clear if the reviews will be made public. make it clear upfront.
- Not sure if you already do, I would like to see "Rating" of the reviewer. That would help to judge the feedback.
Very nice idea and it's something I'd definitely want to use.
Just one issue: it would be great if the service allowed me to accumulate credit by reviewing other sites, and have my site reviewed by many people when I launch it (using my credit).
It'd be more interesting to allow some sort of optional rating system which feeds into something like a gaming matchmaking system. People who consistently send out good reviews will get a rank of "A", for example, and be matched to others with similar rankings.
That way, even low-quality posts ("i liek that u have a logo") can be accounted for, without treating them like spam. As the reviewer becomes better at reviewing, they will see an increase in their ranking and better reviews.
Sure. Better to assume it's ok and have a "flag" option, though. On the contrary, it'll put too much of a burden on the submitter and if they fail to approve your comment it won't count.
Sounds like a lot of the same problems the review forum on ozzu.com had/has. Look how that turned out. Usually when there is some motivation for reviewing other sites the quality of the reviews goes down.
OP: Hi guys, I'll do my best to reply to all comments posted here a bit later.
As it is now we're barely handling the flood of reviews. :) It's a good thing and very exciting but we had to scale up the number of heroku instances the second time in a row. So please bear with all the errors for a while, we're working on some issues esp. screenshot generation as it is not coping with the load.
Wow! We didn't expect an instant success like this. Thank you all for your contributions.
I love the idea behind this site and if I get some useful feedback on my website, I'll be ecstatic.
Anyway, your site not really a success yet. You've just been Hacker Newsed. Traffic will probably go back down after today. It's happened to me before.
Anyway, it looks like a major bottleneck you are experiencing is that you have to moderate every review that's written. I don't think that's going to be sustainable. I'd try having using rate reviews "helpful" or "not helpful" like they do for reviews on most online stores. And then maybe have an automatic filter for easily detectable things like foul language or gibberish.
a. I can't find a way to sign up as a pure critic or a viewer - a user without a website.
There will be plenty of people (the category also known as end-users) who don't have a website yet who would still like to look at and review designs by other folks.
I wasn't able to sign up because my website is down for now.
b. It'd be nice to have a way to 'save' credit and expend it later on reviews for sites that I may build later.
I didn't get an email confirming my account, on the sign up area after doing my first review, I tried both of my gmail addresses and have yet to see anything in either.
Cool concept, add a meta aspect as others have said.
I just wrote a lengthy review about a site that I feel could be improved but after clicking Submit the button went into a loading state and nothing has happened since. The comment field was blanked out after I pressed "Add comment". Any way to get my review back in the JS console?
Re the site itself, I think it's a good idea but I agree with the meta-scoring. Add a point system that levels you up both by writing a review and later by reviewing reviews of the same site/commenting on them. The higher your level or the more points you have, the more your sites will be reviewed.
And please make https sites fail more elegantly. It's okay that you can't load them in an iframe, but make it look less broken. I submitted an https site and I doubt many people will review it when they are met with a huge screen saying "http only" through no fault of their own. Instead say "This site is using https (learn more), please click here to open the site in a new tab"
Similar experience here. What's the difference between comments and reviews for that matter? There's an add comment button and an add review button, apparently tied to the same field.
Also got some after-the-fact request to sign up before getting the review of my site I was promised... not too interested in that.
Yeah, the 'Add comment' link is pretty confusing. I'd consider moving the '1 comment' text below the comment box, and display the truncated comment text instead. It felt very much like the comment I'd entered had just disappeared with the current implementation.
Cool idea. Great job on launching - I'm sure there were many things you'd like to clear up and fix for a "proper" launch, but you finished something and you got people using it.
As many people said already, it would be great to know the difference between "Add comment" and "Submit review". Seems like they could do the same thing.
Keep polishing it, and then post a changelog somewhere - it's always nice to see the list of bugs people have worked through at the early stages of a site. It's also really motivating for the developer as well.
Thank you for the positive feedback. I really appreciate it.
"Add comment" is on its way out. There will be no way to post more than 1 comment, as ppl. post 1.1 comment on the average so it's just unnecessarily complicating things.
I'm keeping a backlog; good idea, posting it, thanks!
You might want to make sure that you're first on queue if you sign up your site and do a bunch of reviews.
I just did three reviews and it should've been possible to make sure I got one in return by now, but that didn't seem to happen yet (or is it because my site is HTTPS only?)
I can second the comment about star ratings as well; the default of 1 star especially was not in line with my intent for at least one of my reviews.
Unfortunately, it looks like you're already getting spammed. I just reviewed Google.com and what I think would have been nsfw site had I let it load.
Also, it takes too long to generate the screenshot. I waited a few minutes before giving up on my third review. Why not just open the site in an iframe?
Getting to try it out without having to log into anything is fantastic.
There were some temporary problems. Google.com sites were also eliminated, some 'spam' will leak in but anyway provided the people contribute, they submit google.com or microsoft.com because they're not ready to use their own site yet. An option to just post reviews without submitting site will take care of it. Thanks for the heads up!
Generating the screenshot was slow but didn't take anywhere near a few minutes for me. Maybe ~5 seconds. Site probably getting a lot of traffic, I got some errors at first but it works fine now.
Ah, I checked when it was first posted but it seems to be better now. It was probably the site itself and not the screenshot functionality that was the problem.
As suggested in another reply, linking via iframe would still be a nice option to have when submitting.
Thanks for the follow-up. IFrame had actually been there in version 1 but people use really non-standard stuff to break out of frames and some are not even aware of it because they use third-party frameworks.
Screenshot + link to open in a new window actually started as a sort of a workaround but I think this allows people to focus more on first impressions instead of going into usability testing.
Great idea, but the UI is a little confusing. After writing a comment do I need to click 'Add comment' AND 'Submit review'? Or does either do the job? Also I forgot to set a star rating, perhaps default to zero and prompt user if they don't set it? Why can't I see the reviews/comments I've submitted on my account page?
Thank you for pointing that out! I may want to try it in the next release. It would require a clear point system though. I want to keep things as simple as possible for the time being.
After some use, I definitely see the direct and immediate value of getting feedback from an alternative perspective. I'm hearing people question the credibility of the reviews but I don't think that's the point. If someone tells me my header graphic is illegible or my site doesn't display in IE6... that's helpful as is.
That said, this feels a bit like a Ponzi scheme in the sense that I'll only get reviews as long as there are new/active users... I could see that being a long-term issue (as with any site following this model).
Uh oh. I added my website (http://www.pisoftware.com in case you want to look it up) and did a review of another site. Signed up for email alerts, got no confirmation email, and I can't see the page for my site anymore. Something no bueno.
That's my immediate critique of a website critique website called Criticue. It's like Inception.
Email probably in spam, esp. if you use Gmail. I counted 5 reviews for the site. Have they been useful? If not, feel free to email me (martinb __at criticue __dot com).
Services like this concern me because I don't know the credibility of the person on the other end, nor do I know if the effort I put into someone else's review is going to guarantee that I get an equally verbose one. In the same vein, the function of a site defines the design and someone telling me that my content portal "is way too busy" isn't going to help me when I wasn't going for some sort of minimalistic trend the reviewer may be more likely to rate higher.
I'm not sure if you're changing things right now or A/B testing, but when I initially went to your link there were examples of what I could expect. Now there's no way of knowing what the process looks like and how these viewers see my site. Is it a screengrab of the homepage, do they get the link and surf around?
Also, is there anything stopping someone from writing a bunch of useless one-liner reviews just to get their site seen?
Thank you for the feedback. I'm working hard to improve the service as much as possible.
As far as spamming, reviews are moderated but one-liners are very often passed through, because they are useful. The idea is to get, say, 10-20 reviews of your site, very quickly.
I suppose as a designer I'm probably not your target market anyway, since I have other outlets to get critiques. One-liners would irritate me too much. :)
Can you make a version for startup ideas, instead of websites? Keep everything else exactly the same. Then add leaderboards, scores, etc.
So many discussions here about startups, I feel like we have enough aggregate brainpower to create a great peer-reviewed top 50 ideas list. Then all we need to do is create teams and crack on!
Like the concept. Just submitted 4 reviews and haven't got one back for my site after 15 mins. Any way of shortening the queue to provide a quicker reward for my critiques would be awesome. It'd also be nice to see your site's screenshot to know what people are looking at when they review my site.
Moderating is a bottleneck but it'll improve. There's currently a queue of sites but the idea is to make it as short as possible. Yeah, instant feedback would be really cool! :)
I like the idea but I fear for the lack of quality. Whenever you have "outsourced" n-to-1 projects to platforms like Mechanical Turk you know you need something to guard the quality of the input that people are entering. It's good to get positive feedback on your work, but what do the comments mean otherwise?
"Very clean, I like it" - clean in what sense? Design-wise? Or is the HTML markup nice & tidy?
"Nice big calls to action" - Nice & big, but is bigger better? Do they work?
Etc.
I am keeping my eye on this one, could definitely evolve in something (even more) valuable. Keep it up!
The concept is flawed. Getting random feedback from random people is not at all useful. Feedback needs to take into account the target users and the goals of the design. The designer needs to ask specific questions, not just "what do you think?".
Maybe this product would be useful if users could put out a call for people who meet certain criteria (i.e. music-lovers, 25-40), and then focus their attention on a small set of questions. In this way, there's a possibility of getting something useful to go on.
what does that mean? how/when/why will I lose access to that account? I really want to see my site reviewed but I do not actually want a "proper" account.
The current length of the waiting queue is about 2-3 hours. Please feel free to contact me at martinb at criticue dot com if there are still no reviews after some time.
I like where you're headed with this. I'd really like it if you provided an interface for marking and critiquing specific parts of a web page. For instance: let me draw boxes and drop pins and write comments specifically on those areas.
In short, having a single comment as you would a typical Amazon product review provides less depth than I'd like to see from peer reviews in this domain.
Well done. Good concept. I look forward to seeing you make some of the enhancement that others have suggested. The meta-reviews is a great idea.
Also, signup email went to Gmail spam for me.
Additionally, a feature of perhaps marginal utility would be to see all the sites that I've reviewed (e.g. I want to take another look at a cool looking site that I remember reviewing but can't remember what it was).
A Hot or Not clone for screenshots that does nothing to give the designer/developer actual feedback and unfairly pits e-com sites against content-heavy sites and brochure sites?
Only Issue I have is that all the feedback I get is "can't click on any links". I know that its a screenshot (hence you cannot) but you might want to add that its a screenshot other then the loading message somewhere. I suspect a quick check of your feedback logs will show a lot of people with the same comment.
The button says "Go to [website]" but points to the same page.
I think this has a lot of potential, but it should be easier to offer better UI controls and testing. For example, I may want users to sign into a dummmy account or see specific pages. The home page of a website is rarely good enough to get meaningful impressions.
What about reversing the order? Make it so you can't submit your site for review until you've reviewed another. Obviously quality may suffer, as people will just type garbage for the sake of getting theirs submitted, but with volume that can be fixed via other means down the road.
Good concept because professional review sites charge an arm and a leg for just pointing out the obvious (for the most part).
I was stoked to review my first site but the first thing I got was a shitty image porn blog. If you want it to be legitimate you gotta weed out the smut for people to take it seriously.
I agree with all the others who say this is a neat idea. I've reviewed several sites, but there doesn't seem to be any confirmation of this on my dashboard. There is a button to "Get review" but in reality I should already be inline for a few, right? The dashboard should say so somewhere.
You get _another_ website for review. Once you post your review of that site, your review -- 2600hz.com will get queued for review by another designer.
If that explains it then I'm sorry -- I'll try to make it more intuitive in the next update.
Great idea, getting close on execution. I've reviewed 5 sites so far though, and all of them are awful. Would be great to put together a list of 1000 great websites and let users review those, because it makes being a reviewer more fun.
The star rating seems almost unnecessary, compared to the rich feedback you get from the comments. I notice my time incrementing, but what does that mean to me?
Overall, nicely done. LOVE the on-boarding process.
Great job on the UI and UX! I think it would be more valuable if the reviewers are from the target customer/market, not just random people on the Internet. That would be a review service worth paying for!
I'm actually using a Mac to write this on with Chrome + Safari. There was a problem with too much traffic (fixed now by scaling the site up), I suppose CSS might not have loaded. Do you think this could be the case? Thank you.
I got a free one. Submitted two reviews and have gotten three back so far :)
Idea: take a page from the Yelp/Quora/HN/StackExchange playbook and gamify it by letting people build up some kind of reputation points/karma/credits or whatever. You'd be amazed how much time some people will invest in your site just for some pixels on their profile.
Specifically, let say I am on a receiving end of reviews made by some random people. I don't really care what they have to say unless it's credible or representative.
Let's look at latter, as it's an easier one. The way to ensure a review is representative, is to ask other people to look at the review and say if they agree with it or not. Once it gets 10-20-30 meta up/down votes, it'd be more or less possible to say if the review is meaningful or if it was a brain fart.
Make the meta reviewing process mandatory. You want to get your site reviewed, pull a "meta" duty on a dozen of past reviews and (perhaps) write a review. When selecting a review for meta, do it randomly, but favor those with low vote count and controversial ones (over those voted way up or way down).
This would be an MVP of your idea. Without this, it's nice, but it's generally useless.
(edit) Oh, and perhaps simplify reviewer's life a bit and have 3 sections - "what's good", "what's bad", "free-form comments".