>I’ve been wrestling with whether to bring back Craigslist listings in the search results. I’ve found a way to include them that I’m told is legally kosher since it doesn’t touch their servers at all, but it still seems somewhat dickish to go against their wishes in this, and I’ve always had a lot of respect for what they’ve done for the world. Also, who wants to waste their time in court?
>But then I did some back of the envelope estimates of how much of people’s time and effort it would waste if I didn’t, and it became clear how much less nice it is to waste the time of millions of apartment hunters out of stubbornness or some clearly inaccurate assumption about the will of the community.
Christ, what an asshole.
Translation: "We found a loophole that lets us get around the spirit of what was communicated to us so we could continue to build out our product. I'm going to conveniently step over the moral grey lines of using someone else's data without their consent by claiming my service is better."
Ouch, sorry, I thought it was the right thing to do, on the balance. Like you said, it's a really morally gray decision, either way, but I think it's actually worse to do nothing.
Dude, I have no doubt your service IS way more useful than craiglist. I look forward to using it when I next move.
Just… don't dance around your motivations here. You want to build a better service, and you need CL's data for that. CL doesn't want you to use their data. You said that's too bad.
The why is just as important as the what in these sorts of decisions. So, this post was my attempt to explain the thought process I went through that finally tipped the scales. Some people make life better via charity, I try to do it via software. So that's what's tipping the scales on my moral compass. Money can be useful for that too, but mostly as a tool, it's not my main motivator.
I would respect you more if you just came out and said "fuck Craigslist, we're using their data" than go through a bunch of pie in the sky philosophical reasoning to justify your decision after the fact.
Let's be frank and direct: you're being a dick to Craigslist, and you're doing it so you can build a better product.
I even agree with that decision--Craigslist has brought it upon itself by creating what is (IMO) a mediocre and clearly substandard UI. But your choice is what it is. Embrace it.
The idea that he's "being a dick to Craigslist" is the weirdest part of this whole thing. The PadMapper service consists of advertising that certain content is available on Craigslist. I mean, it literally consists of saying, "oh, you're looking for a 3-bedroom house under $3,000 in this neighborhood? Here's a page on Craigslist you might want to check out." That's the extent of the appropriation.
I mean, imagine I was reading the Craigslist apartment listings for my own housing search, and I knew you were looking for 3-bedroom Craigslist listings in a given neighborhood, so every time I saw one I sent you the link. Heck, imagine I even charged money for that service. In what universe am I being a dick to Craigslist? How on earth does that follow?
I understand that it's a little different here because Craigslist has asked PadMapper not to help people out that way. But the information PadMapper is passing on, it's not getting from Craigslist anymore. It's getting it from a third party. So it's the equivalent of me saying, "hey, you know you were looking for a 3-bedroom in that neighborhood? I'm not allowed to visit Craigslist myself, but a dude told me about these pages on Craigslist you might want to check out." There is purely no way that this constitutes dickish behavior toward Craigslist, and Craigslist has no moral right to forbid it.
Yeah, I agree, it's just that advertising them when they don't want to be is dickish. It's not super dickish, but it is somewhat, and I feel a little bad about that.
> "Disintermediation -- getting your users used to coming to my site to look at your data is the first step to making you irrelevant and forgettable. PM knows who the renters and the listers are; it's a small step to convince some listers to list with PM first. Maybe PM will agree to repost on CL as well, but as long as listers come to PM first, the relationship between PM and CL is flipped."
Oops, I think I misunderstood what was being said. Sure, if PadMapper became the main place renters go, then it could pose a risk for Craigslist. I don't see that as likely, though - it's well structured for the technically literate, but my impression is that it's somewhat confusing for many of those who aren't. To add to that, it's extremely hard to build a brand strong enough to reach the landlords of the world, who are on average older and less likely to be early adopters of hip web technologies. Anecdotally, most PadMapper users tell landlords that they found the place on Craigslist, because it would confuse them to say "PadMapper". So, in my opinion, it's unlikely to be a real threat. Hope that makes sense.
Perhaps, the dynamics of a system like that are impossible to fathom beyond abstractions, especially if you've never observed a similar situation (I haven't). If it asymptotically reaches a relatively small subset of all renters, and it never becomes the main place renters go, then I don't think there will be a tipping point.
"...I knew you were looking for 3-bedroom Craigslist listings in a given neighborhood, so every time I saw one I sent you the link. Heck, imagine I even charged money for that service."
You mean like a realtor? Maybe craigslist should go after realtors.
> I would respect you more if you just came out and said "fuck Craigslist, we're using their data"
The material facts of a listing on Craigslist don't belong to Craigslist. You can't copyright the fact that a given listing is for $2400 a month. You can't copyright that it's at a given street and cross-street and the number of bedrooms and bathrooms on the premises. I don't even think anyone should copyright the aggregate of the above data. To do so would be evil. (In that there would be widespread implications.)
It's not their data. It's their listing. Those are two different things.
Are you saying that Craigslist doesn't own the data, but it still owns the aggregation of the data?
I have no real skepticism that what Padmapper is doing is legal, and Craigslist doesn't own that data in any legal sense. Sure.
But once you get down to arguing "yes, I have a legal right to use this data, as your attorney can see by looking at section VI.A.5.b of the Online Bullshit Act," you've already given up the game. Legal arguments are ones that are essentially saying "I have government violence backing me up." Padmapper likely does.
But if that's what it's coming down to, it doesn't change the fact that Craigslist is being screwed. Maybe they had it coming and maybe it's good for the grand moral arc of human history, but whatever: they provided the forum and a way to structure and organize all that data, and without that structure it'd be hard to pick out the signal from the noise of the internet.
In this case, just like in most cases about freedom to use information, a legal argument means the opposite: rather than saying that they have government violence to back then up, PadMapper is saying that Craigslist doesn't. In the absence of any sort of enforcement, Craigslist would not be able to do much about people's using their data, after all.
Now, I'm not commenting on whether their behavior is justified or not; I'm merely talking about the role of the government in this particular case.
> Legal arguments are ones that are essentially saying "I have government violence backing me up."
I am not arguing from that standpoint. I'm thinking about the implications for the freedom of data.
You are essentially saying that anyone who has done work resulting in the aggregation of public data somehow "owns" or holds the rights to the aggregation of that data. By this same logic, the first company that published a general-interest encyclopedia would own the general concept of the same aggregation of data. That can't be right, otherwise there could only be one encyclopedia in the world. By the same token, the current search companies own the reference graph of the web, and anyone who starts another web crawler is in violation.
In so far as what Padmapper is doing is the re-aggregation of already public data, he has to be in the clear, otherwise the concept of "owning" an aggregation of data becomes restrictive.
How would Padmapper be using "government violence" when they successfully counterargue Craigslist's attempt to use "government violence" against them, resulting in no damage to Craigslist?
> Legal arguments are ones that are essentially saying "I have government violence backing me up."
Even if this phrase served your larger argument regarding Craigslist's moral position -- and I don't think it does, given that thus far, Craigslist is really the first to threaten to make this a matter requiring state enforcement -- it'd still be a bit mixed up.
A legal argument -- like any other argument -- can be about persuasion based on established principles instead of compulsion. In fact, that's essentially all it is through the point the parties involved are subject to an arbiter whose decision is still essentially based in the relative persuasiveness of the arguments.
It's true that at the stage where a ruling is either ignored by an unhappy party or enforced by another party, violence of one form or another may be required (if there is to be a system where there's enforcement -- and not all systems choose this). But between that point and the "legal argument" stage, there's enough of a buffer zone grounded in discussion that any easy equivalence between the argument and violence itself should make a careful thinker suspicious.
In the UK and EU, database rights do exist, although Padmapper could argue that Craig's List makes no investment in actually compiling the database since listings are user submitted (as opposed to the investment they continually make keeping the database accessible).
> I even agree with that decision--Craigslist has brought it upon itself by creating what is (IMO) a mediocre and clearly substandard UI. But your choice is what it is. Embrace it.
So are you saying that if CL had a more elegant UI you would be against the OP continuing agasinst CL's wishes? Seems like very arbitrary ethics.
If CL had a more elegant UI, I would also be against the OP continuing against CL's wishes. What makes CL the bad guy in this case is that they're locking up a critical system-economic element, not doing anything with it, and trying to prevent others from using it; exerting minimal effort to progress human economics while also exerting legal force to prevent others from progressing human economics. Consequentialism! Consequentialism! If CL had their own Google Maps API then the loss of renter-hours would not be nearly as severe and the negative consequences of Padmapper not adding the service back in would not be as bad, nor the gain to other people from adding it as great. Of course the right thing to do might change, if the consequences change!
You consequentialist morals then seem to be operating under multiple unfounded assumptions:
1) that the only thing that a company such as this could be doing with their value/income is building "an elegant UI" (it seems an insulting stretch to believe that their 30 employees are just sipping martinis at their office; one would imagine that there are complex social management problems that Craigslist has become experienced with and spends most of their time managing)
2) that "an elegant UI" provides positive value to this type of interaction (there have been reasonable arguments in previous PadMapper v. Craigslist posts that, in fact, the UI provided by CL is semantically more optimal than competitors providing fields; part of this argument hinged on the benefit of not requiring certain kinds of information, which makes some sense if you look at the low-key way people prefer to interact with it)
3) that there would be a long-term benefit to handing these keys to PadMapper <- this one is exceptionally bothersome due to your multiply-emphasized cry of "consequentialism"; please remember that the long-term goal of companies with this business model (I have consulted for multiple, and have multiple friends personally involved with them) is to become the portal and then marginalize the data sources as they become the new de-facto standard: in this case, PadMapper's service called PadLister.
4) that the result of this set of policies would not actually increase the number of "renter-hours" wasted, due to the (I will happily argue, but will not bother here as to me the mechanism is trivial and obvious) likely outcome that there would no longer be a centralized source for this kind of data, and renters would instead end up scouring numerous sites in the attempt to piece together who was renting what where, despite "elegant UIs". (I will point out, in case it isn't clear: I am not arguing that this is necessarily a net negative, but your position did make this assumption, so I am pointing it out)
There is absolutely no reason to believe that PadMapper would be a better long-term holder of that vision than Craigslist, and in fact numerous reasons to believe that they would actually suck at it (including, but certainly not limited to, the "this is a shady way to make this argument" comments in the parents of this part of the thread): you really have to ask yourself "who would I be happier with as ruler... Eric, Craig, or no one"?
Regardless, as I have others, I encourage you to read articles on the history of Craigslist. In this case, the article published by Wired a few years back "Why Craigslist is Such a Mess" comes to my mind as a rather key one to start with.
I don't think it's arbitrary, just selfish(1). As a user, I want the best experience possible, and don't really care wether I get it from Bob's House of Widgets, or Alice's Sprocket Emporium. I don't even especially care if Bob sort of copies some of what Alice is doing. I just want the best widget for my dollar/time spent.
(1) In the Objectivist/Randian sort of way, not in a negative way.
> So are you saying that if CL had a more elegant UI you would be against the OP continuing agasinst CL's wishes?
I believe it's more that if CL stopped stomping on other businesses for virtually no reason, without any discussion or even possibility of an agreement, then [I] would be against the OP continuing against CL's wishes.
Wait. Cragslist accepts and publishes classified ads, under their own policies based on how they want to [do] business. Their objecting to other people using their publication system without following their policies is "stomping on other businesses for virtually no reason"?
I wanted to explain what I was doing. In a lot of ways I still admire Craigslist, so I wouldn't say fuck them. I feel some remorse for being a dick, but that's strongly outweighed by the other side of the argument for me.
In contrast to what other people seem to be saying in this thread, I personally appreciated your explanation of the logic behind your decision. It didn't feel sleazy to me and I think I would have been somewhat turned off by a confrontational tone.
I am definitely biased though. I used PadMapper a ton in May when I was looking for a sublet for the summer. Last week I started browsing again for the next apartment and felt the distinct lack of CL listings. Honestly, my solution was to start using PadMapper (for non-CL listings) + another site that shows CL data on Google maps. I can't imagine going back to CL's vanilla interface for apartment searching.
In conclusion, thanks for making an awesome service and good luck with figuring something out with the Craigslist folks.
the blog post was changed during the time that comments were being added to this thread. from the original post: "If it takes half of PadMapper’s millions of monthly users 3 hours longer now to find an apartment, that’s over 350 man-years wasted per month, or 5 lifetimes. Sorry for cursing, but fuck that. Seriously, fuck that completely. That really pisses me off….So, effective as soon as I can bring the new code up (almost certainly by the time you read this), I’m bringing the Craigslist search results back."
YOU might respect him more, because that's what YOU would say.
But Eric isn't you. He isn't a dick. He doesn't believe he's "fucking" Craigslist. He found a solution that fit with his own moral framework. If this doesn't fit within your framework, that's fine.
The world doesn't revolve around you. So please deal with it and stop calling him names.
In a sense, building any company is a charitable act, because you leave consumers better off than they were before
Are you daft?
It's not a charitable act to start a for-profit company, because you get paid for providing the service that leaves the people better off. The whole point of charities is that they provide the service without getting paid.
(If your business is a startup that has deferred monetization until later, that still doesn't make you a charity; you've just chosen to turn away payment now in the hope that doing so will yield you a bigger payment down the road.)
That's not to say that starting a business is a bad thing or that businesses can't improve the world; but there's a clear line between what a business is and what a charity is, and it involves the expectation of getting paid. Which means a business is always going to be lower on the Do-Gooder Scale than a charity is.
Would you consider Google or Facebook a charity, then?
Or the person at a sports event selling me beer? Are they a charity? I do want beer, and they are facilitating that...for $10/glass.
(I think that most people would not consider the beer vendor a charity, and since English is a living language, I think that the cited definition is incomplete)
Obviously I wouldn't call either "a charity" because charity used in that (comparatively recent sense) means a nonprofit.
Crazygringo's point is that there is a much older and more important sense of "charity," which simply means helping those in need. In that sense, Google is a very charitable project. In fact it is hard to imagine a nonprofit doing a better job.
Not a charity, but generates huge consumer surplus (i.e. makes my life better by a lot more than they cost).
If Google didn't exist, my life would probably be $20k/yr worse due to search, $5-10k for maps, and $3k/yr worse due to Reader. Maybe $500/yr for News. Google Plus not existing would make my life better (since people who post would post on fb instead, where I'd actually read them).
I dunno about Google's definition, but Merriam-Webster's makes it pretty clear that "charitable" involves giving, not making a mutually beneficial exchange as one does in the marketplace: http://www.merriam-webster.com/dictionary/charitable
Let's say I invent a hyper-efficient, clean, alternative energy source that averts climate change, makes cheap energy universally available and stops a lot of the causes of war.
Would me also becoming the richest man on earth as a result matter to you? Would it diminish the value of what I would contribute? And if so, ask yourself why that should matter at all?
This isn't some Rand Objectivist thing. I just don't understand why we use words like charity and philanthropy that tie the outcome/value of something in the world to the self-sacrifice of the outcome's author. It doesn't seem all that relevant to most discussions about the value of a thing.
Heh corny, I know, but I think that's the majority of my motivation. Otherwise I would be making a lot more money off of it and it would be a very different site.
Padmapper certainly helped me find an apartment. Twice, and the second one was so difficult that I might not've successfully moved to be near the rest of the Singularity Institute (no relation to SU), if not for Padmapper. If you did it mostly out of charity and haven't been trying to profit much, then I am even more grateful for that. Please let this flow of positive reinforcement encourage you to continue doing it, and ignore negative reinforcement from haters, because haters gonna hate. (You should see the hate I get for the things I give away ad-free online!) Also, consider making larger profits, because I wouldn't hate you for it, trust me.
Add me to the countless number of people who found their place on padmapper (just moved in last week!) and are thankful for its existence. It's a great service.
Agreed. The argument seems to be about the semantics of the word "charity". That's entirely beside the point IMO. You're providing a service. And a quality one at that.
Last month, after spending hours filtering through CL listings to no avail, I used padmapper and found the perfect CL listed apartment in 10 minutes.
Frankly, I don't care about whether you're doing this to help people, or you want to get really filthy rich. What matters is your actions. AFAIK, you're not massively ripping me or anyone else off, you're not torturing puppies, and have no plans for world domination.
Yes, you knee-jerked on a contrasting of what he does with charity. He does claim that his software improves life, and this is his stated motivation in creating it. You may object to the latter?
I think that the statement you quoted is explicitly saying that for-profit software isn't a charitable act. Instead, in an implicit comparison to charitable acts, it's saying that for-profit software can also make life better for users.
Self-interest isn't incompatible with passion and personal moral indignation. You're essentially accusing him, if I read you right, of being disingenuous: claiming he's restoring that data for one set of reasons, when really, he's doing it for another. I don't think it's fair of you to call his integrity into question, and I don't think he's done anything to merit it. I disagree that your reasons, not his, are "all there [are] to say."
Here’s a comparison. You walk into a used instrument store looking for an eight string bass. Another fellow walks in at the same time looking to sell the bass. You wink at each other and meet outside, buy that bass, and cut out the middleman.
It’s his bass and your money, but the reason you knew he had a bass to sell was the store. Likewise, it’s “your” listing data, but the reason padmapper is able to hire someone in the third world to read the internet and manually screen scrape the data—or whatever it is they’re going to do—is because craigslist aggregated it for them.
This is very similar to the kerfuffle over allegations that Bing was scraping results from Google rather than organically indexing the web. It’s quite possibly legal, but it’s not particularly admirable.
And I join Phil (disclosure: He once called me a “giant,” so I owe him) in suggesting that arguments that the ends justify the means are suspect.
I’d prefer to see the padmapper folks put up a pirate flag and openly declare war on craigslist. Just come right out and call them the evil empire already.
> the reason padmapper is able to hire someone in the third world to read the internet and manually screen scrape the data—or whatever it is they’re going to do—is because craigslist aggregated it for them.
Here's where the rubber meets the road. As far as I can tell, 3taps is taking data out of the Google Cache and re-aggregating it. While stealing someone's aggregation of data is clearly wrong, re-aggregating data from public sources clearly isn't, both from a legal and moral sense, at least in the US.
The material facts in each ad are public data. The fact that such an ad appeared on Craigslist with such facts is also public data.
> And I join Phil...in suggesting that arguments that the ends justify the means are suspect.
I'm also suspicious of ends justifying the means, but that isn't the whole story here.
> I’d prefer to see the padmapper folks put up a pirate flag and openly declare war on Craigslist. Just come right out and call them the evil empire already.
To me, Craigslist is the one being selfish and shortsighted here. If they back legislation seeking ownership rights over re-aggregation of data, that would be an "evil empire" move. It would be better for everyone if they subvert Padmapper instead of fighting it.
from a moral standpoint, how does scraping the google cache of craigslist differ from scraping craigslist directly? it's the same information and the google cache is clearly filling in some "infrastructure of the interwebs" role here.
Yes: the copy of your listing that resides on Craigslist's servers is theirs.
You are free to post the same listing to as many other sites as you want; Craigslist doesn't demand exclusivity. In that regard, your listing remains yours. You have no other claim to the posting on Craiglist itself.
But he said this doesn't involve touching Craiglist's servers at all. I admit I am drawing a blank as to how this would work, but the (seemingly impossible) description he offers seems to get around that objection.
By your logic, they've "gotten around the objection".
Not so fast. The full objection that people have with spammers isn't that the spammers have their email address. It's exactly how the email is used which comprises what people object to.
I've been looking at Padmapper, and as far as I can tell, they are basically providing a better interface to Craigslist largely by aggregating data that Craigslist doesn't claim to own.
Specifically how is Padmapper competing with Craigslist? How would such a mechanism not also apply to Google News, reddit, HN and various news sites? Is a "search bar" extension in Firefox competing with Google?
Yes, but this can't reasonably include the material facts concerning the listing: how many bedrooms, the rental amount, the street and cross street. The data I just mentioned and the relationship between them is not something that Craigslist could reasonably own even if they did ask for exclusivity.
Ignore this guy, you're doing the right thing, and I thank you.
e: To expand, CL obviously picked on you specifically, and it deprives all the apartment hunters of your service when applied to the largest apartment listing available. I could see it if they somehow filled the void with a comparable service, before they turned it off, but by depriving padmapper access, it makes it worse for everyone.
Be careful reading these comments. There's an overwhelming bias on HN in favor of "rules that impede startups are there to be broken", and, of course, 3/4 of the site is going to cheer you on. Remember when Mark Pincus said early stage startups need (as I remember) to lie, cheat, and steal to get off the ground? HN cheered that on too.
Just remember that every nerd that has ever broken any rule --- from seeding a torrent of a movie to scraping a competing site to (yes, go look it up) exploit SQL injection to dump a company's mail spool to the Internet --- has come up with a personally compelling rationale to do it. MAFIAA, "we're helping apartment hunters", "STRATFOR was an evil intelligence company". These are all of a kind.
The fact is, you compete with Craigslist. However much "better" your service is, customers don't agree: they overwhelmingly prefer Craigslist, because that's where the market is: the "killer feature" in your space is the market.
So, to stay viable in the face of competition that has you beaten dead to rights, you've rationalized cheating them.
I don't find what you're doing detestable, or anything like that, but it sure is annoying to watch you pat yourself on the back for it. I'm with Phillip. Have you read a lot of his comments? I have. He's not dumb, nor is he a troll.
http://www.craigslist.org/about/jim_buckmaster
Possibly the only CEO ever described as anti-
establishment, a communist, and a socialistic anarchist,
since 2000 Jim has led craigslist to be the most used
classifieds in any medium, and one of world's most popular
websites, while maintaining its public service mission,
non-corporate vibe, and staff of 30-some.
"Non-corporate vibe", like sending out cease and desists. Buckmaster wants to abolish private property. He shouldn't be against the reposting of public property, then.
You're right, it does come off as arrogant to say that it's that much more efficient, and that that justifies making this decision. I think that the potential for doing good plays into any moral decision, though, so I don't think it's out of place in trying to explain my rationale. My intention definitely isn't to pat myself on the back here.
It is more efficient and that does justify the decision. Consequentialist ethics runs on quantitative considerations rather than right-of-way, so "it's more efficient" isn't an absolute right-of-way that would mean that you ought to do anything efficient regardless of the consequences - a lot of deontologists have trouble comprehending this - and if Craigslist were acting differently, or Padmapper, "efficiency" might not be enough to make it right. If Craigslist had their own version of the service, or they were about to roll one out, then efficiency wouldn't make it right. In this case, though, I'm willing to say that Craigslist doesn't have a moral right to prevent there from being a better UI to the renter's market, which is a key system resource even if they happened to create it. How would you feel if there were a private company that had invented DNS, taken on the task of gathering DNS data from many submissions, done it ad-free for a few years, and then they started deleting sites they didn't like from the Internet and suing anyone who tried to scrape their data and add the missing sites back in?
Okay, and how would you feel if there were a company that claimed the DNS system was an insecure mess (or maybe one with centralized power at ICANN, a single US organization, and thereby had US-centric agendas), developed a replacement for DNS, initially passed through to the underlying DNS network most queries but added additional features on top, eventually got to the position where most DNS queries were going through them (with lockin to the specific extra features they added), and then played the same game you are bothered with (which, I feel the need to point out, isn't even what is going on here: CL is not "deleting posts they don't like" with PadMapper "adding the missing sites back in"... attempting to draw a parallel to censorship in your argument is downright disingenuous rhetoric)?
There is no reason to believe that PadMapper would not easily end up in the same position going forward, and in fact the incentives are sufficiently poor that it seems like a bad assumption to believe that it would not happen: it, at least, has to be considered. To the extent to which you might argue "the conclusion wouldn't happen with PadMapper", it seems even easier to argue that "the conclusion didn't even happen currently with Craigslist" (again, read my earlier parenthetical).
I bring this up to point out that there is an underlying assumption in your argument that presumes the result: that you already don't like Craigslist, and that that is what causes you to like other people over Craigslist. The DNS argument can be used in either direction: the only pull it has is an appeal to things the listener already might like or dislike (in this case, liking the idea of a theoretical open DNS system, and disliking the idea of hypothetical censorship).
Just remember that every nerd that has ever broken any rule --- from seeding a torrent of a movie to scraping a competing site to (yes, go look it up) exploit SQL injection to dump a company's mail spool to the Internet --- has come up with a personally compelling rationale to do it
"It is difficult to get a man to understand something, when his salary depends upon his not understanding it." -- Upton Sinclair (http://en.wikiquote.org/wiki/Upton_Sinclair)
Nah, it's not morally gray. You guys are a search engine. Craigslist may not like what you're doing, but that doesn't make it wrong. Traditional publishers hate Google, despite all the traffic it sends them. So what?
Getting a C+D from Craigslist is a good sign. It means you matter. Hang in there long enough, and Craigslist won't be able to cut you off, because a good chunk of their traffic will come through you.
It's kind of weird to think of Craigslist as stodgy old-media using the courts to defend an out-dated business model because they don't like being disrupted by new technology, but here we are.
Craigslist has every right to do what they want, regardless of whether or not you think its the right or wrong choice. If they choose to spit out C&Ds without self-improvement, then they knowingly run the risk of long-term obsolescence.
Sure. I'm not trying to give advice to Craigslist. It may well be that legal threats are their best strategy. They certainly have the right to sue anyone they want.
I'm just saying that ericd needn't feel guilty for ignoring Craig's wishes. Do you think Craig feels guilty about destroying the newspaper industry?
> Hang in there long enough, and Craigslist won't be able to cut you off, because a good chunk of their traffic will come through you.
If Craiglist was motivated that way, why would they have sent the C+D in the first place? It seems that CL is opposed to anyone violating their TOS, period, regardless of benefit to CL.
Craigslist dominates apartment rental advertising. Landlords post to CL because that's where the renters are looking, and renters look on CL because that's where landlords post apartments. But CL does have competitors. Lots of sites want a piece of that business, they just can't overcome the network effect that protects Craigslist.
Right now, the overwhelming majority of pads on Padmapper are Craigslist posts. But that's just because the overwhelming majority of all advertised apartments are Craigslist posts. If people start using Padmapper in large numbers, it breaks the symmetry of CL's dominance. Landlords are still posting to CL, but renters are finding those posts through Padmapper. It won't be long before landlords realize that those renters will also find posts on Kijiji or Apartments.com or whatever. When that happen's CL is in trouble, because it's revenue comes from landlords in certain cities paying to post.
CL doesn't care about the TOS per se; the TOS are designed to prevent just such a scenario.
Just thinking "out loud" here, it seems CL would need to stop others from using their data derived from Google or other search engines. If they can legally do that at relatively low cost, they would. Alternatively, if I was Craig, I'd have to seriously consider disallowing well-behaved search bots from scanning CL data. As a moderate user of CL I've never found a CL item through Google; after all, ads are good for only a week. I don't see a big loss to CL by cutting off Google/Bing etc.
I've used padmapper and its definitely teh number one choice when apartment hunting. Its not even close.
Rant/
Just keep going man. If you stop doing what you do you will lose the millions(?) of people you are helping.
Seriously other apartment sites just don't get it. They SUCK ... suck so bad that I'd rather bang my head on the table than use them.
Without PadMapper, craigslist is worse than the other sites. I don't understand why they just don't buy you outright - thats what should happen if anyone cares about the effing consumer.
It depends what your intent is. If you're on the consumers' side, people's side, wanting to make their lives easier, better, through efficiency and therefore increased productivity - then being integrated into everything will of course cause this. Best case scenario would be some form of partnership, however Craiglist seems to want to create and control an ecosystem instead of being the management piece, instead of finding a way to survive as a management piece.
It actually doesn't align, though. I'm not going to go into detail, but my expected finances are probably higher without doing this. That was an argument against doing this.
Color me skeptical. Padmapper is a for profit business, and Craiglist ads make up a huge fraction of all online housing listings. In the absence of any explanation at all, I hope it's understandable why I wouldn't just take your word on it.
In any case, I agree with others that you don't need to bring these kind of justifications to bear. Craigslist has no right (either legally or morally) to facts about rental locations.
As someone that knows Eric personally, I can confirm that he'll make less money by showing craigslist listings. However, I'll leave the details up to him, though I understand why without them, you're skeptical.
PadMapper is actually a pretty unusual "business" since I'm the only person who works on it full time and it grew out of a hobby project, and still isn't really run like a business. But yeah, that's generally a reasonable assumption.
I don't have to, but I thought it'd help to explain my reasoning to people.
Eric, if you are genuinely doing this for the good of the many and not for yourself, then you would use an alternative that is neither "somewhat dickish" (you're admission, not my judgement) nor somewhat cowardly (You are really just hiding behind 3Taps lawyers) if such an alternative were available. Would you not?
Eric, it's not even close to a grey area. Like...not even remotely close to a grey area.
Consider the following:
I leave a bowl of candy on my doorstep for Halloween with a note that says "Please take one piece of candy.".
You, however, have a bunch of friends back home who aren't at my house to get the candy, so you take the entire bowl. You defend this, citing a "grey area" because your friends want the candy, and you're making them happy because you took it from me and gave it to them.
That's what you did.
No grey area. You just wanted the candy.
(Do I support craigslist's decision? No, I think it's stupid, but it's their stupid decision to make.)
As somebody currently looking for an apartment, I wholeheartedly agree with his claim.
It's fine if craigslist wants me to use their competing mapping service, I would even pay them a small fee for it, but if they don't have one yet then I just get fucked. Craigslist isn't being "nice" to me, so as long as padmapper isn't doing anything illegal, I really don't care if they're being "nice" to Craigslist.
By CL's Terms of Use, it's not CL's data. They just post it.
It's clearly a gray area, but let's be clear: IT'S NOT CL'S DATA. Craigslist actually makes it clear that it's the user's data and that when it's posted on CL, it's being posted publicly.
It's data that a CL user has entrusted to CL. The user could've gone with PadMapper but they didn't. Being posted publicly is irrelevant, since the user agreed to CL's TOS not PadMapper's.
The direction PadMapper should be going in is to forget about the CL data and do things that make it number one over CL. Stealing CL's clients is fair game and in the long run aren't one CL TOS change from being in the wrong again.
Let's be honest here. User's don't post to CL rather than Padmapper because of their TOS.
1. The average user has never read the TOS and doesn't care about it.
2. User's don't post to padmapper because A) they don't know about it and B) posting to craigslist is (until now) a superset of posting to padmapper and craigslist
3. The data Padmapper is taking is your address and posting time. You'd be hard pressed to find more than a handful of people who mind Padmapper reposting such info.
Besides, it's incredibly difficult to steal CL data for apartments. The posters are incredibly technology unsophisticated; I'm amazed how apartment owners can't take a simple 2 minute video walk-through. A lot of places don't even have pictures!
This does not seem like an important distinction. The user implicitly authorized it to be posted on CL by virtue of, well, posting it on CL. They did not authorize any other sites to post it. Or am I missing something?
I don't think PadMapper puts up the entire post written by the user, just fact that the post exists and is in reference to a particular location. Likewise, it's kosher for me to tell Alice that Bob posted a poem on his window so that she can go read it, even though it would not be kosher for me to make her a copy.
> If Craigslist is explicitly disclaiming copyright in the data they post, how can they sue for copyright infringement?
By having the user sign their right to sue over to Craigslist. They're saying "it's your copyright, but you're giving us the enforcement rights to it". It's in their TOS:
> You also expressly grant and assign to CL all rights and causes of action to prohibit and enforce against any unauthorized copying, performance, display, distribution, use or exploitation of, or creation of derivative works from, any content that you post (including but not limited to any unauthorized downloading, extraction, harvesting, collection or aggregation of content that you post).
I don't think that's possible, but I'm not a lawyer. I'm very interested in seeing an example of a clause like this being applied. Another commenter cited the Righthaven case as a counterexample: https://www.eff.org/press/releases/righthaven-case-ends-vict...
I'm copying and pasting from a reply I had below, but to respond: "Although I think this is a bad place to stake your claim, the posts themselves are arguably purely public data the second they hit Craigslist. The data conveyed is factual, there is no IP, and it's being disseminated publicly. It's similar to (and yes, there are a lot of ways to poke holes in this analogy, but for the sake of argument...) a town crier shouting an advertisement out in a crowded square and one of the listeners taking that information, traveling to another public square, and repeating it."
One of the wonderful things about HN is that it provides a forum for civil discussion. If that's your style, there are lots of other places on the web that are better venues for you.
I hat "unknown/expired link" - It wastes my time and frustrates me when I just leave the page up for a time then click more and get that error. I also can't stand the URLs not being human readable.
I can dig it, but you know that CL is extremely fast, right? Even in the oft-lamented interface, it's easy to go to the right place. Everybody can figure out Craigslist.
Outside Developers like/want the CL customer case, their reputation and their page views and see them as a quick, easy path to success. However, you are not entitled to them. Understanding this and moving on are key to doing it yourself and blazing your own path. Attempting to work around access limitations and usage restrictions and then attempting to justify your actions is only going to end badly.
> Outside Developers like/want the CL customer base, their reputation and their page views and see them as a quick, easy path to success. However, you are not entitled to them.
As far as I can see, Padmapper is not using the Craigslist reputation or page views. The data Padmapper is aggregating is treated as public information by Craigslist's own policy. Given these facts, I don't see what he's doing that's wrong, specifically.
It's not public information. The listing data belongs to the original poster who entered into a contractual relationship with CL -- not with PadMapper. If anything the ads are covered by copyright, held by the person who wrote the specific ad. If the service is so great, then start calling the ad posters and offer them the chance to post on PadMapper. Otherwise, it's theft. We can moralize all we want but reproducing someone's ad without their permission is a violation of the poster's rights, even if some of us feel that it's in their best interest. The choice should belong to the person posting the ad.
As far as I can tell, it is public information. Craigslist has a claim on the listing and not the information in it.
> The listing data belongs to the original poster who entered into a contractual relationship with CL -- not with PadMapper.
If the data belongs to the original poster and not Craigslist, then the original poster can have a beef with Padmapper, and Craigslist has no claim.
> If anything the ads are covered by copyright, held by the person who wrote the specific ad.
In as far as Padmapper is using the data in the listing, and not the actual listing, I don't see any violation of copyright. Otherwise, one encyclopedia could sue another one for also publishing the same material facts in an article. If your logic held, there could only be one encyclopedia in the world.
> If the service is so great, then start calling the ad posters and offer them the chance to post on PadMapper. Otherwise, it's theft. We can moralize all we want but reproducing someone's ad without their permission is a violation of the poster's rights, even if some of us feel that it's in their best interest. The choice should belong to the person posting the ad.
I don't think you're taking all the implications into account. By this reasoning, Google bots should only put something in the Google Cache if the original poster gives specific permission.
I agree that reproducing someone's ad without their permission is wrong. However, reproducing the material facts of their ad is clearly not immoral or illegal.
Also, if the first person to aggregate public data effectively has a copyright to that data, there would be widespread and harmful implications.
>then the original poster can have a beef with Padmapper, and Craigslist has no claim.
So what you're saying is that, "The original poster can sue PadMapper, but at least CL isn't going to be a plaintiff."
Unfortunately, you don't know what you're talking about:
From the CL TOU: "You also expressly grant and assign to CL all rights and causes of action to prohibit and enforce against any unauthorized copying, performance, display, distribution, use or exploitation of, or creation of derivative works from, any content that you post (including but not limited to any unauthorized downloading, extraction, harvesting, collection or aggregation of content that you post)."
That clause just made CL the agent for the original poster and it expressly grants CL the right to prohibit unauthorized use. RTFM.
This whole situation is excessively asshole-ish. It seems like PadMapper is hell-bent on scraping CL data by any means (or rationalization) necessary. We can debate about CL running their business the way they do, but this argument is starting to look like some pseudo-intellectual freshman philosophy conversation. It's almost like debating Lyndon LaRouche supporters.
What PadMapper really needs to do is consult a lawyer and figure out how to work within the law rather than trying to be cute about skirting the CL TOU. Any investor would run far away from a company that seems so determined to do something without any professional legal advice instead replying upon some sophomoric interpretation of a contract and the law.
This comment: http://news.ycombinator.com/item?id=4160061 outlines a strategy that could work that doesn't involve playing games and skirting Craigslist. It's also a way to actually build a business based on something besides just stealing other people's data. Of course it requires real work and doesn't appeal to someone that thinks code is the answer to every question.
Data posted to craigslist is subject to the CL Terms of Use. If the data comes from CL, it can't be used without CL's permission. There's no gray area.
if data.from_craigslist?
cant_use
else
use
We can moralize and cry all we want, but their Terms of Use are pretty damned clear and the law is pretty damned clear as well.
Returning to the Google example, it's irrelevant, "Google does it" is not a legal defense or even a moral defense, depending on ones opinion of Google..
They're using the Craigslist reputation and page views indirectly. The information is only on Craigslist because people placed it there, and what was their reason for choosing Craigslist in the first place? Reputation and page views.
By that same argument, Google News is using the reputation and page views of The New York Times and The Wall Street Journal. Google as a whole is using the reputation and page view of the entire web.
As someone who knows Eric personally, I can vouch that he's not an asshole. In fact, that is literally how he thinks and those are his real motivations.
He's not the type to come out and call someone else, like Craigslist's CEO, an asshole. In fact, he comes out saying that he admires Craigslist. That's just the way he is.
> step over the moral grey lines of using someone else's data without their consent by claiming my service is better
OK, IANAL, but isn't the content of craiglist's post owned by the author of said post, not the Craiglist itself? (sorry haven't read their TOS/TOC). Just like you own your tweets, not the twitter?
Otherwise, I could keep posting copyrighted material (like books) and then suing Craiglist for copyright infringement (something Vivendi did to Youtube) since CL would claim responsibility/IP rights. I am sure that CL protect itself from acts like that pushing the blame/responsibility at users (I don't think DMCA would work in each and single case). So one could argue that this is each and every CL author right/responsibility to file copyright infringement claim with whomever harvest CL posts, and CL should, for lack of better words, "mind their own business". In other words, they lack copyrights over the content their users are creating. Sure they can remove the post if it violates TOS, but in the best scenario they could email the author "hey we stumbled upon your CL posting on XYZ website, do you want to file a claim?", but really nothing more than that, right?
I know it sound bit weird, but isnt't it what AirBNB and others do: while your local/state law may forbid subletting, AirBNB hands are clean because they are in business of providing a social platform to find and post vacant apartments, not enforce or monitor if their users follow their local renting laws.
Craigslist is being the RIAA/MPAA in this situation. Users (buyers/sellers) want to remix the same content in a different form, and the RIAA/MPAA/Craigslist won't let them. So then it comes down to whether this can be technologically stopped or not.
The thing about morality is that it's much more subjective than law. If someone does something that is legal (or is presumed to be legal), on what authority do you claim their actions to be "morally grey?"
>But then I did some back of the envelope estimates of how much of people’s time and effort it would waste if I didn’t, and it became clear how much less nice it is to waste the time of millions of apartment hunters out of stubbornness or some clearly inaccurate assumption about the will of the community.
Christ, what an asshole.
Translation: "We found a loophole that lets us get around the spirit of what was communicated to us so we could continue to build out our product. I'm going to conveniently step over the moral grey lines of using someone else's data without their consent by claiming my service is better."