'I don't need to think much about retrofitting the data for all instances of that model. I just add an attribute where its needed for the new use case, ensure I have basic checking in my ruby model object and my system keeps incrementally improving.'
That's exactly the same as adding a new column to your DB with NULL as the default value.
Mongo has the notion of undefined and Null. You can just start putting the new field on new records without having to backfill. Also, you don't have to do the migration thing, which can get messy in big teams (from my experience).
Moving to a doc store from an RDMS really does bring with it an odd sense of freedom when it comes to the schema.
You don't need to do any 'migration thing', you just add the column to the DB and choose a sensible default value? I don't see what you gain by having both 'undefined' and 'null'.
The 'odd sense of freedom' is not always a good thing either. It's like BASIC allowing you to use a new variable without declaring it. It may be convenient but nobody calls it a good idea.
"You don't need to do any 'migration thing', you just add the column to the DB and choose a sensible default value?"
Taking the team I worked with at the BBC as an example:
1) There were staging, integration and production environments. Staging and integration would often not be aligned with production, or even one another, because we might find a bit of code turned out not to be production suitable/needed. If this happened we would have to drop the database back to a known, good state. You can't have columns with constraints left around when the code which might have satisfied those constraints is reverted. Doing it without migrations would have been idiotic to say the least.
2) Developers work on different features in different branches, often collaborating. Different features apply new attributes to the db schema. It's important for a developer to know his DB is in the correct state when he starts hacking. You do that with migrations.
Because you almost completely remove the need for schema definition (and what little of it you do, you can do in app. code) you simply don't need the migrations any more. Using mongo means you can pretty much just export your applications domain without having to coerce it into the relational model.
"I don't see what you gain by having both 'undefined' and 'null'."
They mean totally different things. Undefined means that the field has never been explicitly set, null means the field has been set. This means you know what's been backfilled and what hasn't - you can't tell without extra metadata in mysql. Also, in mysql if you provide a null default then every row has to be updated.
"It's like BASIC allowing you to use a new variable without declaring it. It may be convenient but nobody calls it a good idea."
I don't know BASIC but you can put Perl into a certain configuration that allows this. That makes for horrible scoping issues that aren't analogous or applicable to what we're talking about.
If you added a column and you want to revert back, you just drop the column again! What's so hard about that? No 'migration' needed.
In BASIC you can 'declare' a variable by simply using it. The compiler will not warn you if you use an undeclared variable. That's the analogous situation here.
No doubt in part because they benefit from the protection afforded by nation-states. Cruise ships tend to be filled with the citizens of powerful countries (or countries allied to powerful ones).
Bing had already said they would pay for firefox, just as they would for Apple. It is disappointing to see all the doomsayers here, you would have thought this was an obvious point.
Great point. The 10s of GB was anecdotal data from a CarrierIQ recruiter.
If we go by the official letter you've posted and assume conservatively CarrierIQ has only 1 Petabyte of data, and that they've been collecting since 2006 (when they received their Series A), they've been collecting 456 GBs of data per day. Its probably more than that today since the data collection rate has surely accelerated over time.
That's an order of magnitude beyond 10s of GB per day.
Well realistically they would not have been collecting 456GB per day from day 1. They would have grown from almost nothing, which means that today they're probably collecting around a terabyte per day. That's two orders of magnitude, 7KB per user per day - and could definitely include all keystrokes of all users.
