- Obtain enough computing capacity
- Stay within the electrical power limits of our data centers
Somehow it feels like the world is so backwards. Here you have all these smart hard working people with billions of dollars of resources at their disposal and they made a load of kooky videos that people could 'like'.
The videos I'm sure are cool, and I realise that it was quite an achievement, but still, our priorities as a Hunan race are pretty much fucked up.
Food for everyone, diversify our habitable planets portfolio, higher education en masse, stop killing each other, prevention of diseases, keeping up with the Kardashians, a few others as well...
Engineers figure out ways to make this sort of thing cheaper, easy to do, and scalable. Most of all, the end result is something tangible, interesting to individuals, and a product of a lot of data that the engineers already had laying around.
Then that code and how it was created circulates to other orgs (hopefully through open source), then it's picked up by researchers and grad students who start to apply it to more important research. Maybe also privately funded or state-sponsored aid types read about how it was done and they apply those principles to the high-minded projects that you want to see.
Blaming society for having "fucked up" priorities is a very college freshman view of the world. There is no way that even the most talented engineers would also have the wherewithal, experience, connections, and local knowledge to produce sustainable projects that would, I don't know what you were imagining, lower poverty or increase literacy rates.
The people from Statigram[1] came up with this idea last year. I have always been interested in how they did it. It wasn't the same scale as Facebook, but I assume it was a big challenge for them, especially not having the resources.
The feature consisted in generating a 15 second video with your most liked Instagram photos and sending it to you as and MPEG email attachments. They had to do this because third party apps cannot post to Instagram.
I initiated my video generation probably at their peak, and still, in less than five minutes, the generated video was in my inbox. If someone from Statigram is reading this, I would really love to hear your side of the story, being able to compare it to facebook's approach now will make it even more interesting to me.
Really annoyingly, Facebook's selection of photos was way off for me. I've been married over 3 years, with my wife for over 4.5 years, yet it still chose to include photos of ex girlfriends. Facebook has all the data it needs to discern that was a bad idea. Quite an oversight in my opinion as I know several people who it affected too so they didn't publish the video.
Ex-girlfriends are part of your past whether you like it or not. I can speak with most of my ex-girlfriends, and it was a nice trip down memory-lane to watch the pictures of me and them.
I am not running from my past, and I appreciate the time that we had together.
The point is that Facebook should be "smarter" enough to account for the majority of people who don't ~need~ pictures of their ex-girlfriends brought up post-marriage.
Your pompous and aloof reply doesn't really apply here. I'm good friends with my ex, invited her to my wedding, and occasionally meet down the pub. What I'm sure my wife, nor my ex's significant other, would've been happy with was cosy pictures of the two of us which I've failed to prune from Facebook. Whatever floats your boat I suppose, each to our own.
Good point, when we ask pictures not to show our ex, we're asking for a history rewrite, as if they never existed. It may be hard to face them but we may have to.
I assumed the videos were created lazily, on first view. This article makes it pretty clear that they decided from the beginning that they needed to render all the videos in advance. I wonder how many videos they rendered which were never viewed, and whether rendering on first demand would have been viable.
Probably to ensure compatibility for old browsers and mobile users. Obviously the more elegant solution is to do the animation client side and avoid rendering millions of videos but it's not the solution that would have worked for the most people.
Considering how facebook has more than a billion users from all the corners of the world accessing it from literally every internet connected device, they had to make something that would work everywhere. Doing it the video way seems like a good decision in this case, at the scale of facebook.
Is there really the need to use 80 people for this feature? Why it seems to me that is just because of excessive programming hours in total available at Facebook rather than the real need for this new feature. Just curious.
About 3-10 people worked on this full time depending on which day in the 3 week development cycle you looked at. Most of the others helped out in their spare time in one way or another
There are no excessive programming hours at Facebook. I assume most of those people worked on the massive backend challenge, not on making the videos (but that's my guess).
I understand that the difficulty of making this new feature is scaling the backend, but at this point wouldn't Facebook already have sort of some processes or "frameworks" to follow when making a new feature.
An impressive engineering task, for sure. But this shows how far we are from the fundamental ideas of the internet and the web, and instead fall back to the good ol' feudalism.
In France we had that thing called Minitel [0], which was heavily widespread before the WWW took off (I think it was given to everyone or something like that). The economic (and technical) model was completely different than the internet: you had a passive, stupid terminal, and you connected to a server that offers services for a fee. So you paid for connecting and using a service that could technically not be done on your own minitel (at least in the earlier version). Just like a browser that is completely stupid and allows you to connect to Facebook, Twitter, Google et al. for things you can't do on your own machine. You were depending on a third-party to do things.
We speak a lot about the Minitel model and how it is after all these years a model for all the big WWW players and ISPs to monetize their services. But it's not a common reference, so 'feudalism' works well in this context.
> We had anticipated that about 10% of the people who saw their video would share it. Instead, to our surprise, well over 40% of the viewers were sharing.
That 10% estimate seems way too low to me. If something like this catches on, it catches on big time. If they didnt expect it to catch on, then why even bother. That said, I never had a problem with videos loading etc during this look back craze. And hardly ever any other time for that matter - it is remarkable how stable that site is.
It's easy to forget given some of the press they get just how active their users are, 720m users who care enough about your product to watch a video about how they use it must be incredible. The algorithm seemed fairly naive but it certainly seemed to work well for some I saw, and really impressive engineering too.
It unbelievable to think about how much synchronization must have gone into this project for all the pieces to come together successfully. Just some more evidence that goes to show facebook has some of the best engineering teams in the world.
While I understand they want their own server farms, it is interesting that Facebook does not use any cloud compute. This is almost the perfect usage example and pattern for cloud compute.
Facebook has some of the largest data-centers in the world, with plenty of spare capacity. In the article they describe that running these HUGE jobs put them at risk of not potentially not being able to provide power to all the machines at once.
If you're thinking they could have just started up a ec2 job to process a billion videos, I really doubt there's any cloud solution in the world right now that can handle this scale without capacity planning before hand.
The videos I'm sure are cool, and I realise that it was quite an achievement, but still, our priorities as a Hunan race are pretty much fucked up.