The thing I found most interesting about this interview is just how badly Peter Norvig has failed to understand what Facebook is all about - I don't use Facebook so that friends can give me recommendations about what camera to buy - I use Facebook because knowing what my friend Sally got up to this weekend is far more interesting to me than what Britney Spears has been up to. I might be interesting in knowing what camera she used to take that brilliant photo of Jim under the fireworks, but that's about as close as it gets.
This has a huge implication for the value of Facebook - they know who I am, really. They know where I live, my age, my sex, and they can even tell what sort of activities I like. Google tries to guess this information from my page view habits, but they often get it wrong, big time. Here's an example - if I get a dating site ad on Facebook it correctly identifies me as a woman and throws up pictures of guys. If that ad is being served up by Google, I get identified as a man because of all the time I spend on tech sites, so the same ad has pictures of women in it. Facebook gives me ads about clothes, Google almost never does.
I suspect that Google's core business is at significant risk here. People are spending more and more time in the Facebook universe, those 'Like' buttons have popped up all over the Web, and that spells trouble for Google. It'll be interesting to see how they respond, but I for one won't be buying any more stock in Google for a while.
I think Norvig looks at Facebook as a lost opportunity for data mining: Facebook isn't used as a recommendation engine any more than Google is used as an ad company, yet all user actions always gets piped back to ad results. If John is friends with Sally (who is kind of cute, and kind of a photography geek, and recently bought a good midrange point-and-shoot), John subconsciously wants that brand to show up somewhere in the recommendations, even if it is too expensive. But Google doesn't know that John's friends have bought that camera, because who posts what camera they bought on their Google profile? If Facebook has the ad instead of Google, then Facebook might get the all-important click.
That is right. Norvig doesn't see what Facebook is good at and ask how it will make money. He turns the question on its head, looks to a successful business model and asks how Facebook could be used for it. And it is an uphill battle for Facebook to monetize in this fashion.
I think what we've seen so far is that Facebook is about creating social spaces, not solving problems, so it really can't be used that effectively as a recommendation engine. Facebook doesn't have a great way to seek rent on these social spaces because they've built their user base on free service. So where is the money going to come from? Probably selling user data to corporations.
Facebook knows what you told them about yourself. Google observe what you actually doing, and if you often visit tech sites then they should show you tech related ads instead of clothes and this is spot-on. The dating ad with women is just noise in their system (after all, if you are frequent in tech sites, you are statistically male... :)).
That's been my experience. Google's ads (in search and Gmail) have been consistently better-targeted to me than Facebook's ads. The only ads I get on Facebook are dating ads (and insulting ones at that).
Full Disclosure: I'm not running an ad-blocker, but I am running a hosts file, so I still see Google and Facebook ads, but maybe not "all of them".
Googlers are fine with admitting the small errors. I had a bug, this design isn't so great, the evals show this change to be a big loss. Those are the easy ones to admit. The big errors -- this product sucks, this idea is bad, eighty percent of the support personell on this piece of infrastructure are uncooperative jerks -- come less easily.
Anyway, eng doesn't really run Google any more. PM does. Eng can still push back to some extent and feels important when they control huge budgets for new datacenters. But who designs, specifies and ultimately controls the products for which those datacenters are being allocated? Most everything you see, good or bad, that happens on Google these days came from a PM.
What justifies a product management role separate from the developers? A PM that is also a developer understands better what is the cost of all these features.
Maybe a separate PM would see the forest better as he is not looking at specific trees, but there are a lot of people who can do both well.
Don't know about Google. I'm working for a defense contractor, so we may have more hoops and red tape to blunder through... but my PMs are always busy talking to the customer, writing documentation, figuring out what the specs actually mean, managing, or something. There's no time left in the day to hunker down with the code (or digital logic or whatever engineering discipline the PM came from).
Perhaps it depends on the industry, but in health IT I've yet to meet anyone who could perform both roles well. Developers mostly only make good PMs when the product is intended for either general consumers, or other developers.
Funny you ask, going through the middle of an organization where ProjM is turned into ProdM right now. As far as I can tell very little changes except the titles on business cards ;-)
The distinction I usually make is that a "project manager" is responsible for scheduling: maintaining a picture of the current state of the project, the current best estimates for when things will happen next, and make sure that everything that must happen before the next release is accounted for somewhere. It's not a "management" role in the traditional sense (people management), but it's not an engineering role, either.
A product manager's job is to understand why people are (or are not) buying your product. They need to understand the market, keep track of competing products, and be able to make decisions (or offer advice) about which of a list of potential new features are most likely to make the customer's mouth water. It's fundamentally a marketing role, but, especially with a complicated product, can require a fair amount of technical skill to do.
Disclaimer: This is all Big Company language, but not necessarily identical to Google's language.
Product Manager sounds better. It makes it sound more like you're responsible for shipping something that generates revenue, rather than overseeing an org-chart rework or some other corporate circlejerk.
And, like you say, it does create a very different attitude toward error. If you're a politician, admitting you're wrong is a weakness, but if you're an engineer, you essentially want to be wrong half the time. If you do experiments and you're always right, then you aren't getting enough information out of those experiments. You want your experiment to be like the flip of a coin: You have no idea if it is going to come up heads or tails. You want to not know what the results are going to be.
That's idealism talking, but practically speaking experimenters are invested in the outcomes of all but the least interesting experiments. Nobody wants the search feature they spent a month implementing to come back with negative results on evaluation. This is not only true at Google.
Some of the stuff in this sounds nice but I feel it lacks some honesty. "What is the thing you have been the most wrong about?" And you say that the worst error in judgment you've made is an off-hand remark in a meeting? Let's be serious here.
His call on big mistakes from Google seems good but I doubt Google could have executed on it even if they had known how big Facebook would be beforehand. Big G is not a product company and is best at making products that minimize the time you have to spend using them.
"Some of the stuff in this sounds nice but I feel it lacks some honesty. "What is the thing you have been the most wrong about?" And you say that the worst error in judgment you've made is an off-hand remark in a meeting?"
Huh? That isn't how I read it at all.
From the article:
"Question: What about you yourself—what have you personally been most wrong about?
Answer: One thing is how fast things change.[Stuff about meeting here]You think you have this experience—and we talked about how important experience is for having intuitions—but experience can go out of date very quickly."
To me he seems to be saying that one of his major mistakes is underestimating how fast things change, and underestimating the erosion of the value of his experience in guiding his intuition (He just finished talking about the importance of intuition for situations where statistics and data can't help/don't exist.)
Given that he was speaking about a professional context [1] and the position he occupies at Google (Director of Research), where anticipating change would be presumably part of his job description, that sounded to me like admission of a major lapse, somewhat different from "My biggest mistake was an off hand remark I made at a meeting", as you seemed to imply.
He was using the meeting incident as a concrete example of a rather large failing. How is it "dishonest"?
[1] The whole interview is about errors in a professional context , about Google and his work there, so I wouldn't expect to hear about whether he got stoned as a teenager and got thrown in jail! ;-)
I think it was more of a regret that his intuition about the problem area gets invalidated so quickly. The top technical guys are supposed to have good intuitions about what works and what doesn't; this is a major part of the value they can add. So I take this as a very serious self-criticism.
"is best at making products that minimize the time you have to spend using them"
An excellent observation - I use Google a lot the total amount of time I spend on their sites really is pretty small and the "brand loyalty" for me is almost non existent.
This reminded me of information theory. The idea of entropy in information - that if you can predict the next 0 or 1 with better than 50% certainty, you can compress that information (or in this case, you could have designed a better sequence of experiments).
I do think it's worth differentiating between types of failure, however. You don't want 50% of your experiments failing in the sense of failing to give you an answer in either direction (i.e. being inconclusive).
And this leaves aside the topic (covered in the article) of the more serious kinds of failure - of live code etc. You probably don't want that happening 50% of the time.
I had a boss who always pushed me away from low-probability experiments. Whenever I suggested trying something that was unlikely to work, he probed to see if the single low-probability experiment could be restructured as a series of 50/50 experiments. It turned a lot of failures into partial successes that could be applied to future problems.
That's precisely where Norvig is coming from. Given their infrastructure, it's entirely possible that some portion of their A/B testing is run this way (i.e., test thresholds chosen to maximize information gain).
It depends on whether you're a production engineer making things for use (ICs, software, bridges) or a development engineer trying to push the limits of what can be done.
This has a huge implication for the value of Facebook - they know who I am, really. They know where I live, my age, my sex, and they can even tell what sort of activities I like. Google tries to guess this information from my page view habits, but they often get it wrong, big time. Here's an example - if I get a dating site ad on Facebook it correctly identifies me as a woman and throws up pictures of guys. If that ad is being served up by Google, I get identified as a man because of all the time I spend on tech sites, so the same ad has pictures of women in it. Facebook gives me ads about clothes, Google almost never does. I suspect that Google's core business is at significant risk here. People are spending more and more time in the Facebook universe, those 'Like' buttons have popped up all over the Web, and that spells trouble for Google. It'll be interesting to see how they respond, but I for one won't be buying any more stock in Google for a while.