Pearl won the Turing Award for his work applying Bayesian analysis to machine learning, among other accomplishments.
For HNers just out of college, it's important to note that Bayesian analysis has not always been as popular as it is now. In fact, even as recently as the 1990s, it was regarded with suspicion by many statisticians, who strongly disliked the idea that prior and posterior distributions are meant to represent subjective states of belief. I was fortunate enough to have a very progressive statistics professor in undergrad in the 1990s, who was interested in Bayesian analysis. It's my understanding that most upper-level statistics and probability coursework avoided doing much Bayesian analysis until around the turn of the millennium. (if you went to university in the 1990s or earlier, was this your experience?).
For those interested in learning more about this topic, an excellent book on the history of the Bayes theorem controversies was recently published by Yale Unviersity Press: _The Theory That Would Not Die_ by Sharon Bertsch McGrayne
To be fair to Judea Pearl, I do see that he wrote an essay entitled, "Bayesianism and causality, or, why I am only half-bayesian". Nonetheless, much of his work appears to involve what we now know as Bayesian analysis. So like many (all?) scientists who make major breakthroughs, Judea Pearl was going against the accepted understanding of probability by pushing Bayesian analysis in the 70s, 80s, and 90s. It's good to see his daring rewarded.
Part of the reason Bayesian analysis was avoided was because, prior to Pearl's work, inference was impractical in non-toy examples. A major contribution of Pearl's work was to make it feasible, by structuring the probability distributions as Bayesian networks that limited the possible dependencies, coupled with the belief-propagation algorithm to do approximate updates.
I don't see Pearl as primarily interested in the Bayesian v. frequentist debate himself, though, but rather in how to efficiently do probabilistic reasoning in non-trivial problems in general, with a heavy tilt towards questions of representing causality. Methodologically his work over the years has used all sorts of things from various camps; for example, he was also an authority in the early 1980s on heuristic search.
I'm trying to persuade people to abandon the terminology of "subjective" versus "objective" and talk instead of "individual" versus "situational" versus "transcendental". If the simple counting argument (that you cannot talk sensibly about three possibilities using only two words) does not convince, worry instead that law and science put the subjective/objective boundary in different places.
The problem is that the headline is wrong and the Turing Award isn't a Nobel Prize. It may be similar in prestige to a Nobel, but it is awarded by different people.
Maybe there is no harm in saving people from having to do a search, but there is harm in being misleading.
I am torn between the urge to move to another news sharing site where people know who Turing is, and the thought that a little diversity can do good. I guess the latter wins, because I come here to get surprised, and learn; it doesn't happen as much when you are around people like yourself.
Basically he brought Bayesian Networks to machine learning. He did a lot in their structure and in particular efficient/approximate inference on Bayesian Networks. He has been contributing brilliant ideas since the early 80s I think.
Search belief propagation, junction tree algo, markov blanket, belief propagation statistical physics. The last search showing his ideas finding use beyond where he introduced them. Whenever ideas keep showing uses in different places especially in something as well evinced as the thermodynamic/information link then you know you have done something profound.
A few years ago I looked up his earliest papers, and it's interesting that his work on Bayesian networks essentially seemed to start as an effort to detect causality among sets of random variables, i.e. given a Bayesian network, which random variables can be said to "cause" others? Of course, Bayesian networks start by assuming you know every random variable's distribution, so verifying independence is easy but finding it is hard. I've always thought an interesting research project would be to try the same approach with correlation functions instead of distributions, and see how far you can get.
Well, as the linked article says, he basically coined the term "Bayesian network" and developed much of the underlying theory (e.g., which kind of independencies it can/cannot capture). This was arguably a crucial step for many areas of modern AI in terms of dealing with uncertainty, cf. the full ACM press release: http://www.acm.org/press-room/news-releases/2012/turing-awar...
He is also credited with first describing the ever-popular belief propagation algorithm for approximate reasoning in Bayesian networks and graphical models in general.
Judea Pearl's work wasn't about machine learning so much as introducing the idea of using probability to AI reasoning. In the early days uncertainty was handled in AI using ad hoc techniques (e.g. Mycin) and the use of probability was regarded as too complex because the joint probability distribution grows exponentially with the size of the domain. Pearl showed that probability could be used for reasoning in a natural way using conditional independence to simplify the joint, and that belief networks behaved in interesting ways (e.g. "explaining away"). He then described an elegant algorithm for the propagation of probbilities in a simple form of a belief net (trees). This still forms the basis of inference today, most algorithms create a clique tree from multiply connected networks and use message passing. Shacter et.al. later showed that all exact algorithms are a form of this method. I'm a big fan as you can tell.
You've probably heard of Hidden Markov Models. They're widely used in many machine learning applications. An HMM is just a simple, easy-to-compute Bayesian inference network.
The idea of Markov chains predates Pearl. His work was a demonstration of the accuracy and power of Bayesian inference networks. He revitalized that idea at a time where programmers were just beginning to have the data and processing power to apply machine learning.
Bayesian Networks tend to be used not as classifiers but a tool to explore joint probability distributions.
Interestingly related to your topic, HMMs and Naive Bayes are related in that HMMs are kinda like the sequence sensitive version. HMMs and Naive Bayes are generative models. They both model/estimate a joint probability on the data with very strong conditional independence assumptions. Where as Logistic regression and Conditional Random Fields estimate the conditional probability of the output/labels directly.
HMMs : Naive Bayes as linear chain CRFs : Logistic Regression. CRFs are state of the art at sequence and time series prediction. I have not yet gotten my head round them though. The relationship between logistic regression and naive bayes is not commonly known (although the comparison of log reg to a simple Neural network is common). Knowing when Logistic regression outperforms Naive Bayes is useful (simple rule of thumb: logistic regression less sensitive to independence assumption, more data use log reg, less data use naive bayes). I've implemented a multi class sparse regularized logistic: SMLR. Its up there with linear SVMs but simpler but also gives a probability.
PG's old spam filter (http://www.paulgraham.com/spam.html) is another example of applied Bayesian inference. Not directly related to Pearl's work, but he basically deserves credit for the wider renaissance in Bayesian techniques.
> Not directly related to Pearl's work, but he basically deserves credit for the wider renaissance in Bayesian techniques.
Post hoc ergo propter hoc? The success of naive Bayes classifiers for spam filtering must have improved popular perceptions of Bayesian techniques, but let us not go overboard.
Congrats to Professor Judea Pearl! Though I did not take any of his classes when I was at UCLA, in chance encounters he gave me a very good impression - nice, intelligent, and unassuming. I ended up using a lot of his materials (Bayesian analysis) after college in my projects... Here's a Tagxedo I made of today's announcement in the shape of UCLA's logo: http://daily.tagxedo.com/march-15-uclas-judea-pearl-named-20...
I'm personally sympathetic with your comments on his politics, but the only part of your comment that is relevant here is "He is an awesome scientist and has made real contributions to the field computer science."
According to Wikipedia he is the father of Daniel Pearl, the journalist who was murdered by al-Qaeda. This may colour his views on things.
I flagged your post, and since you're relatively new here, I figured I should explain why. Simply, no good discussion will happen based on your comment. It will devolve into a political discussion, which while important, is not relevant to the purpose of this site. I specifically flagged your post instead of downvoting it so that the entire discussion about the award and his contributions are not derailed.
In case anyone else didn't pick up on this right away, Judea Pearl is the father of Daniel Pearl, the Wall Street Journal reporter who was kidnapped and decapitated in Pakistan.
The NGO Monitor aims to undermine non-violent actions against the Israeli occupation in order to maintain it. It tries to cut funding to peaceful NGOs because they don't align with Zionist values.
NGO Monitor is pro-occupation and pro-Israeli settlers.
So your response seems to be out of place. I criticize A, you say "but B" which is unrelated to A as a counter argument.
> It tries to cut funding to peaceful NGOs because they don't align with Zionist values.
Ah yes, "Peaceful NGOs"....
Even if they are "peaceful" (many aren't), that doesn't imply that they're good.
There's nothing inherently wrong with attacking NGOs. In fact, it's actually good to attack some NGOs because said NGOs are evil.
The fact that you've taken a side does not imply that the other side is wrong to oppose your side, let alone that this guy has done anything wrong in his opposition.
.
For HNers just out of college, it's important to note that Bayesian analysis has not always been as popular as it is now. In fact, even as recently as the 1990s, it was regarded with suspicion by many statisticians, who strongly disliked the idea that prior and posterior distributions are meant to represent subjective states of belief. I was fortunate enough to have a very progressive statistics professor in undergrad in the 1990s, who was interested in Bayesian analysis. It's my understanding that most upper-level statistics and probability coursework avoided doing much Bayesian analysis until around the turn of the millennium. (if you went to university in the 1990s or earlier, was this your experience?).
For those interested in learning more about this topic, an excellent book on the history of the Bayes theorem controversies was recently published by Yale Unviersity Press: _The Theory That Would Not Die_ by Sharon Bertsch McGrayne
To be fair to Judea Pearl, I do see that he wrote an essay entitled, "Bayesianism and causality, or, why I am only half-bayesian". Nonetheless, much of his work appears to involve what we now know as Bayesian analysis. So like many (all?) scientists who make major breakthroughs, Judea Pearl was going against the accepted understanding of probability by pushing Bayesian analysis in the 70s, 80s, and 90s. It's good to see his daring rewarded.