Great article, and agreed on most everything you said. Regarding Prismatic itself, some constructive feedback (feel free to ignore).
My immediate first impression was that this was another app that would waste my time. I want less of those, not more. The absolute best news feed app I've used is Flipboard for the iPad, and even then I don't use it too much as I feel too much like I'm wasting time with it (like HN!).
Secondarily, the homepage doesn't grab you enough. The text in the graphics is too blurry and the pictures are generic. The homepage below the icons doesn't have the production values to explain why it's cool.
I'm not trying to be negative here. Some of the points in your article really hit home (email summarization would be awesome and paradigm shifting). So I wonder if you might focus more on making something that will save people time and solve a pain point vs. "another web-based time-wasting thing" (that may not be fair, but that was a first impression).
For example, can you scrape an inbox and list of Facebook/Github/Tumblr/RSS/Twitter feeds to get a single high-sensitivity greatest hits from all monitored news feeds? This way you can check a single page in five minutes on your phone and feel reasonably content that you saw the top headlines for that week. Kind of like news.ycombinator.com/best.
> For example, can you scrape an inbox and list of Facebook/Github/Tumblr/RSS/Twitter feeds to get a single high-sensitivity greatest hits from all monitored news feeds
You've missed the point already. The web is vast, on the whole filled with 99.99% crap. But .01% of something large is still very big. You cannot list the interesting things a priori with a few feeds. Prismatic is about learning about what you actually like to read and giving it to you. It's got the smooth/sleak usability of Google reader (read title, keep smashing J when something isn't worth reading, essentially letting you skim/reject more than 10 uninteresting articles/minute), but without the low recall "enumerate every site I think I am interested in" subscription model.
Ok, I might well be missing the point. But enough people upvoted that I think this is a common perception: "Oh, not another time-waster". So, maybe make the homepage address this issue head on with a good video that shows why this is compelling, awesome, and productivity-increasing.
I do think it's mostly a time filler like say, hackernews or a good subreddit. Maybe you'll find some article that will increase your productivity, but that's definitely not the focus. It's just about giving you a stream of articles that you'll probably enjoy reading.
j/k: Jump to next/previous article
up/down: Scroll to next/previous article with animation
space: Scroll to next article
s: open share box for active article
o: open active article in new tab
b: bookmark active article in new tab
f: Go to search field and find new interests.
r: to recommend active story
gh: go to home feed
gd: go to your feed of saved stories
gr: go to your feed of recommended stories
gg: go to top news 'global' feed
Another comment on Prismatic (it took me a few days to get around trying it): Hacker News and Reddit are good for finding interesting new information, and they combine it with a pretty good discussion that usually attracts experts. That combination is really fantastic, and even if Prismatic improves the information-finding, can it also link me to an expert discussion of the article?
Hi Aria. I have a question regarding the use of automatic sentiment analysis and it's relationship to choice. It seems to me that there is a fundamental divergence between the program of "give me what I will enjoy" and "give me what I need". It seems that your startup is focused entirely on the former question, and there's no doubt that if you are successful you will indeed be consuming more attention, which is of course the primary commodity in the attention economy.
But wouldn't it be better to use NLP to emancipate people from the attention economy altogether, to help identify not what people want, but what they need to he happier, and more engaged in the real world doing real things for real people?
>But wouldn't it be better to use NLP to emancipate people from the attention economy altogether
I work on ads recommendations, and yes, it would be "better" if I stop recommending KFC to the 36% of obese Americans & instead recommend Crossfit or Yoga or Gold Gym or Myoplex. If I do that, they will quit the site en-masse and the negative engagement rates will go through the roof, I'll be out of a job, will end up eating at McDonalds to save money, and then become obese, and then click on the same KFC coupon recommended by another data scientist hired to replace me :)
Yes, it would be "better" if people read the Economist instead of Cosmo, or went to grad school instead of building crud web apps, or watched Frontline PBS instead of Jerry Springer, but then, it would also be much "better" for the environment if we got rid of the 1000s of walmarts & safeways selling groceries & Uncle Sam shipped you healthy lettuce and spinach weekly via USPS....communism 101 :)
Sorry you got downvoted - wasn't me, I swear (it actually annoys me that people use downvoting to mean "I disagree".)
The problem with your comment is that you present a false choice.
I believe that there is strong - very strong - demand for real, lasting happiness. Indeed I think that is the strongest demand there is. NLP and technologies like it can be at the core of the tools that help people choose who they want to be and help (not coerce) them to act consistency with those choices.
I cannot reason with somebody whose main thesis is "I believe" and "I think". The real world actually shows the opposite of your beliefs. The obese guy does click consistently on KFC coupons recommended via NLP, and does dismiss ads of fitness products. Its really his choice. Maybe he has found "real, lasting happiness" eating the KFC :) I mean, who am I, a mere data scientist, to decide what leads to "real,lasting happiness" for the masses ? Even saints & philosophers can't decide upon that. So I do what NLP is best for - if the obese guy wants KFC I give him that.
Am not being flippant...I actually share your concerns very much, just that data mining in the real world has made me uber-cynical for the future of mankind.
Ah, but you miss the crux of my point: people act inconsistently with their own choices! You're right that it's not up to me to decide what's good for people, but what gets me is when people decide for themselves what is good, and then find themselves unable to do it.
How does prismatic actually use NLP for finding articles you like? Does it use sentiment analysis of reviews from people it thinks are similar to you?
Btw I think "subreddits on steroids" is an incredible testimonial catchphrase - it would have been enough for me to buy your app, but I don't own a smartphone.
I recently answered a Quora question about this that has many more details http://qr.ae/1viyQ.
The core NLP behind Prismatic is topic modeling; we don't use anything off the shelf, but something crafted pretty specifically to our needs.
We do model user similarity based upon interest overlap and social graph analysis. So we know how likely you are to care about what someone in your extended network shares.
Do you plan to publish the algorithm/model at some point, or is it too closely connected to the business's "secret sauce"?
The latter would be perfectly understandable, so not meant to be a hostile question, more curiosity out of personal interest of how you handle that aspect of private-sector research. Worries about that are part of what keeps me personally from jumping from academia. I'd like to do less paper-writing and more applied work, which fits nicely, but I'd still want to do some paper-writing and be able to discuss techniques publicly, which seems to fit more awkwardly. I assume it's possible to pull off both, but all the people I know who've left academia for startups have stopped publishing completely, and many of them doing ML/big-data stuff are quite secretive about their techniques.
We will publish on interesting aspects of our models. Frankly, our main reason for not doing so is time and resources.
In terms of industry research on the whole and publishing. It is true that you can't publish everything, but look at Google. Arguably some of the influential systems papers of the last decade or so have come out of Google. Google doesn't publish all the details of their search algorithms, but it turns out that because they address real-world problems they've done enough great stuff that some of it can safely be shared without endangering their moat.
Another aspect to consider is that while industry publishes less, we do tend to churn out useful open-sourced (Prismatic will definitely be doing that soon) that is of at least comprable utility to people out there as most papers.
Hard to say: systems has a long lag time before good ideas get "proven" good by being successes in the marketplace. For example, paravirtualization was investigated from around 2000, and then Cambridge released Xen in 2003. It caught on by the end of the decade, in the late 2000s. If something released in 2010 will end up having similar impact, we'll know it by the late 2010s...
Hi. I'm trying use natural language generation (NLG) software, and found lots of long dead projects, two large lisp-based packages (FUF/SURGE, KMPL) that are not updated in years and not user friendly at all and a simple java package (SimpleNLG) that is indeed simple but not very sophisticated. Is this representative of the NLP/NLG software?
Controlled languages are interesting for natural language generation purposes - for example, GF (http://www.grammaticalframework.org/) may be useful for building sentences (in multiple languages, if needed) from an abstract data structure.
- Do you think you would have the opportunity to avail yourself of the fruits of "academia" if the current approach was fundamentally broken?
- What would happen if viability of academic research was tied to one being a successful business person? (Asking in light of your comment elsewhere in this page re. "tenure system".)
Loved your post. Very articulate and nicely written.
I've been amidst the academia -> industrial fun transition myself this past year, absolutely best decision i've made in the past half decade.
One point you only lightly touched on, but which I think is worth restating is this: in academia theres this frustrating sense of "if you're not narrow, you can't be deep", whereas I see a lot of the more sophisticated bits of industry really valuing folks who strive for broad depth.
Would you agree with that assessment?
(admission: i'm presently having a go at building some tech products/tools that tie into a whole range of my researchy interests)
Really interesting article, especially since I'm an assistant prof idly considering the same sort of move. How much of a "business model" did you have before commitment -- did you have enough connections that you were confident you'd get funding once you had the idea mapped out in detail? Was UMass still a viable safety net, or were you fully committed?
