The DevTools Protocol is the primary API for headless Chrome, but we are excited for higher-level abstractions like PhantomJS & NightmareJS's API to manipulate the browser as well. Plenty of details to work out, but hopefully sometime this year you'll get a drop-in solution for some of your testing to upgrade from Phantom's older QTWebKit to the latest Chromium.
> >Is there interest on your side in adopting Chromium as a runtime? There's some existing documentation [2] around the API and embedding, but admittedly, this would be some work.
> We are interested. But I am afraid not in the current state. Currently, PhantomJS heavily relies on Qt and QtWebKit. It's not that easy to adopt Chrome as a new runtime.
> But I think we could implement PhantomJS as a completely new (with the same API) project that will use Chrome - Phantomium!
It was my understanding that Google's crawler literally is Chrome. Does Google have any plans to open source those parts of the browser to make integration easier? Maybe I was mistaken
That seems unlikely... It would seem they have two sets of crawlers... one that does typical/advanced scraping, and another that runs the JS (Chromium) and takes DOM snapshots. This is reflected by changing certain properties (window title etc) and seeing them reflected in Google's search results. A couple years ago, the lag was several days to a week behind on new content via as-rendered from the server vs. via JS.
Interesting! I find it puzzling that insider knowledge on how google search works never seems to leak into the public domain. Is google killing leaks in their search algorithms or do they pay the search team such ungodly amounts of cash that nobody has ever left?
Well, the point of their engine is to make it harder to game the system... everytime someone figures out a trick, someone takes advantage of it... that alone would discourage leaks.
We laypeople can only find out things via word of mouth or observational tests on assumptions.
The DevTools Protocol is the primary API for headless Chrome, but we are excited for higher-level abstractions like PhantomJS & NightmareJS's API to manipulate the browser as well. Plenty of details to work out, but hopefully sometime this year you'll get a drop-in solution for some of your testing to upgrade from Phantom's older QTWebKit to the latest Chromium.