Rendering HTML to WebGL textures would be big, which I think this could do - though probably not as performantly as the browser could. It's currently not possible, partially due to security concerns if I remember right.
Awesome idea! I've seen tricks where CSS 3D transforms were used to position the entire DOM within a scene rendered in Canvas, but that approach is extremely limited. [0]
Theoretically, now you could render the DOM to a texture instead of to the screen and do some sweet deferred rending.
== Edit ==
Looks like one of Mozilla's Distinguished Engineers blogged about the technique. [1] [2] The creator of emscripten used this technique to render tweets as a texture in the bannabread demo. [3]
One of the drawbacks with rendering DOM content to a Canvas/WebGL texture is that interactivity is lost in the process. At the very least an intermediate layer would have to be created that pipes scene input into DOM input.
The nice part about CSS 3D transforms is that interactivity is maintained, and it's still hardware-accelerated. A fantastic example of this was made by the creator of Three.js over a year ago:
As you said above, there are severe drawbacks with that approach. It is very limited in terms of what's possible compared to a traditional 3D scene. Moreover, since CSS 3D transforms are essentially DOM content rendered to a texture, rasterization occurs, and as such there isn't clean scaling, even with SVG content.
It renders to Canvas, so it should be already possible; three.js even already conveniently allows creating texture from a canvas:
var texture = new THREE.Texture(theCanvas);
texture.needsUpdate = true; //refreshes the texture from the source canvas
Now combine it with React.js to avoid redrawing unless necessary and forward the mouse events back to Webkit.js, and you have an interactive HTML texture :)
It actually does, turn on chrome's canvas profiler its kicking up bindTextures for everything. On the demo it even says its a hardware accelerated version and requires WebGL.
At some point, it won't matter whether your browser supports a particular feature of the web. Any site could just embed a copy of WebKit for browsers that are too old (i.e.: when the latest version of Firefox is a few dozen versions behind) and render the page using the latest rendering engine.
Neat. Getting some good use out of HTML2Canvas as well. But always wondering why there is no way to get a bitmap from screen... something like window.getPixels(x, y, width, height).
One solution is for the browser to provide a "anonymized" bitmap picture, e.g. setting the links to default colors. I don't know how feasible that is. It would require a new rendering of the page, at least.
That's not a solution for the general problem of getPixel + iframes though.
Here, I'll give a simple example. Let's say there's a site that uses cookies to track logins. For example, hacker news. Now, hacker news does have the X-Frame-Option Deny, but let's assume it doesn't.
So to figure out my hacker news username, all you have to do now is create an iframe with hacker news, and then getPixel on the area of the frame that contains usernames, run some trivial OCR, and done.
Now, this issue is even more serious in other instances. For example, google talk widgets are embedded by iframes I believe.
In both these cases, a fresh rendering would still have the inappropriate material due to cookies. If cookies aren't sent, e.g. you do a fresh render in a private tab, it would still have security concerns for anything that displays different, sometimes private, content based on ip address.
For an example of that, you could render "private.internal.company.localsite" in an iframe and if a visitor from that company visited, even a cookie-less load would probably show private data due to the internal site relying on ip/nat controls.
You could, but it would change scope a little. Right now, it's reasonable to expect that information will not be leaked to external servers, but user interactions can be faked.
On an information only webpage (no user interactions) there's no need to have the X-Frame-Options to remain secure. If there's a way to access the data in that frame suddenly, that changes the necessity of that header.
I'm not sure what X-Frame-Options has to do with it. I'm suggesting either not allowing rendering of bitmaps containing different origin iframes/images, or blanking out those elements in the rendering. Canvas does exactly that by not allowing different origin images to be drawn into a canvas element.
Hmm, if I can get all underlying DOM, including CSS, what is the difference except massive convenience? I could imagine iframes of other origins to be blank, but that would be it, I think?
Also cross-origin images, which would affect a large number of sites. And visited link coloring [1], although I supposed the browser could re-render the page without it for screenshot purposes.
I've written GUI builders that really could have used this for icons of views. Instead I had to render on the server. It'd be nice if it got more broadly adopted.
Yes! This would open a huge number of possibilities. Simple things like a colour picker, to transitions based on the content of the window would be made possible. I would love to see this happen.
It's a little contrived, but a "practical" use of this might be for an online HTML editor to embed the document (or display a preview) without any interference from the parent document.
If you allow custom HTML via an iframe and make it shareable you have to host the iframe from a different domain, otherwise it's XSS vulnerability heaven. The trick will be authentication because separate domains don't share sessions (and you wouldn't want a secured domain access to your session anyway) but you can do communication via postMessage.
Anyway to get this to run in node with the output going to say Cairo? It'd be a fun little project to see if you could build a simple but working browser in nothing but JS.
Yes, there's a software rendering version, it actually uses Cairo. Software rendering just bitblts up (out of a webworker or as a SDL_UpdateRect) callback in javascript as a RGBA array.
I'd like you to consider the fact that your operating system is already a significant abstraction and that a browser is essentially a virtual machine that allows to download and run programs from any source safely.
You know all those malware issues regarding unsigned or compromised executables? Not a problem in the wonderful world where programs are incapable of acting overly malicious. There're specs like webgl and webcl that lets you take the scary topic of graphics programming, which is communication to an external piece of hardware, and make that as safe as loading some javascript to sort a table by dollar value.
Our entire modern day computing experience is built upon 'Layers upon layers of complexity, abstraction, technologies, etc.';Simple and easy doesn't necessarily mean less.
What exactly are iOS users missing in Safari, in your opinion, that they would get from Firefox? I can see that Chrome has some cool sync features if you're into that sort of thing. But it still uses WebKit. And from what I've observed, people seem happy with the "Chrome" app in the app store, even though it uses a handicapped renderer. (Although in iOS 8, it won't be handicapped anymore, and will be as powerful as native Safari)
(Also WebGL support is new in iOS). Are people really clamoring for the Firefox engine to be supported in iOS?
Most iOS devs aren't clamoring for alternative browser engine implementations because they're happy with the objective C runtime. Web developers shouldn't be happy with only having one browser implementation available because there's no competition on iOS. Safari on iOS can lag in features and not care because there is no competition on iOS. Chrome on iOS offers nothing in the way of additional html5 APIs and is just a new UI on the same old Webkit (to my knowledge).
For instance, as of about a week ago Safari on iOS just got WebGL support (as you alluded to). How long would we have had WebGL support on iOS if we could ship an alternative browser engine implementation with competitive JIT'ing JavaScript engine (like Firefox's Gecko & SpiderMonkey engines) on iOS since the beginning? Would alternative browser engines with more APIs drive the Safari for iOS team to ship features faster? Bottom line: competition is healthy; developers and thus users win.
> if we could ship an alternative browser engine implementation with competitive JIT'ing JavaScript engine
Which is not possible on FirefoxOS. So I don't see how 'open' wins here (I'm using 'open' as the GP used it: a marketing buzzword synonymous with Mozilla).
I agree that "FirefoxOS will host Safari / Chrome apps, before iOS will allow a Firefox." doesn't lead to "Open FTW." I think Open still wins, for other reasons I've been trying to document. [0] Firefox OS is still young, and if it can establish meaningful marketshare, it puts Mozilla in position to change the industry.
For example, what if Mozilla had leverage over device manufacturers to say "no locked bootloaders" and device manufacturers complied? I'm not sure Mozilla has that leverage today, but what other ways could Mozilla affect the industry for good once it does?
> Chrome on iOS offers nothing in the way of additional html5 APIs and is just a new UI on the same old Webkit (to my knowledge).
No, they forked Webkit due to all the browser related #ifdef's they had scattered across the source code. They were diverging to much and damaging the quality of the code.
Because competition in the browser market is vital to keep the web platform progressing and to prevent a single party from controlling web standards. Do you remember the dark ages when IE ruled the web and things like HTML5, canvas, webgl, etc... couldn't happen?
>competition in the browser market is vital to keep the web platform progressing and to prevent a single party from controlling web standards
I don't see much difference between a single party and a cartel of 3-4 parties. They are all mega corporations that don't share my interests and even when there are several parties any one effectively has veto power over any changes to standards.
The push for browser-as-OS is only going to make things worse. You don't get any browser choice at all when your hardware is running ChromeOS or FirefoxOS
That's true that we don't let other web runtimes be used on FirefoxOS so far. There are good technical reasons for that, not just "mozilla is evil".
Doing while preserving security would mean that such a runtime would run sandboxed (in the same way we render web pages with gecko in sandboxed processes). For that to work, the sandboxed process need to delegate some functionality to the "main" trusted process, like network access. This is happening through an ipc protocol (we reused the one from chromium) but the messages exchanged are very much gecko specific currently, so a lot of work would be needed to turn that into a more generic ipc embedding interface.
I'm pretty sure that the folks working on Servo would be very happy to plug in their engine there ;)
'It's more work to be open' is an excuse we have heard for many years now from the likes of Microsoft. Often it was valid from a business point of view, but it's pretty amusing to see Mozilla using it now after years of PR campaigning about the importance of openness.
Perhaps you are starting to realise that running an open platform isn't as easy as you supposed, and a lot of Microsofts actions were not just "Microsoft is evil" either.
> You don't get any browser choice at all when your hardware is running ChromeOS or FirefoxOS
Both browser based OS's are still in their early phases. What if the hardware could run both, just as easily as you could put whatever flavor distro of linux on your desktop?
There's too much closed source OS specific drivers to really allow you to run whatever OS you want on whatever hardware you want; and everyone is disincentivized unless money is involved. But what if drivers were open source? I'm not sure how to incentivize that (the community fixing shitty GL driver implementations, maybe), but I would bet you it would be a hell of a lot easier to get you favorite software running on your favorite hardware if they were.
Right now, closed source drivers are a thorn in the side of users having choice. With the rapid progression of the Mesa project in particular, when hardware manufacturers come around to releasing their drivers, then ChromeOS, Firefox OS, and Tizen will be poised and in a good position.
>Both browser based OS's are still in their early phases. What if the hardware could run both, just as easily as you could put whatever flavor distro of linux on your desktop?
This isn't a valid replacement for just being able to run multiple browsers. Switching OS has a large cost. Your FirefoxOS local apps won't run on ChromeOS (and vice versa). You have to work out how to get your files from one to the other and so on. If you want to just try another browser out you have to go through a process that is very intimidating and complicated for most people. The average user is not going to do it (or if they are persuaded to do it many will get angry and complain that all their apps are gone).
Making the cost of switching browsers much higher is an attempt to lock people in to a single browser.
FirefoxOS and ChromeOS are more than just browsers. They have a set of (currently) non-standard APIs for doing various things because the current browser standards don't cover many of the things you need to build a usable mobile OS. They are designed to run locally stored 'apps', not just browser web pages. Some proportion of apps will not be portable. Even if they were the user is going to have to manually reinstall them. Another way of looking at it: if FirefoxOS is functionally identical to ChromeOS (ie.e. the user won't notice switching) then why does it even exist at all? What is the point of a close if from day one you rule out the possibility of ever doing anything better or differently than the competition?
It really doesn't matter if it's possible to work around these hurdles to changing OS, it's still a massive amount of effort to think about all these things rather than just being able to install a different browser alongside. Normal users will not be re-imaging their phones to different OSs just to try a different web browser. It just won't happen in any great numbers. After a hard fought battle over many years to get people over the relatively small hurdle of installing Firefox on open platforms, I would expect Mozilla of all people to appreciate how difficult it would be to get people to try out different browsers in a world.
Also, I know it's kind of taboo to talk about this around here, but it's worth also mentioning what web apps mean for user privacy and data security. You seem to be suggesting that people only use web apps where their data is never stored locally, only in the cloud. In data centres where governments can mine it (often without warrants or any kind of due process), where it can be mined to build profiles on users for advertising or insurance purposes. That kind of app also leaves user vulnerable to all their data disappearing when the web service they use runs out of money or gets bought out or just decides to ditch a feature.
I agree on principle. Many of us still have emotional baggage from the days of 95% IE market share, where the web really, REALLY sucked.
But nowadays, I see it as, multiple companies have a vested financial interest (including MS with their new leadership) to ensure that browsers are very powerful.
Even Apple, with iOS 8 has enabled 3rd party devs to have a full speed browser now with a JIT'd JS engine. They are simply not doing this from a position of weakness. There are no market forces. There's nobody out there saying "I won't use iOS because it only has Safari." Yet they still did it.
Your mobile platform won't succeed today unless it has a powerful browser built in, apps or not.
> There's nobody out there saying "I won't use iOS because it only has Safari."
FWIW, I said it and did it.
I didn't like Safari, and I couldn't change the default browser to anything better. Even worse, most 3rd party apps used their own utter-crap in-app browsers instead.
So I switched to Android where I have freedom to set my preferred default browser, and 3rd party apps actually respect that.
I do agree with the rest of your comment — these days you can't have a platform without a great browser.
Having to convince users to install your fork of the OS really isn't the same thing as being able to run a different native browser alongside their existing software. If the only way to use a browser other than Internet Explorer had been to switch OS then Firefox could never have been created in the first place.
How is that any different than with native code? You can't just rip out your OS' implementation of libc and have all of your programs use another (not all implementations of libc are ABI compatible).
You can't boot your computer into two operating systems at once (or can you?). This whole post is analogous to running Windows in a VM on Linux.
On an open platform I have a choice of web browsers. On a Javascript sandbox based browser-OS I don't have any choice.
>You can't just rip out your OS' implementation of libc and have all of your programs use another (not all implementations of libc are ABI compatible).
I'm not talking about using a different browser engine to run local apps. Obviously there are going to be platform specific APIs that mean things won't be compatible (unless of course Google and Mozilla agree on a set of APIs and don't implement any vendor specific ones at all, but that is highly unlikely since it would mean killing backwards compatibility with current ChromeOS and FirefoxOS apps).
I'm talking about being able to use a different browser stack to browse the web. FirefoxOS makes it impossible to do (with any kind of respectable performance) but it's trivial on a native open platform.
>You can't boot your computer into two operating systems at once (or can you?). This whole post is analogous to running Windows in a VM on Linux.
This is just a silly semantic argument. You are defining the web browser stack as part of the OS just to avoid the awkward reality which is that you have totally locked out all other browsers from your platform. It doesn't make sense to run two kernels (unless you are doing things with a VM) because the role of the kernel is to talk to the hardware. That isn't the role of the browser stack, so there is no reason at all to only allow a single browser stack to run at a time. It's quite possible to run several, which is of course trivially demonstrable on an open platform like GNU/Linux or Windows. If you really want to make analogies, it is more like running a second windowing system. Some OSs like linux make it possible to swap your window manager, others like OS X don't. But the most important thing is that on either you can always just run native software full screen and draw whatever windows you want with no performance penalty (which in this analogy is akin to being able to just browse websites using a third party browser stack, but not being able to run the local apps using it).
Safari in iOS is missing many newer APIs. For example WebRTC is not in Safari but it is in Chrome/Firefox. No idea if it's been added to iOS8. Even WebGL in iOS8 is currently hobbled.
Also it's not a matter of "clamoring". People have no idea what they're missing. Do you think anyone would have been clamoring for Firefox on Windows in the 90s for they had never seen it and were limited to IE?
Maybe I'm showing my bias here but remember the browsers before Chrome and V8 came on the scene and heated up the competition? Well there's no competition on iOS because you can't write your own browser. Heck, I'd certainly like to see ASM.JS on iOS as one example.
The funny thing is that the "RAWR NO SAFARI, ONLY CHROME!" people don't seem to realize that both are WebKit at the core. Sure, some features are in Chrome first, but that's only because Apple is very cautious when releasing the new features to the general public.
That's because Apple's TOS only allows their system build of WebKit in approved apps, so Google can either use approved WebKit or they can go sit on their thumbs and spin. It's not a voluntary act on Google's part; they would use Blink if Apple allowed it.
Sort of, they're very similar still. In actually the layout is almost exactly the same, its the rendering and compositing that has significantly changed between the two.
Google cannot innovate with Chrome for iOS the way they have innovated on other OSes. All iOS browsers are competing only on the UI layer. Fixing missing web standards or inventing new ones is not possible.
I can't edit my comment for some reason, but as thefreeman pointed out in response to agildehaus' comment, I was referring to how the iOS Chrome is still WebKit.
Very nice and with JS support it would be perfect! I wonder if this is a solution to potential XSS attacks with embeded iframes or embeded user-generated html+css+js.
Doing this in the browser might be a little silly, or at least I can't immediately think of a good use, but I think using this in a node.js program as an easy way to template something out and render an image/pdf locally could be useful.
Is this a big needed feature? It's actually really easy to pull of, cairo's backend can be switched from using images to svg, to pdf's. It's just switching the backend and once the renderer is done just spit out the results.
Add it as a feature request on github and i'll add it to the api.
I remember when for some reason people started writing SVG and HTML engines for Flash. Because, I don't know. They wanted to put a browser in your browser, so you can browse while you browse.
The engines worked, but where are they now? Nowhere. Just like Flash.
No matter how much effort you put into something, no matter how functional it is, now matter how much geek cred you'll get for it, if it doesn't have some sensible use, at least superficially, it'll linger and die. Wasted effort.
this is pretty cool but network requests and javascript support would make it even more crazier.
I viewed the html source for this page and pasted it. I was quite impressed. It looks like this is supposed to be a browser engine written in Javascript?
So essentially we would have a sandbox browser within a browser? That would be awesome.
> javascript support would make it even more crazier
There's interesting research being done in the area of meta-circular interpreters. There's techniques for speeding them up; they're not as slow as I'd expect. For example, one interpreter can do type analysis and rewrite chunks of code.
another interesting question would be will webkit.js support single origin policy? basically, what if you could use webkit.js like an iframe but with the ability to execute javascript and manipulate that window.
https://www.destroyallsoftware.com/talks/the-birth-and-death...