As a front end developer, I am really happy to see resources like this.
Developing for the browser is a real challenge. I think working with html / css /js has been a neglected skill for a long time - most software engineers look down on that type of work and its rarely covered in comp sci course work.
Still, its good to see a lot of progress has been made, this book included.
My only critique - why use python instead of node.js?
Basically: server-side JavaScript is just not as widely known as Python, and it'd be additionally confusing when our browser starts running JavaScript. And in-browser JavaScript is a bit too restricted (by things like the same-origin policy) to do the whole thing inside a browser.
I see. I tried searching the table of contents as to "why python" and could not find it, but that link does more than enough to explain the "why."
I am resisting the urge to disagree, but since you did (literally) write a book about building a browser, I will defer to your expertise and try to learn from you :)
>I am resisting the urge to disagree, but since you did (literally) write a book about building a browser, I will defer to your expertise and try to learn from you
I can see that you'd want to pick a language that is good for teaching with, hence why you went with Python. It's probably too late, but I imagine Elixir would also be a great choice for this project.
I agree, JS _the language_ would be a more "obvious" choice for this, since it is both generally much faster and more popular in web development. I assume they are using Python _the ecosystem_ here. It probably comes with packages better suited for rendering specifically?
I don't think either is a super compelling choice anyways for this type of work. I think you want to use a systems language here. However Python is completely fine as a teaching language. Pretty much anyone who knows a similarly structured language can read it. And there is very little noise. So it can serve as a good reference if you want to follow along with a different language.
I think writing this in javascript would add some confusion, as people learning about this might have a hard time understanding exactly where the javascript is running.
> Developing for the browser is a real challenge. I think working with html / css /js has been a neglected skill for a long time - most software engineers look down on that type of work and its rarely covered in comp sci course work.
IMHO, those are job skills and not comp sci topics. They shouldn't be part of a degree program (except the most superficial treatment required to get some ugly UI up that may be required for something else). You have your whole career to pick them up.
Python seems like a good match since the tutorial seems to use what comes with a base python install. Python ships with Tk, for example, so the drawing section is cross-platform and not bogged down with a lot of instructions on adding new stuff with a package manager. That would detract from the lesson, IMO.
This made me start digging into whether this was considered a "safe" way of executing untrusted JavaScript in a sandbox.
It's not completely clear to me if DukPy currently attempts safe evaluation - it's missing options for setting time or memory limits on executed code for example: https://github.com/amol-/dukpy
I'm pretty paranoid though any time it comes to security and dependencies written in C, so I'd love to see a Python wrapper around a JavaScript engine that has safe sandbox execution as a key goal plus an extensive track record to back it up!
If you want battle-hardened, I figure you can't do better than V8. Here's a Python wrapper that I've poked at a bit (it's not quite 100% feature-complete but it seems to essentially work): https://github.com/sqreen/PyMiniRacer
I don't see any specific claims on isolation/memory safety or safety in general on duktapes page. Both V8 and spider monkey actively fix new JS vulnerabilities, and V8 isolates are used in the wild to success. Cloudflare workers is an example.
So far, my co-author Pavel has taught from this book multiple times (including this semester). In the spring at least one other university will offer a course. We'll list all known courses offerings on the website.
Also, if anyone would like to teach from this book, please get in touch!
Looking at the book's table of contents, I disagree. Browsers may resemble OSes by the size of the code base or by the amount of optimization involved, or by their importance for the modern world, but not by the types of technologies involved. I doubt the Browsers course will intersect a lot with OSes course.
EDIT: Reading you comment again I suspect you might have been joking :)
They all happen at a slightly higher layer, but they all exist similar to an OS. (I'm not sure if the higher layer makes it easier or harder to understand - but in terms of what you need to know, an OS class or three is definitely helpful)
OSs are built on hardware, browsers are built on OSs. Even though they "sort of" do the same types of things, the abstractions are very different.
- OS memory management is more about managing hardware page tables, copy-on-write, page sharing, page protections etc. Browser memory management is typically custom allocators optimizing for different types of structures.
- OS process management is about using CPU primitives to multitask segments of code, context switching, timers, wait queues, interrupt management etc. In the browser, it's more about managing webpage/process relationships, application threading, rendering pipelines, and the JS runtime.
- OS network management is all about interfaces, packet processing, buffer management, low level protocols (ethernet, IP, etc.) Browser network management is all about protocols, and not really much about the hardware.
Sure there's a lot of broad-stroke similarity between OSs and browsers, but for a university course, where one typically gets deep into the details, they're entirely different.
I think that highly optimized browsers have to deal with memory usage and page sharing as well. And QUIC started out as a userspace implementation shipped with Chrome handling packets and such, so I think browsers and OSes are not "entirely different".
> IPC, memory management, process management, network management.
I imagine the non-trivial parts of these are done for the JS VM (correct me if I'm wrong), and therefore a VM design course would have more intersection with Browsers with respect to these disciplines than an OS course.
> security
This one is everywhere, it has no special connection to OSes.
> They all happen at a slightly higher layer
Slightly?! That's an understatement of the week! The difference in abstraction levels is huge here and the specifics of the two levels are very, very different.
> in terms of what you need to know, an OS class or three is definitely helpful
Sure. But I think it's as useful as any systems programming course. I can agree that systems programming is a good preliminary for both Browsers and OSes, and learning either of the two will teach you a good deal about systems programming, but I doubt they will repeat each other.
These are all part of the browser outside the VM too.
Multi-process architecture requires you to think deeply about IPC.
Memory management is all over the place - there isn't a browser without custom allocators, investment into GC, etc.
Process management -> see multi-process architecture.
Network management: Browsers need to handle a tremendous amount of network issues. I mean... that's what they do. Outside of the VM, too.
As for "security is everywhere" - the whole point of an OS (and a browser) is to make it possible for security to be everywhere. To provide the primitives that you can securely build on.
> Slightly?! That's an understatement of the week!
Not really, no. I've worked on embedded networking stacks, on full-fledged OSs, and on browsers. I stand by "slightly". Yes, granted, a browser doesn't get quite as bit-fiddly as a on-the-metal OS, but it's a matter of degrees, not quality.
Can you work on many areas of a browser without ever touching OS-like code? Absolutely. This particular book has a good chance of avoiding most, because it focuses on the rendering part.
But a browser, as a whole, provides an abstracted platform just like an OS. And it echoes many concepts, if in slightly different forms.
I think a good example of two solutions used in browsers and OSes that are meaningfully similar (and not inherent to all systems software) would convince me better. Do you have one?
The security problems of a browser are very similar to the security problems of a traditional mainframe OS.
You need to execute potentially malicious code in isolated containers (processes / pages). You need to protect the system from these processes, and protect these processes from one another. You need to run some code with elevated privileges / additional capabilities (setuid / permissioned APIs), including some with superuser privileges (root user processes / browser extensions).
Looks like RemoteHQ launched something called Remote Browser as well, which is a SaaS browser, but more for collaborative purposes with more than one person using the same browser at the same time.
In my view, a critical part of really learning how something as complicated as a browser works is by trying to build it yourself. That's why our book is oriented around building a browser as you go.
This also looks great. Might you or anyone else know where to find the accompanying video? I get "Sorry
This video does not exist." for the embedded video link.
Many years ago, probably 20, I went on a task of implementing a web browser. I remember I gave up at rendering tables. I couldn't wrap my head around on how to properly size them. It has become extremely complex quickly to address edge cases and I eventually gave up when I couldn't understand what's going after having a two weeks break. Probably if I had money and was able to commit full time I could eventually get it, but I had to focus on commercial work and putting food on my table.
edit: it's a great article! But nothing on rendering tables :-)
> Many years ago, probably 20, I went on a task of implementing a web browser. I remember I gave up at rendering tables.
HTML 5 effort has cleaned up a lot of behaviors and specified how browser tags should behave. So it is, possibly, an approachable task now. Still daunting though.
Tables are still not that clean! But luckily tables are way way less important than they were in the past, so much so that browser differences in table rendering leaves most pages readable.
I'm the rendering lead for Chrome, and know quite a lot about how it works. I also recently wrote a series of articles about the new rendering architecture of Chromium, see here:
Pavel is a professor at the University of Utah and has extensively studied CSS from an academic point of view. He also has a lot of experience teaching the material and making it accessible to students.
This is awesome. I've always wanted to know how the actual layout portion works (or at least, can work in a simple way). I think these kinds of resources are really valuable and people should be empowered to make bespoke-ish web renderers as the need arises.
Author here. Layout is my favorite part of the browser—my dissertation is largely about formalizing browser layout—and so far the book only covers the basics, like the layout tree and how layout is computed with multiple tree traversals, but even understanding those basics I think gives you a huge advantage when thinking about web page layout tasks.
This book's browser supports JavaScript but the book itself skips over it because the browser just uses DuckPy. Which is a reasonable pedagogical choice. Building even a moderately "realistic" JavaScript interpreter would be its own semester-length course.
(This is a really fantastic resource; I'll probably use it as the basis of course once I get around to semi-retiring and taking a teaching job. Thanks, OP!)
It's now nearly impossible to build a web browser from scratch because of runaway explosion of web browser features, and proprietary API extensions.
W3C here is unfortunately a part to the problem.
Standardisation is good, but letting google pour streams halfassedly written RFCs onto other browsermakers is not good.
Non-enforcement of standards is also bad, and it's bad to extend W3C privileges to companies who themselves selectively implement their own proposals, so others' browsers can't match their behaviour.
"The total word count of the W3C specification catalogue is 114 million words at the time of writing. If you added the combined word counts of the C11, C++17, UEFI, USB 3.2, and POSIX specifications, all 8,754 published RFCs, and the combined word counts of everything on Wikipedia’s list of longest novels, you would be 12 million words short of the W3C specifications"
It's well known that Drew Devault count is meaningless since it includes dupes, drafts, and unrelated specs. Still, the space to cover for a from scratch browser is huge.
Flow didn't start "from scratch" recently, it's an evolution of a primarily SVG+CSS renderer for set top boxes. They also re-use Spidermonkey as their Javascript engine.
> It's well known that Drew Devault count is meaningless since it includes dupes, drafts, and unrelated specs.
It's not meaningless. Because in order to implement a browser, you have to figure out which of them are dupes, deprecated, drafts etc.
And even that won't help you. Because a huge amount of "deprecated" standards are in the browsers. A huge amount of stuff in the browsers is still at the "community draft" stage, and yes, you have to implement that, too.
Author here—the browser the book works through is, uhh, pretty limited, so I don't imagine you'd want to actually use it for web browsing. It's more like writing your own toy compiler or operating system, to learn how they work.
Developing for the browser is a real challenge. I think working with html / css /js has been a neglected skill for a long time - most software engineers look down on that type of work and its rarely covered in comp sci course work.
Still, its good to see a lot of progress has been made, this book included.
My only critique - why use python instead of node.js?