Hey Aparsons, thanks for your feedback. As mentioned in the post, the Handbook is not finished and we'll continue to evolve, adding more chapters, meat to the current chapters, and we have other ideas like exercises.
It's good that you are interested in Scaling WebSockets, because that is the chapter I am writing now! I hope to get it live in a couple of weeks, once it is I can send you the new version.
Hi HN! I'm Alex, and I've been researching and writing about WebSockets for a while now. I'm the author of the recently released WebSocket Handbook. AMA about the WebSocket tech, the realtime web, Ably or anything related to Liverpool FC.
Hi! I want to create a web app like Google docs where multiple users can collaborate in real time to edit the document together (using a special link like gdocs generate). I want to save the docs in a MySQL db (no firebase)
My questions are:
1) Since multiple people are working together how does one manage conflicts, i.e. 2 people sending different edits simultaneously.
2) If one clients gets disconnected (4g) and then reconnects later how does it sync the changes it made during it was offline?
I recently watched this RAFT presentation (1) and I think I would need to use something like this?
What other alternatives are viable?
Also can I make it happen using just PHP, Javascript and MySQL?
It uses operational transformations ("OT") to manage conflicts, and it saves the data in MySQL. Technically any Django DB backend will work for storage, but the public demo instance uses MySQL.
One of the reasons I made this thing was to show that realtime apps don't need to require heavy frameworks or unusual databases. And it loads super fast.
I don't think you need Raft if you have a central database storing the document. You could also consider using CRDTs instead of OT, which may be more powerful but also more challenging to develop.
CRDT solve a part of your problem, and an important consideration is whether or not you want off-line editing. If you don't need off-line editing, then a WebSocket can do it.
I'm actually using my project to build a collaborative IDE (designer like Figma): http://www.adama-lang.org/
I'm going to be launching it as a SaaS soon so people can spin up a new back-end without managing an infrastructure.
As with REST APIs, you'd want to be able to authenticate to a websocket-based API using either basic auth, or a bearer token-based auth scheme. Unfortunately, the browser websocket API doesn't allow you to specify arbitrary headers in the websocket request, so it's typical instead to have credentials supplied via a query param (such as "accessToken" for a bearer token) in the wss request.
> so it's typical instead to have credentials supplied via a query param (such as "accessToken" for a bearer token) in the wss request.
If someone ends up actually doing this in a production system, remember to not to log the accessToken if you're logging full paths/URIs somewhere, as query params usually is a part of that type of logging.
Yup, true, although tokens should be ephemeral so less of a risk. Authenticating inline over the Websocket connection is valid too, but it does expose the socket connections to slightly more surface area of attack i.e. if you pass in a token as a param, the Websocket request can be rejected immediately. If however you authenticate after establishing a Websocket connection, then there is an attack vector where you simply open Websocket connections and never authenticate. Of course timeouts can be used to disconnect rogue actors, but it is a consideration.
It's the same thing as HTTP. Websocket starts off as an HTTP request with cookies, headers etc. Use those just like HTTP to authenticate, and your Websocket server should pass the user data to the websocket object
If you make a normal HTTP request first, the server can issue a standard HTTP cookie to the client. That cookie will then be included when the browser makes the websocket request.
However, websockets are not subject to the same-origin policy, so this exposes you to CSRF [1]. To protect against that, you should check the Origin header on the server side.
In a Golang app I'm writing now, I have middleware that authorizes requests. Authenticated requests have a header with a JWT token. I have an endpoint for websockets where if an authenticated request comes in (the handshake) that request is then upgraded to a Websocket connection. This is the cleanest authentication implementation I've ever used thus far, and I wasn't able to achieve the same thing in Node when I was using socket.io.
I'm sure there are repercussions to this on the client-side, but I haven't gotten to that point yet. I'm still writing the server and testing it using automated integration tests.
If I understand correctly, websockets is a thin layer on TCP that does buffer data so that the application get the whole message instead of chunks. I recommend using wss to secure the websocket so that it can't be hijacked, then you don't need to send a token in each message and can do an application layer handshake once. Basically the first websock message from the client would be an authentication message with a password, token or what not.
The first websocket message is the original request, which will have the users cookies / headers where your session information / bearer token should live.
We are monitoring it closely and super excited about what WebTransport will provide, which is both a more reliable and in many cases more performant transport. However, much like WebSockets, it's still quite low level and only provides a basic communication protocol. As such, like we have done with SSE, HTTP, Websockets and MQTT, our service focusses on what developers can enable on top of these lower level transports, such as presence, deltas, history of streams, limitless scale and fan-out of data, and the list goes on https://ably.com/platform.
When WebTransport reaches prime time, I'm confident we'll be supporting it.
It's good that you are interested in Scaling WebSockets, because that is the chapter I am writing now! I hope to get it live in a couple of weeks, once it is I can send you the new version.