The loop of app building:
1. Build app with native framework.
2. Re-build app with some alternative framework which is slower, but let's you deploy single code on all platforms.
3. Re-build app with native framework, because it is faster.
4. GOTO 2
The best part about this cycle is that developers can always point to some metric which shows improvement to justify their actions, resulting in an Escher-like loop where they think they’re always going up.
These cycles pop up all the time in development as priorities shift (or as developers find more excuses to “fix” what isn’t broken). Real world coding always involves compromise - leadership is recognizing which compromises to make without continually backpedaling.
To be fair, small fast messenger 2020 is "better" (aka more features / millisecond of cold start) than small fast messenger 2016. The cycle doesn't always go up monotonically, even if developers pretend it does. But it is possible to go "full-circle" through the loop and wind up in a better place than you started, and it's even possible for each stage of the loop to be truly justified given product needs.
indeed, it could be rational business needs. at a time when it's more expensive to hire devs, maybe it looks better to have a single codebase. at a time when performance is more valued, maybe it's better to rewrite in native.
I don't know what world you live in, but in my world developers don't have to look for excuses to do work. Perhaps some companies running on endless VC overhire, but the real working world has fewer employees for most positions than it needs.
And in the case of Messenger, I highly doubt it was the developers who said, "Hey, let's add this completely unrelated thing to our codebase - payments!". More likely it was marketing or "product" that told the devs what to add.
From TFA: "We reduced core Messenger code by 84 percent, from more than 1.7M lines to 360,000."
Sounds like not only did they make the app quicker and lighter, they improved code maintenance too. If the android app is as the same size as the new ios messenger app, they'd still have less code to maintain than the previous ios messenger app.
How in the name of everything did they manage to write 1.7M lines of code for a messenger app? That is an astonishingly large code base! We're talking about an instant messenger app with heaps of features, having a LoC footprint 3.5% of the size of modern Windows.
> Since then, we’ve added payments, camera effects, Stories, GIFs, and even video chat capabilities
Simple: they added tons of crap nobody care. On this list, I only use video. Messenger is valuable because of the network effect, but feature wise 2003’s MSN was good enough.
That's not true, the application, whether you like or not, it full of micro-interactions, well-done animations and transitions, features like the payment or stories, you can take photos directly from the app, etc.
With that many features, it doesn't surprise me that the previous codebase was 1.7M LoC. Sure if you count only the messaging part, I'm fairly certain that the LoC is way lower than this.
It doesn't make sense to compare the number of LoC of messenger to the number of LoC of software like Postgres because Postgres tries to be good at 1 thing and messenger tries to have a ton of features with good UI and UX.
I don't like many of their features (I'm not using the stories or the payment), but some of my friends do and you have to recognize the work that the messenger team did to make them work that well.
One of their points in the article was how they've decided to use more of the native OS interfaces instead of implementing their own versions (!!). For instance, using the OS's native threading library instead of writing their own. So, the number of LoC kinda start to make sense when you think about them re-implementing some not-trivial chunk of an OS's native functionality.
I've never felt the need to justify my own existence. There's always been a longer backlog of work than my teams would ever get through and managers who appreciated the work getting done.
Another plausible explanation is lack of code re-use and reinventing the wheel because you don't know someone else solved the same problem. Also correlates with number of engineers.
I had the same question... And that's not counting LOC on the server. And all of this is for... glorified text messaging, right? I don't use FB so I could be missing context or features.
I wonder how this will effect Lite. I use it over Messenger strictly to reduce bloat and get an app with the bare essentials of chat and calls fol friends that refuse to use Signal.
the reason is they're high paid engineers without skin in the game. Fb is already making money, hence a lot of engineering efforts are not related to having fb make money, but to prove the engineer's worth.
Somehow younger generations are allergic to it, you seldom see it listed as alternative for portable busines code, even though it is available out of the box in all mobile SDKs, with first class IDE support alongside the respective upper layer languages on the platforms.
And the amount of time spend writing a bit of glue code, specially on Android, is still less than debugging integration issues in all those Hybrid/Abstration frameworks.
Rather than reinventing the wheel, we used the UI framework available on the device’s native OS to support a wider variety of application feature needs. This reduced not only size, by avoiding the need to cache/load large custom-built frameworks, but also complexity. The native frameworks don’t have to be translated into sub-frameworks. We also used quite a few of the OS libraries, including the JSON processing library, rather than building and storing our own libraries in the codebase.
This is awesome. It shows the benefits of building an app leveraging the OS, instead of fighting or abstracting it.
They make it sound like the clouds parted and a ray of sunshine struck the programmers at Facebook and they realized that they didn't have to ride the framework-go-round to hell.
These are supposed to be the smartest people in SV, but from the outside, it strikes me more as a "No shit, Sherlock" moment.
Which, for example, gets you more sway with your employer, personal "branding" value, and future income / ease of switching jobs: "I was instrumental in inventing React" versus "I was instrumental in realizing we didn't need to invent React"?
Well yeah, but first someone has to get the credit for creating some "high-productivity" hog of a framework to make it slow, then someone else can come tear all that crap out and claim they made it fast.
I guess it depends on if you frame it: "I was instrumental in adding complexity under the false promise of ease of development" versus "I was instrumental in saving the company time, money, and improving the product for our users."
Things like this are why I'm glad I'm not in the bubble anymore.
"I was instrumental in a major rewrite for [REASONS]" vs "We kept it the way it was because it was good enough and a rewrite would've improved [X] but hurt [Y]"
At a previous employer, I was dumped a client lib and server whose purpose was to maintain a data cache with realtime updates. It was fragile and slow as fuck.
When I got ahold of the source and started diving into it, discovered the client had 2 copies of the data, which was kept in memory. The server had 3 copies of the data in memory! Worse yet, the primary cache would be updated, then a multicast message was broadcast to update all the clients and the server's secondary and tertiary caches...thats right: the server would update the 2 other caches by broadcasting a multicast message, receiving it, then updating the other 2 after message receipt. A re-architecture was definitely in order. Took 3 months. Took another 3 months to get the interface to be backwards compatible with the original version.
Mind, the server was responsible for serving up relatively static market data for finance. But, considering we traded globally, we had +400 objects we cached, more than a few had 100ks of record. Memory usage for the server was many GB.
The rewrite involved moving to a single cache, instead of multiple for both client & server. It was also architected to handle large volumes of updates. Original implementation could even handle 100 updates per hour without bringing the server's and all clients to their knees. The write could handle +10k updates per second amd clients would barely notice a slowdown. Write locks were only held for a pointer swap. Memory usage for the server went down by close to 2/3. Client memory usage by 1/2.
During that time, also moved from static libs to dynamic on both Linux and Windows, so I could make nonbreaking changes without a firm wide recompile/relink, which typically took 3 days. HUGE productivity boost. A small bugfix didn't require a firmwide recompile/relock for the smallest bugfix.
I dunno -- you make it sound like it's a priori easy to figure out what is going to be better: one codebase ala react or multiple implementations per platform. Where the definition of better is some admixture of user experience, hardware requirements, maintenance costs, ease of hiring / training people for the necessary skills; and all these may shift over time.
But could it be that the OS functionality also evolved until today where it’s finally feasible to do this?
I’m guessing the oldest code in Messenger stems from a time when the OS-provided libraries were not up to par or simply didn’t exist yet. Partly because it’s not until now that the OS manufacturers finally see what there is demand for in apps.
Oh yeah, this is a problem with a lot of developers, reinventing the wheel because the native JSON parser might be improved upon. I mean it's a clear case of NIH, and yet it's also defensible because I'm sure most native / builtin JSON parsers are not as fast as a custom solution. And for an app like Messenger, moving to something else like grpc would be very defensible.
> For any cross-platform logic, we used an operating extension built in native C code, which is highly portable, efficient, and fast. We use this extension for anything OS-like that’s globally suboptimal, or anything that’s not covered by the OS. For example, all the Facebook-specific networking is done in C on our extension.
C was the universal donor 30 years ago, and it's the universal donor today!
If you look further down the article, it describes their extensive use of sqlite, and a layer around it for managing synchronization with FB servers, also written in C.
I write a lot of Python and a lot of Go, and I love C. So I enjoy imagining the subgroup of HN commenters who wail that Go is ignorant of the last 30 years of CS technology, or that no one should be allowed to write C these days, reading this.
And then there are those ignorant of the 10 years of systems programming that preceded the inventation of C, and how the computing world looked like at Xerox PARC, IBM, DEC, Wang, Olivetti,....
Quite a few comments seem to imply Messenger was written in React Native, so I want to clarify this is not true. Messenger was a fully native app before the rewrite, too!
RN isn’t the right tradeoff for Messenger — whose new core is written in plain C — but its use at FB in general is growing, with 750+ screens in RN. So the rumours of its death are greatly exaggerated.
I think it’s great to have different options with different tradeoffs available to choose from. But hey, that’s just me.
So, they've reduced 1.7M lines of code to 360K. Let's pause here for a moment:
- NumPy is 360K lines mostly C
- Postgres is around 2.1M of C
- Go 1.13 is around 1.5M of Go code
- Rust 1.37 is around 1.2M of Rust code iirc
- core llvm is around 3M of C++ iirc
...
A chat app is 1.7M. A chat app...
Makes me feel sorry for all the time spent/wasted by all those devs, many of whom are unquestionably brilliant and could've worked together on something truly awesome, big and useful. But we all know that doesn't pay the bills and so here we are.
Honestly, a chat app that 1 billion+ use per month that includes payments, camera effects, extensive social integrations, stories, GIFs, reactions, games, polls, voice recording, calling, video chat, and works across all mobile platforms, with nine years of technical debt, that thousands of developers build simultaneously seems WAY more complicated to me than any of your examples.
Messenger may not have "HN clout" but the devs who work on it are having some of the greatest software impact in the world.
I think that most of people of HN have friend groups which are wildly unrepresentative of world population at large. We're talking not even an average US or European citizen here - we're talking 1+ billion. People from developing countries who very often access internet only from a smartphone (they don't own a computer) and who almost don't visit plain http websites outside of the few social apps.
It's an inner-platform effect, certainly, but not one FB asked for. If they were allowed by the app-stores to factor their app into separate "plugins" that were auto-installed when a friend used them in a chat, the way iMessage does, they'd probably do that. But they're not allowed to have their own seamless plugin ecosystem, so into the monolith everything goes.
No, they specifically used Messenger as a way of pushing all of those different things on its massive install-base. Anything that distances them from the user in any way would compromise on that objective.
Facebook and all the rest benefit from developing closed ecosystems that others can't benefit from. It's not a coincidence that their systems are systems-within-systems.
I'm on both sides of the fence on this one. On the one hand, it's people underestimating how much features are in the app that they've never seen, used or heard of. On the other, if that is the case then is their presence defensible?
I wonder if they could make apps more modular, and only load the binaries for e.g. payments on demand. I know Apple offers a system that does that for content chunks (e.g. levels and assets in games), and in a web-based application (e.g. react native) they could use bundling to remove it from the initial download.
But the chat app is only the client side, not the whole app entirely in itself. If we consider the lines of code in server side, there will be a lot more lines code.
it seem you don't have experience on large scale apps... and the people that make these type of comments almost never have had to work on large applications.....
Spotify had about 500k, lines of code in 2015, (not including external libraries).
We did make it in a feature based architecture, where there was a main container app (and a major core playback library) and for every major feature they had their own sub-project.... and there were 10+ teams contributing into it.
There were about 13 features at the beginning, (think, playlists, radio, shared ui libraries, etc...).
Each feature was about 10-20k lines of code in total. So, about 2-3 features equal a small standalone iOS app.
When I left, there were more than 80 features in the app. Some were essential, many were a/b tests, and some were probably just legacy/waiting to be removed.
80 features * 15k lines of code = 1.2mil loc, easily, and they are the equivalent of shipping 20 small apps standalone....
What are these features and why is just a stupid music app so large? : video, podcasts/shows, genius, etc.. etc.. etc.... To users just looks like another tab but almost every one of these major features could be its own standalone app for size and complexity.
Thank you for saying this. The overwhelming majority of comments on this thread are pedestrian - as if FBs engineers have been making decisions out of naivety, incompetence or self interest.
It might look that way in hindsight, but hindsight is easy. As is armchair theorizing about the competence of other developers.
Most people (including myself) will probably never see a >500K loc application or a project with >100 developers working on it. Biggest ones I've worked with had around 30-50 odd developers?
I would love to read more about why large apps are so big.
I currently work on a project that is much larger than I would expect it to be, both in terms of codebase and people, and while I don't know who to jank, I also don't understand why it has to be so big.
But your garage (the first part of what is now your house), was built with a weird kind of screw, that was only sold by one shop at the time, but was cheapest.
Now your entire house is built with those screws, but they’re not produced anymore so you’ve set up a small smelter next to your home where you’re making them yourself.
Unfortunately the quality is a good bit worse, so parts of your house start degrading faster and faster, but meanwhile you’re still building all new parts with your shitty screws.
Nobody wants to replace the screws because “It’s a proven method”, ignoring all signs to the contrary.
All new parts of the house are only as good as the worst part, the screws, and the moment anyone steps on the wrong plank the whole room comes crashing down. But nobody worries, next time they’ll just use two screws.
Part of it also comes from the tendency of large orgs to ship their org structure.
It can oftentimes be worthwhile to have individual teams each fully "own" a piece of the product, and that also comes with the autonomy to choose to duplicate stuff that other teams might have already built rather than working with them and tweaking it to match their own specific new requirement.
This significantly reduces communication & coordination overhead for delivering new features, at the expense of consistency and code reuse, which can often be a worthwhile tradeoff at scale.
Turns out there's the same amount of code in the Spotify app as there is in the product I work on...
True though, if you want more features you need more code. Sure, loads of people on HN could build a 'chat app' in a lot less but if they kept it going for ten years (same age as our product) and kept on adding new features the lines would start to add up...
Mostly Java.... some scala (some teams might use it), and potentially other languages. It built its own api lightweight java framework around 2014-2015 (called Apollo), which is actually pretty decent/nice to work with. There was a lot of protobuf usage early on, but most teams started transitioning to plain json.
So, each team runs its own servers, for the features it is developing/maintaining, and most of the api goes throw a common gateway, etc.. etc... don't want to give out more than this.
Some of the simple front end web pages (i.e. marketing websites, analytics) might even be on php/python/ruby. Some teams wanted to use javascript (Node), but those are not the critical main app, as java is pretty much required to be used for anything that affects the main app.
Also, if you are doing machine learning, then whatever it gets the job done (usually Python).
btw, I left few years ago, so things might have changed.... I know at least one team that wanted to use GO for some of their services....
I can't shake the feeling that the dominating factor in growth of LOC isn't the number of features per se, but the number of features worked on in parallel by separate teams.
I thought this comment was making the opposite point, that it was impressive that Messenger was now only 360K LoC.
I honestly think it is. Compliers, DBMS, a library such as NumPy— they're all complex, but Messenger, even considering the features removed, is a very full-featured application. We're not talking IRC here. It has activity status, payments, stories, photo filters, audio messages, location sharing, video-calling, etc. It's actually pretty impressive that that fits in 360K LoC IMO.
> We also used quite a few of the OS libraries, including the JSON processing library, rather than building and storing our own libraries in the codebase.
The 1.7m was pre-rewrite and the quote about using os libraries was post-rewrite; it doesn't prove they counted the libraries in the 1.7m metric but it does imply they bundled more before.
> Makes me feel sorry for all the time spent/wasted by all those devs, many of whom are unquestionably brilliant and could've worked together on something truly awesome, big and useful
I'm really not a fan of this virtue shaming that people often tend to do as developers, where if you're not working to solve world peace then you're a waste of a developer.
Sometimes people just like getting paid? Sometimes people just don't really care, they just want a job or want to write code?
While I understand the sentiment, you are making an assumption here that all messenger is a chat app. It seems excessive ( and it is ), but you don't know what kind of heavy lifting is being done under the hood.
I am not really defending it, but number of code lines alone is not a good way to evaluate code.
I have a set of code I use for my VR projects that I've been iterating on over the last five or so years. It includes a ton of not-VR stuff, too, just stuff that I tend to include in my projects. (aside: and I'm not trying to make it a "framework". It's just my own code and it's only publicly available in the sense that I don't charge for it, don't want to waste a private Github repo on it, and won't stop anyone from using it, but I'm also not going out of my way to teach them how to use it).
THAT code, my "all encompassing, personal repository for every VR app I'd ever make", is only 100K lines of code.
I once downloaded all of my dependencies and counted all of those lines and still only got to 200K. Well, not counting Unity. But at that point, you might as well start counting the LoC of the OS.
Yes, I know, but I do also occasionally share code snippets out of it. If someone in a chatroom is asking how to do X, I'll just link to a source block in my repo. It's lazier than copy-pasting the code, but I also think it's good to let people see the broader context, maybe explore on their own a little.
I wouldn't mind if other people started using the code, but I've been burnt too many times during attempts to push my OSS projects over the years that I just don't want to put any actual effort into attracting people anymore.
It's a chat app that works very well even on the crappiest connections.
If it having 1.7M lines of code indicates anything it is that creating a "simple app that provides simple functionality over a network well" is hard.
Look at the garbage generated by Google - text messages and images on Google hangout are not guaranteed to arrive in order ( or even be sent in order ) unless you are on a great network.
> It's a chat app that works very well even on the crappiest connections
Does it? I routinely use Telegram, Line and Messenger. Messenger is without contest the one that gives the most trouble.
I've been using chat apps at least since ICQ (1996) and sometimes I feel the user experience has declined.
That being said, I'm sure that a lot of progress have been made and that there's a lot going on behind Messenger in terms of security and scalability. I'd be curious to know what's hidden between the millions LOC.
I have, with an OpenVPN-over-UDP tunnel as a workaround. If they included Wireguard (or better yet, just used encrypted UDP) it would still be smaller (and probably more reliable).
I don't know if this is what you were referring to, but for everyone else: this is actually super easy to add with Wireguard because Wireguard just uses Linux networking you can do things like:
$ tc qdisc change dev eth0 root netem loss 0.3% 25%
(Lose 0.3% packets on average, but each packet is 25% correlated with the last packet loss, to produce bursts.)
$ tc qdisc change dev eth0 root netem delay 100ms 20ms distribution normal
In my experience in places where connectivity sucks (barely able to get data handoffs between stations, sticky wifi, etc) messenger kicks ass while the other applications hick-up big time.
> It's a chat app that works very well even on the crappiest connections.
That's a joke, right? On a crappy internet connection (e.g. using O2 as carrier on the train cologne-hamburg you get ~12s ping, 30kbps, >90% packets dropped, via edge) most chat apps stop working, but Facebook Messenger is (except for Riot.im and Discord) probably the one that works worst.
Telegram works better, even IRC via Quassel/Quasseldroid works significantly better, and especially WhatsApp works so well you don't even notice the connection issues.
Facebook Messenger doesn't even properly start on such crappy connections.
Repeating a point without adding proof doesn't make your point more valid. When people say "code wins arguments", they're referring to using source code to prove something, not "it wins because it has more source code" (???).
People say "Code wins arguments" because written code that works demonstrates the winning argument against the code that is either not written or does not work.
FB messenger exists and works on crappy connections. Google hangout exists and messes up on crappy connections. Telegram exists and messes up on crappy connections ( see the issues it has in Africa ). Signal exists and does not work well even on non-crappy connections. Whatsapp exists and sort of works on crappy connections. Matrix exists and sucks on crappy connections.
What we need is something that exists and works on crappy connections that has less LOC than FB messenger to demonstrate that it is possible to handle crappy connections and modern messaging in much fewer lines of code.
> written code that works demonstrates the winning argument against the code that is either not written or does not work.
That's why your comment is nonsensical, if not downright contradictory noise.
> Ordering messages and providing reliability does not require 1.7M lines of code.
This has been demonstrated ad nauseum a priori with loads of toy chat apps that reconnect and retain state. It's not really a challenge. EVERYTHING else crammed into it, should be the reasoning for 1.7M lines of code, but as they demonstrated TO THEMSELVES AND EVERYONE, it wasn't (360k now)
> This has been demonstrated ad nauseum a priori with loads of toy chat apps that reconnect and retain state. It's not really a challenge. EVERYTHING else crammed into it, should be the reasoning for 1.7M lines of code, but as they demonstrated TO THEMSELVES AND EVERYONE, it wasn't (360k now)
Yes and they figured out what was needed after first writing a 1.7M line of code thing that worked. That's the process of reiteration and a process of initial LOC bloat before LOC stripping.
Without that process by thinking it has all been demonstrated ad nauseum a priori with loads of toy chat apps that reconnect and retain state you end up with deciding that if Friday was Feb 28th then Monday is certainly March 3rd because date/time handling is simple and end up being Robinhood, down for a day.
React Native isn't necessarily a great fit for something that is intended to be "a simple, lightweight utility" - there is a reasonable start up overhead, for example, with RN, and it also involves bundling quite a few libraries and dependencies.
That doesn't mean React Native is not a good fit for any kind of application - for many kinds of apps, start up time and app size are a much smaller issue - but seems Facebook made the right decision for an app like Messenger which primarily needs to start fast and drain as few resources as possible.
Yeah, not sure where this idea has come from. Seems like RN haters just jumping on any opportunity to slate it. Reality is, just like most technologies, RN is a good fit for some scenarios (small team needing to support both mobile platforms? no native devs on the team? performance not critical? etc.) and not a good fit for others.
I guess many native developers feel negatively towards it because a lot of companies will be asking "why aren't we using React Native instead of building two separate apps?" which I can imagine feeling somewhat threatening towards your area of specialisation
> Yeah, not sure where this idea has come from. Seems like RN haters just jumping on any opportunity to slate it.
There seems to be a much more obvious explanation as to why people are assuming it used to be React Native. The article says:
> > Compared with the previous iOS version, this new Messenger is [mentions improvements].
> > We accomplished this by using the native OS wherever possible […]
That seems to imply that it wasn't native before and they got the gains by switching to native. And since Facebook are well-known as the creators and biggest users of React Native, it's not a big leap to make the assumption that that's where they were coming from.
[Also, in case there's any confusion please note the distinction between "native", meaning it is compiled to native code; and "React Native", which is a cross-platform framework that is not compiled to native code. "React Native" is a horrible name for something that is not native.]
Thing is, all these frameworks are leaky, to debug anything on them, or to add missing functionally, that small team needs to be an expert across two native frameworks, and the hybrid/abstraction framework that they are betting on.
Yes and no, I think that depends how deep you want to go. For your standard "show and update some content and make it look pretty" type app, I can tell you from both my own and others experience you don't really need much familiarity with the native platforms, other than how to use build tools etc. which are well detailed online.
If you want to go deeper or have more specialised requirements, then you might need a deeper understanding of the native platforms though.
Beyond "Hello World" all of them required me to get my hands dirty, at which point I was better off just doing C++ with native views, standard out of the box tooling available across all SDKs.
Nowadays I rather go for PWAs, or plain mobile Web, unless it is something very special that requires OS APIs or hardware access.
I've worked power-user type applications that are mainly keyboard driven, and those are pretty easy to make cross platform with a single custom codebase that has a platform-specific graphics backend (OpenGL covers many platforms) and minimal platform code. Let's say < 300 lines of boilerplate per platform + 600 lines for OpenGL + 2-5K lines for (shared) UI controls.
React Native (or any framework) isn’t good when you can literally afford billions of dollars of R&D given the financial value of the app. Just like the Mars rovers don’t run Python but it is a great language for a experimental SaaS app testing a new market, RN is great for a team of 3 engineers to get a multi platform app shipped, but not a great choice for an app worth ~$100 billion to its parent company.
Interestingly, there was news on some satellite running javascript. But I wonder if the core was running something else, and peripherals were run with js -- and simply restarted when they died, Erlang style.
Man, I hope so. There has beens so much cargo cult built up around React Native that suggesting other approaches is almost guaranteed to kick off many rounds of contentious design meetings.
Fortunately, nothing quite takes the wind out of a platform's sails like watching the creator back away from the platform.
(Some of the comments in this thread, like this one, were originally posted to https://news.ycombinator.com/item?id=22467769, which explains e.g. why they mention the URL explicitly.)
There's no mention either way. The article talks about a single C library and C extensions. And it mentions using native OS functionality. Those could be used from javascript or any language.
When I interned at FB in 2018, there was no RN in the Messenger codebase. However, it did make extensive use of various FB UI abstractions like Yoga layout and ComponentKit.
I'm thinking maybe they moved off of React Native. RN makes sense for a small org or for an app you're just getting started with, but a megacorp like FB can afford to hand-craft a smaller, tighter app for each platform.
Given the headaches caused by Apple finally barring UIWebView (the basis of frameworks like Cordova) from new App Store submissions beginning in April, I'm not surprised FB would do this. There is plenty of rumor-mongering that Apple is going to make it increasingly difficult for apps based on hybrid frameworks through the submission process.
UIWebView was deprecated for years, everyone's had plenty of time to move to WKWebView, and it has nothing to do with React Native to boot.
Apple legitimately doesn't seem to care about hybrid frameworks, they've been there since day one. Any time it's seemed like it got difficult Apple has backed down. The lack of JIT in WKWebView is probably the only thing that you could stretch here.
React-native does not run in a webview. The JS binds directly to native elements. In fact, you can easily wrap any native code and make JS bindings to call it directly.
"We started with the premise that Messenger needed to be a simple, lightweight utility. Some apps are immersive (video streaming, gaming); people spend hours using them. Those apps take up a lot of storage space, battery time, etc., and the trade-off makes sense. But messages are just tiny snippets of text that take less than a second to send. Fundamentally, a messaging app should be one of the smallest, lightest-weight apps on your phone. With that principle in mind, we began looking at the right way to make our iOS app significantly smaller."
This paragraph just restored my faith in developer sanity.
> We extended SQLite with the capability of stored procedures, allowing Messenger feature developers to write portable, database-oriented business logic
I wish the article was a little more fleshed out, I'd be interested in finding out more about how and why they went down this path
I just listened to the Changelog's (a podcast) episode with Richard Hipp, the author of SQLite -- I wonder if he had any part on this extension. But in any case, great to see SQLite thrive.
Keep seeing react native pop up in the comments, there has never been any indication that Messenger used any react native from fb feom all their presentations I have see . RN is used in the Marketplace tab on the Facebook App, the Ad manager app and a few other places.
The architecture they have ended up with is similar to that described in Out of the Tar Pit, which describes how to remove as much complexity from software as possible.
By moving all state into SQLite they have implemented the "store state in relations" layer, and then written (as far as possible) functional code on top.
I've been thinking about how to make my software more like this so it's interesting to see Facebook are thinking along the same lines.
There's a lot of really good ideas in Out of the Tar Pit and I'd really recommend reading it.
Considering that they were shipping 18,000 classes in their iOS app back in 2015, a quarter the size of their 2019 app has to be still unreasonably massive.
Last I heard, Facebook was running into the limits of the Objective-C runtime because they'd register a bunch of classes at startup and overshoot their launch time. They factored a bunch of stuff out into dynamically loaded frameworks off the startup path or something like that.
I don't know anyone who uses Messenger, yet, I've heard of people moving to specific countries where it's the de facto and you can't have a social life without it.
I'm also aware of how iMessages and SMS are used in the US, yet this is very US-specific, and I don't believe any other country uses SMS.
For Chinese people, WeChat is the standard, and what pretty much anyone uses.
In Argentina and the Netherlands (and I believe much of the EU), WhatsApp is ubiquitous.
Yet, I don't know a single person who uses either SMS or Messenger, or iMessages.
So what truly fascinates me, is how different companies put so much work into developing such sophisticated messaging apps, but they're all reinventing the wheel and each consumed in different regions -- I think nobody would have guesses years ago that we'd have so many different apps used in different countries, and not at all inter-operating with each other.
Did you ever use AIM, ICQ, MSN, Y! Messenger? I'm quite certain that in a decade, there will be another batch of dominant messengers "reinventing the wheel" (IG & TikTok I'm looking at you). Real-time Communication is just such a crucial part of people's lives that people will never stop working on it, even if it feels like it's been solved.
A lot of it is driven by differing low end data plans. In some countries even the cheapest tariff has adequate data allowance, in other countries, data is an expensive add-on.
So it's entirely conceivable that in some places, someone with a limited income only has SMS bundled as a free service on their plan, with data either not enabled, or expensive pay-as-you-go. Therefore in that demographic SMS messaging will be the primary method.
In the UK, almost all tariffs come with a bundle of data, so SMS has all but died out (except for automated parcel delivery updates, and 2FA authentication et al.).
I'm in the US. SMS / Apple Messages seems to be the default for everyone I interact with. I use Messenger sometimes, but usually only when someone else initiates.
My favorite is using Apple Messages since I can send/receive from any of my Apple devices. I use macOS for other reasons, but being able to send a text from my computer is awesome.
This is an interesting business decision. To rewrite such a large app must have cost a fortune. Going full native is probably going to increase the cost of future development as well. Aside from the costs of writing/maintaining the code, pushing more work to the servers is an ongoing cost. Realtime, always-connected, apps are costly to run even when you push the majority of the state management to the client and just route packets around. They must really want eyeballs in their messenger app! There must be some future plans for this app that warranted such a major effort. Taking on Tik-Tok, perhaps?
> To rewrite such a large app must have cost a fortune. Going full native is probably going to increase the cost of future development as well.
it's a freakin chat app, and it's freaking facebook ! come on, people were doing full rewrites in entirely different languages of software insanely more complex than fb messenger back in the 90s and with near-zero tooling, eg from cobol to pascal to C or similar horrors.
I've built a few "chat apps". I wouldn't discount the complexity. They carry with them a wide array of problem spaces. And of course anything at FB scale user counts is going to be a challenge.
While I have absolutely no doubt there are scalability problems, it seems to me that most of those problems are going to be (or should be) on the back end, not in the client. How is the client 1.7 million lines of code?
I can see it. It's pretty easy to hit that much code when you include your own code for things as trivial as a JSON parser rather than leveraging what the native SDK provides (which was alluded to in the article). It sounds like they had a bit of Not Invented Here syndrome going on and this rewrite addressed a lot of that. It also sounds like they had a bunch of systems (probably different teams even) doing similar tasks like synchronizing state. And they also had their own UI framework. All that stuff adds up to a lot of code. But, all that being said, it still has a ton of features. It's not your father's IRC client sending text around and failing to transfer files through NAT/firewalls.
> But, all that being said, it still has a ton of features. It's not your father's IRC client sending text around and failing to transfer files through NAT/firewalls.
Meh. It displays text, pictures, and videos. That's about it. This is just Messenger we're talking about here, not the full-blown Facebook app. The only state to keep track of is "Have you received message UUID?".
> And they also had their own UI framework.
This is where I suspect the bloat occurred.
Writing your own JSON parser and so on doesn't add up to any 1.7 million lines of code.
Well, browsers just display pictures and text, yet to build Chrome one needs to download gigabytes of code. Databases just keep an ordered list of records on disc. Compilers just look at the code and produce another code. Web servers only need to accept connections and respond with some data.
Image editors are the simplest — they just let you paint with colored pixels.
For android you have to ship basically ffmpeg for correct mp4 encoding, on all platforms you have to carry some library on top of classical ui framework to make it just not suck at performance, you have to get your own json parser, yes, since built-in one will be 10x slower, you may be want to carry even low level QUIC library for networking and websockets, etc, etc.
All this also not increasing complexity since built in one stuff makes your code much harder to read/write.
> For android you have to ship basically ffmpeg for correct mp4 encoding
hopefully they aren't counting ffmpeg in their LOC count, right ? else they may as well count the lines of code of the C and C++ standard library, V8, etc - this would be a very useless metric
Just downloaded the latest update on my iPhone XR. Does seem a lot more snappy and I'm loving the recent UI/UX changes too, especially the sending photos & GIF changes.
Not sure how much impact this will have, most of the people I know and have discussed this with fall into two camps: people who use FB a lot an will put up with a bad mobile client, and people who don't want FB on their phone, ever.
I have to go years back to think of someone who told me they just couldn't take the bad UX of the clients so they deleted it.
It's probably not meant for us, but instead the third world phones where they've gotten to the point that it might not even install on a large part of their user base.
Like I've heard from peace corps folk about how a large part of the third world thinks of messenger as "the internet", which is absolutely terrifying.
I've tried FB lite, and found it to be slower than the real app. FB lite seems to act like a fancy web browser, so each interaction needs to download it's UI along with content.
For a while, before it really started to bloat, FB Messenger was the slickest, most well-rounded, most enjoyable messaging client I could find, and had the advantage that all of your friends were already on it. The bloating from Stories and everything happened to coincide with my ejection of all things Facebook from my life, but Messenger was a genuinely good product at one point. Sounds like it may be returning to that (though unfortunately that ship has sailed for other reasons).
I don't think that just because people don't complain about it it has no effect.
re app size : Google has already published stats urging apps to optimize their size as much as possible since it has direct effects on acquisition.
I suspect that long download times lead to more errors + more wait time before you can use the app = more likely that the user will abandon its current task and
Same with slow app or ui patterns that don't fit with the platform = easier to lose users in the first crucial minutes.
Here's a way in which they can save 100% of the size: Stop stubbornly barring mobile clients from using the web interface. This is why I don't use Facebook Messenger at all.
Still no. If an iOS app is using your microphone while backgrounded, your phone's taskbar at the top of the screen will be red. iOS does not allow apps to use the microphone while the screen is off. The microphone thing is a myth fueled by paranoia and confirmation bias. FB already has plenty of data on you and doesn't need to run NLP on everything you say in order to show you relevant ads.
Well I have messenger lite on Android and they, as the parent says, "ask permission to listen to your microphone". Presumably they do voice chat, which needs that permission, surely.
For photos, I wish iOS would not require a permission at all in the case where I just want it to open an OS-controlled file picker and then give the app the one photo, akin to copy-pasting it. No reason to give access to every photo when only N are picked.
It’s been a while since I worked with the photo library APIs in iOS, but as I recall there in fact is an API that apps can use to request the user to pick individual photos from an OS picker without further permissions. Most apps just don’t use it because they all want to do their own custom photo picker UIs (even privacy focused apps fall victim to this).
I used mbasic.facebook.com for a while when it was not possible to use it from the regular mobile web. But for like a year or something they appear to have turned the web back on (at least for firefox, haven't tested anything else).
That’s a workaround for a on purpose decision to cripple the mobile website, where the chat and messages worked flawlessly. Another decision against users was to use messages as notification in some cases.
I didn’t pull the trigger on canceling the account as from time to time I find news and event from pages that I follow, but there’s no way I’m installing an app that uses so many shady tactics.
Sounds neat, but I’ll continue to use “request desktop website” on mobile safari in a vain attempt to prevent running whatever else this app does besides chat.
This seems like an incredibly low bar to hurdle. FB Messenger on iOS is legendarily bloated. In the couple of weeks I had it installed it also managed to be the biggest battery drain on my phone, even more than some 3D games.
I think they went through a period (or still are, I honestly moved over to messenger lite a couple months into it) where they were seemingly intent on making messenger the WeChat of the west. It got real chunky real quick. It was around the same time everyone was excited about conversational ui and Microsoft announced their bot framework.
It may have been slow and bulky before that, but that's when I started to notice.
They were trying to make it the WeChat of the non china east and africa, not the west. And they've been pretty successful at that, "messenger" is almost synonymous with "Internet" in some parts.
I wasn't aware, but honestly quite happy they succeeded. A lot of the extra features they were adding seemed pretty out of place for my local market, but they definitely put a lot into it.
definitely. Facebook is the internet in my home country. But still lots of Messenger features were not used. So still bloated. Not that anybody complained or moved away from it.
Have you seen how much memory and CPU time Signal uses on the Desktop? Or Slack? The era of running a chat app for almost no resources is gone unless you're an IRC holdout.
i use telegram web and it is awesome. Messenger web just sucks, i don't why actually, maybe it is because of the design. I have not checked the others (signal, wechat).
tl;dr web clients are probably more performant than their electron counterpart
It doesn't look like, though. I refused to use the desktop Telegram for long because it looked like just another Electron app, until one day a friend enlightened me that it's actually a lean(er) QT app.
Consider two competing apps. One looks 5% nicer but uses 1000% more memory, cpu and battery. That one will be preferred by users. And this is why things are how they are.
If you know exactly how you want your app to look forever, sure. But you don't. So you want a flexible app that can easily be redesigned. And that means prioritizing developer comfort very high. Which means bloated frameworks.
This is a bit cynical (and borderline paranoid) but what if the reason for the rewrite was to remove privacy-violating code that was becoming a greater and greater liability? Also, wonder if they needed to do this to make way for end-to-end encryption we've been hearing about?
Depends, if they allow WhatsApp and Facebook to intercommunicate through it, then they could get 100M people using it by next week.
I'm not quite sure why I need two apps on my phone for communications through one company when 12 years ago I had one app on my desktop for 5 or 6 different company's messaging services.
off topic, but I absolutely hate the millennial put-down of 'ok boomer'. They use it for anyone who is older than them, regardless. I'm Gen-Z and being called a 'boomer' is offensive.
To me a lot of the new architecture just seems obvious (don't keep the same view 40 times, don't invent your custom app specific everything ...) and I really wonder how the original project could derail that far. Too many resources? Institutional bloat?
That said I find it a bit heart warming that all the major apps are now between 1/2 and 1/4 the size of what they were just a few years ago. I didn't expect the tech companies to ever realize the problem with insane sizes. Is a similar debloating happening to Electron apps?
I wonder what this means for older versions of Messenger? Facebook talks about rewriting their server logic to support all the removed code from the mobile interface, but does this mean that we will soon see support for < iOS 10 (which Messenger does not support for new versions, as far as I ma aware) be dropped?
It's pretty sad that a Messaging app can't be run on a perfectly fine nine year old device. Bring back IRC :)
I feel like I had an aneurysm reading this. For 6 paragraphs the repeat that they shrunk the lines of code down, that the rewrote the whole thing and that it’s a massive undertaking and that a message app should be small. 6 paragraphs of that slightly reworded and I just gave up. I honestly think this was written by a bot.
I don't know if it's clear it will suck? You just advertised their company, it's the first time I ever heard of them. It's not like LightSpeed is an unheard of name? It's a stock trader, web filter, data warehousing company, and probably a million other things. Novel names are hard to come by.
Wish they had gone into more details about the budgets. Unless a project is bug fix only for life, how can one set a binary size or lines of code upper limit? I've heard that the best developers delete more than they write, but in practice I don't think that happens very often.
I really wonder if in their meetings they sold the speed and lighter weight as a bonus for the user or for the collection of data on the user. This must reduce their overhead and be kinder on their servers.
I still aren't 100% sure what they're using Sqlite for? Are they storing templates as blobs? Are the storing template logic in the table? How did that make things faster? Anybody who has more detail care to explain?
I think some things do not show up on the mobile pages - reactions and polls, I believe - maybe stickers? I'd also have to check if replies show up correctly
Now if only they would rewrite the web application too, or release that desktop app they were talking about last year. The web app is much, much slower than the phone app.
> And yet, for the people using the app, it won’t look or feel much different.
Yikes. Okay, I know I'm maybe being a bit unfair here, but I'm sorry to say that this is one of the weakest engineering blog posts I've ever read. This seems to be a massive "let's end the craziness and finally pay our tech debt" effort. Good for them, but not something that excites me about working at Facebook.
I'm truly shocked how many resources are dedicated to maintaining a chat app that essentially just talks to an api. Seems like you could easily build and maintain a simple chat system with <5 developers. You wouldn't have shiny new features, but it would by coincidence be faster and simpler. Networked chat is a standard project in university.
> Seems like you could easily build and maintain a simple chat system with <5 developers
Yes you could. Note however 'simple chat system'.
To build a global messaging app used by hundreds of millions of people is a whole different ball game.
By comparison, before acquisition by Facebook, Whatsapp had a team of ~50 people, and that was considered lean. Based on that, it would seem you're about an order of magnitude off.
True but, be aware that those 50 people did everything including the servers, the operations, the BI, oh and also maintained clients for like 8 different mobile platforms including ultra constrained ones like j2me and blackberry. If you look just at the iOS client, 5 people at WhatsApp on it would probably be a bit too high.
The issue here is only partly inherent complexity of the feature set. The real problem Facebook had is one I don't see discussed in either the blog post or this thread, which was organizational. The way they structured their mobile apps was as a huge shared codebase with many disparate teams just checking code into it whenever they wanted. To the extent there was any architectural planning at all it came from some small shared library teams who were in no position, managerially, to enforce their will on the others. The blog post makes clear that a huge portion of their problem was multiple teams inventing their own ways of doing things and creating stuff that want necessary, along with a lot of pointless duplication because there was so little coordination and thus so little code reuse.
The new app doesn't sound like anything special design wise, and that's the point: for the first time they've done things more conventionally. They now have someone who can dictate to every feature team "thou shalt use sqlite with these schemas" and other rules.
The technical meat of this post is below. It looks like they mostly eliminated functionality in their app code wherever it was redundant with either OS functionality, SQL Lite capabilities, or the Messenger backend servers at Facebook's datacenters.
>One of our main goals was to minimize code complexity and eliminate redundancies. ... To build this unified architecture, we established four principles: Use the OS, reuse the UI, leverage the SQLite database, and push to the server.
>We accomplished this by using the native OS wherever possible, reusing the UI with dynamic templates powered by SQLite, using SQLite as a universal system, and building a server broker to operate as a universal gateway between Messenger and its server features.
>... the existing OS often does much of what’s needed. Actions like rendering, transcoding, threading, and logging [and JSON processing] can all be handled by the OS.
>To simplify and remove redundancies, we constrained the design to force the reuse of the same [UI] structure for different [UI] views. So we needed only a few categories of basic [UI] views, and those could be driven by different SQLite tables.
>Now, ... All the caching, filtering, transactions, and queries are all done in SQLite. The UI merely reflects the tables in the database.
>We developed a single integrated schema for all features. We extended SQLite with the capability of stored procedures, allowing Messenger feature developers to write portable, database-oriented business logic, and finally, we built a platform (MSYS) to orchestrate all access to the database, including queued changes, deferred or retriable tasks, and for data sync support.
>MSYS is a cross-platform library built in C that operates all the primitives we need. ... With MSYS, we have a global view. We’re able to prioritize workloads. Say the task to load your message list should be a higher priority than the task to update whether somebody read a message in a thread from a few days ago; we can move the priority task up in the queue.
>With MSYS, it’s easier to track performance, spot regressions, and fix bugs across all these features at once. In addition, we made this important part of the system exceptionally robust by investing in automated tests, resulting in a (very rare in the industry) 100 percent line code coverage of MSYS logic.
>For anything that doesn’t fit into one of the categories above, we push it to the server instead. We had to build new server infrastructure to support the presence of MSYS’s single integrated data and sync layer on the client.
>Coordinating logic between client and server is very complex and can be error-prone — even more so as the number of features grows. ...
>Similar to MSYS on the client, we built a server broker to support all these scenarios while the actual server back-end infrastructure supports the features.
>[To minimize code-base growth] We also built a system that allows us to understand how much binary weight each feature is bringing in. We hold engineers accountable for hitting their budgets as part of feature acceptance criteria. Completing features on time is important, but hitting quality targets (including but not limited to binary size budgets) is even more important.
Am I the only one who thinks 86mb for an app is about 86 times too much?
Windows Messenger was about 900k. Sure, it only did text, but Skype wasn't that big either. I'd accept 20mb for stickers, givs and stuff but 80? And 300k lines of code? For a messaging app? It just seems... unreasonable to me.
