We also get the benefits of a mono-repo and the benefits of microservices in the same application footprint, because every controller method is automatically deployed as its own independent lambda (this is core to how Jets works), but we're still in the usual Rails mono-repo format we all know and love. Also very strong integration between ApplicationJob and background lambdas has been killer for us.

One thing I've always said is the real difficulties in software development happen at the seams where services connect with other services. This sort of strategy (and particularly, the mono-repo format) minimizes the number of seams within your application, while still giving you the scalability and other benefits of microservices and serverless.

From a business standpoint, that's a pretty great pitch.

Is it actually the most cost effective solution? No more so than any other tool. It depends on exactly what you're building, how, and how you measure the cost. AWS can be extremely costly, or cheap, depending on your engineering needs, constraints, and practices.

The companies I have been at where computing costs did matter were doing extremely specialized, long-running calculations that are inappropriate for lambda.

I'm sure there are rare exceptions, but this all smells like premature optimization. Developer salaries are, 9.99 times out of ten, the cost you want to optimize.

For reference, 90% of our bill is database related currently.

Huh. With the context of the rest of the comment, I realize (the very obvious comparison) that a database engine designed to shard to many thousands of small workers could potentially be a very attractive future development path.

Iff the current trends in cloud computing (workers, lambda, etc) continues and some other fundamental doesn't come along and overtake.

Which is probably (part of) the reason why this doesn't exist, since I think I've basically just described the P=NP of storage engineering :)

Yes. I've been patiently waiting for the database community to realize this for the last 5 years now :)

The optimal solution would of course be to shard both the compute I/O and the storage footprint, so each worker only needed to hold onto maybe 1-100MB of data.

Perhaps some existing (simpler) designs could be modified to "hyper-shard" the compute angle, but would still likely require carrying around a large percentage of the database.

In any case, you'd need an internal signalling fabric capable of (cost-effectively) handling very bursty many-to-many I/O across thousands of endpoints to make consensus work in realtime.

It would honestly be really interesting to see how something like this would work in practice.

Serverless as an architectural pattern is about more than just Lambda and tends to incorporate a multitude of managed services to remove the cost of managment and configuration overhead. When you use IaC tools like Serverless Framework that are built to help developers put together an application as opposed to provisioning resources, it means you can get things up fast and ready to bill you only for usage and that scales amazingly.

How do you resolve this in your projects? (Serious question).

This is such a big problem for some of the projects that they are now only able to hire senior develops (which brings it‘s own set of problems).

There's not getting around the networking and such - that's the full part of full-stack (FS) - it's more than simply FE+BE. Maybe call them distributed systems engineers instead.

What it sounds like though, is your organization (regardless of what we call it) is large enough to organize into FE, BE, and FS roles, with FS running the platform and being in charge of the fleet and for them to work on the system itself so that FE and BE can work without having to know about fleet - and FS folk are building internal tools that the rest of the org use to do their job, and to shield them from any of the implementation details your fleet has.

All of that said, I vastly prefer google cloud functions personally and would switch to that in a heartbeat if they had capabilities like API gateway but it's not there yet.

I also regret that there isn't a better cross-cloud solution currently, but that's something I have a lot of open source ambitions about creating soon. I don't like serverless that much so stay tuned for something from me in the coming years, probably in Rust.

Lambda also supports Docker and is pretty damn fast now, so it's less painful to use.

1. You have a maximum of 30 seconds to reply to the request

2. You cannot stream data from lambda through the api gateway. Though you can stream to s3 etc.

I agree streaming needs work, but if I had to deal with that I'd probably just use naked endpoints.

If you're talking about web sockets, yeah, I feel ya. I haven't had to navigate that situation yet but my plan of action is to bypass API gateway entirely and use a naked lambda running at its full execution time, and when we approach the 15 minute boundary, send a command that resets the connection. Don't know if it would work well but I haven't had a need to use web sockets for any projects I work on for a while.

For streaming, like you said, maybe an elastic beanstalk cluster is more the way to go depending on your workflow. If you can find a way to get it all to work in Lambda though, would probably be a game changer cost-wise I would expect as long as you figure out a way to deal with resetting every 15 mins.

I migrated some mid business (500-1000 people company) lambda setup to ec2 spot instances and a standard app and the costs dropped.

It would have been even cheaper on something like heztner, but good luck getting a buy-in from the infra team.

I venture most small businesses will have less load than them. Certainly it's not worth it for my tiny side businesses.

I was running some calculations for a project though, and if you "abuse" them with lengthy / expensive calculations run very infrequently (think like a cron job), you may end up being quite cost effective.

While we did have to spend some $$ to get things up to snuff (particularly so we could pass SOC 2), this pales in comparison to how much we would have had to spend to do devops and the usual server wrangling to get our use-case up and running in a typical elastic beanstalk sort of situation.

One area of difficulty was Oauth so if you ever need help implementing that in Jets feel free to reach out or read the public issue history of how we got it working.

Is there a blog post about the tech stack? My main concern with serverless/lambda is cold start time. How do you deal with it? What does the p99 latency look like?

Also how do you scale the usual bottleneck which is the database?

The author addressed the cold start problem here(timestamped) https://youtu.be/a0VKbrgzKso?t=439

Hope it helps :)

More generally though, all of these "newsql" offerings feel a little too good to be true for me, I can't see how you could really have all the relational integrity of postgres with the elastic scalability of a distributed DB without trading something off. Am I too cynical?

1. Elixir + Phoenix

2. TailwindCSS

Elixir because you can build a monolith with it and everything is naturally separated by design. The runtime makes it impossible to hurt yourself. It addresses virtually every web use case in an ideal fashion that balances development speed with long term maintenance and scalability because of the language design trade offs.

TailwindCSS because it removed the decision about which of the million frontend stacks in should be using. It makes it so easy to do anything I can dream up without feeling like I need to hire somebody to help with a pet project.

And honestly, even though I like DevOps and database work just using Heroku/Fly/Gigalixir so I don’t have to worry about it makes my life easier.

But I feel no typesystem would bite me in bigger project.

  def foo(mymap):
     blah

  public static HashMap<String,Integer> foo(HashMap<String,Integer> mymap){
    return mymap
  }

  def foo(%{"username" => username, "password" => password}) do
     {username,password}
  end

    def foo(login_data):
       match login_data:
          case {"username": "foo", "password": "blah"}:
              return True
          case _:
              return False

    def foo(login_data: dict[str, str]) -> bool:
        match login_data:
          case {"username": "foo", "password": "blah"}:
              return True
          case _:
              return False

IMO it's a disservice to compare Elixir's typing to languages like Python, and we need a new category to distinguish the two.

I tip my hat though to people diving into Elixir --- it is definitely a cool direction and one I would follow if I weren't already so sold on Rust/Crystal/etc.

personally, I still enjoy the extreme dynamism of lisp and lisp-like langs (julia), but we may never see a resurgence of them in popularity

I can’t think of another language, even Rust, where you could run a full database inside your codebase without negatively impacting everything else.

That BEAM runtime just makes things possible that aren’t in other languages.

In particular there's a theme of JS being totally optional: "Rails 7 takes advantage of all of these advances to deliver a no-Node default approach to the front end – without sacrificing neither access to npm packages nor modern JavaScript in the process." "The same approach is taken with CSS bundlers that rely on Node. [...] If you’re willing to accept the complexity of a Node dependency, you can pre-configure your new Rails app to use any of them"

That seems like a smart goal. They're not trying to go it alone, but they're clearly trying to draw some boundaries that keep the JS chaos/complexity at the edges.

If you want to learn it I think it's at a mature enough place.

Popular libraries like redux have been rebuilt to use hooks and simplify the integration.

I'd also check out Remix[0] if you wanted to get into a React framework. It's fairly new but extremely promising and easy to get up and running and even deployed anywhere (express server, cloud flare workers, deno, etc)

[0] remix.run

Now if only the React ecosystem settled on standards too, that would be wonderful.

But alas. There is flux, redux, mobx, hooks, routes, sagas, thunk, observable, styled components, emotion, tailwind, react-antd, axios, fetch and so on and so forth. Edit: on top of, obviously, webpacker, grunt, npm, yarn etc.

Contrary to e.g. Rails, none of these come with React. You'll need to organize it all yourself (or go with something as react-boilerplate). You'll need at least some of these pieces to have something workable very early on. Things like redux or saga are not some "we've grown out of vanilla React and need additional tooling", they are essential to do things that practically every app needs: pages, communication with a backend, some styling, some consistency and deploying it to a public server.

When I had to learn a little ruby on rails I found the convention over configuration far harder to wrap my head around. It's quite nice to be able to see for yourself exactly how pieces are wired up and configured instead of having it magically done for you and dictated to you that it has to be done this way.

There are standards in some of what you mentioned. Mainly hooks, fetch, npm, and css.

Ruby doesn't "solve" css either, you're just using regular css files which you can do trivially in React as well (<Foo style={myStyles}/> and define myStyles as an plain old javascript object wherever you want - same file or not) or just include the stylesheet on the server response.

The rest of it, state management, webpack vs grunt, etc is pretty simple to select from and even in all that there are obvious choices - redux, webpack, npm that are certainly considered standard.

There isn't much magic in rails, and just stepping through the methods in a debugger makes everything pretty clear if you do want to know exactly how everything is wired up.

That was my entire point, yes.

The rest is nasty, nasty bloat. Saga in particular is one of the most pointless and libraries I've ever seen. Total solution looking for a problem.

My point, however, was that this is not obvious for anyone starting with React.

This is caused by the fact that React is "no batteries included", which means that you have to find the right batteries at a moment when you lack all knowledge about batteries in the first place. You search "how to send to backend" and saga's pop up, explaining why they are better than useEffect or redux or whatever (at a moment you may not even know useEffect or its downsides).

Compare that to Rails which has "all batteries included" (and which is a nightmare in itself, too, though), where all those choices are made for you. You can choose to ignore Rails' testing suite and instead erect an rspec setup next to it, but you'll do so consiously. Because that moment when you asked yourself "how do I test in rails" the One True Way was there, configured, ready for use and documented.

Both have tradeoffs and pros and cons. But the React community (with tutorials, this weeks best practices, breakspeed iterations of tooling) is not helping here. At all.

There are some default choices in Rails that I disagree with. E.g., I think it's too database-focused, and I build things that often aren't. But I still know I can get started with Rails and expect it all to just work. I know that people with Rails experience will be able to jump in without much pain. And then if I depart from the standard path, I have a good sense of where I might get into trouble.

There are some very clever things you can do with them, but this really need to know all the ins and outs of it first, and by the time you've learned them you implemented a giant nightmare.

Where sagas _do_ still make sense is highly complex async workflows, including responding to dispatched actions, "background thread"-type behavior, and lots of debouncing/throttling/etc.

But yes, I've heard of plenty of cases where sagas made a codebase unreadable, and it's a shame that they get so heavily pushed by some early Redux users.

FWIW, I've actually been working on designing a new "action listener middleware" that we'd like to ship in an upcoming version of Redux Toolkit. It started off as very simple callbacks, but by adding a few key primitive functions like `take`, `condition`, and `delay` I think we've been able to to come up with something that can handle maybe 75% of what sagas can do with a much smaller API surface and bundle size. I'd love to have you or anyone else using Redux take a look and give us some feedback on the current API design and let us know if there's other use cases it ought to cover:

https://github.com/reduxjs/redux-toolkit/discussions/1648

It was never clear to me why/when I would need React. I read and worked through some tutorials years ago, and while the reactive/one way flow pattern was nice, I never had a problem with Angular.

I recognize there have been many versions of Angular since then. And there are several web development frameworks, complex build tooling, etc. etc.

Meanwhile, I’m still writing services in Java that handle 20,000 requests per second for a service that brings in revenue in the billions of dollars a year. And I’m writing C++ (very straightforward, no templates, few uses of pointers, etc) for cross platform mobile code. I have some Spark pipelines all written in Scala.

Meanwhile the web stack continues to evolve… new technologies all just for rendering web pages. I don’t understand why. Admittedly there’s far more complexity in our use cases today than the HTML I was writing in 1999, but much of it is unnecessary bloat from bored developers.

But most of the times it's a pain, because of it not scaling well to larger projects. I've actually seen projects where controllers grow to the length of thousands of lines because the developers didn't feel comfortable with introducing new ones in the pre-existing codebase, due to how the scoping works, due to how they'd need to set up the data models and care about initialization of everything, as well as custom integrations with validation libraries and other JS cruft (due to it not quite being "battieries included", like the new Angular is).

Now personally, i really liked how they did error messages (which gave you URLs that you could navigate to, with a bit of context in them; though it would be better with a locally hosted docs server), but a lot of things were awkward, such as their binding syntax: https://blog.krawaller.se/posts/dissecting-bindings-in-angul...

Luckily, AngularJS seems to finally be dead, so we'll get to migrate to something else soon, hopefully. And the projects that won't will be red flags enough for the people who don't like such challenges to turn the other way - truly a litmus test of sorts.

Oh god, I'm so old.

in particular, someone said react is better for building complex systems. do you agree with this opinion?

if you want to try something different I'd go with svelte

which is better for building complex systems?

It's more important to pick one and just get started with it. You can do some reading about how they're different or their different philosophies but they all work.

It's more important to get started. If you want to build something to learn a skill to get a job, react is the most popular. If you want to build something to learn something new, any of them are a good choice. Just get started.

What's optional is whether you need to use node or npm itself as a developer-facing tool, and needing ESM support in the browser for the way they've done it means it's only relatively recently that that's been practical.

Recently I switched back to Rails after 10 years. I can't say I enjoy the whole asset pipeline business but I yearn for a simpler time and Rails gives that (to a degree).

My only wish it was simpler to host. I don't necessary want to buy into render.com or heroku.

Of course the larger your project is, the more hassle, but that's not specific to Rails.

It is not simple. Python or Node.Js suffer the same issues, though.

Compared to dropping a go-binary or rust build somewhere and firing that up, Rails/Ruby (Rack, Sintra, Rails etc) are a mess.

In no particular order, the issues I had to deal with in the last three months:

* asset pipelines crashed randomly; turned out the ~2GB build VPS had too little memory to run the asset pipeline.

* puma and unicorn and phusion cannot be ran and configgured easily on one server.

* rbenv builds are hard to manage.

* getting the correct gems on your server is a mess. There is bundler, rbenv, vendored, rubygems.org, gem, Gemfile, Gemfile.lock, ruby-version in gemfile, ruby version .ruby-version, all of which can -and will- conflict at some point.

Add to that, that a typical Rails app requires at least redis, postgresql/mysql, quite some lib-x headers to build native gems, and imagemagick etc, and you have a very complex setup.

This isn't more complex than your typical Django, Laravel or ExpressJs setup. But it is far more complex than deploying a go, rust, or Java bundle.

Some things I'd note-- * Don't use puma or unicorn. Phusion Passenger and Apache is just fine performance-wise and is much more robust IMO. * To avoid ruby version conflicts use .ruby_version as your sole source of versioning. I use this in all my Gemfiles: `ruby File.read('.ruby-version').strip` * RVM isn't as popular as rbenv but it's amazing and you won't run into the issues you mentioned. It integrates with Capistrano and Passenger+Apache really well. * 2GB might be a bit light for a production app even with light load. We start our new VMs at 8GB and then adjust up or down as makes sense.

We still do a single Ubuntu LTS VM for each app rather than anything fancy. Once you've added the stuff you need via apt it just works. Our sysops folks use Ansible playbooks that automate everything so there's no mystery when we need to do an OS upgrade or spin up a new VM.

And please don't try to tell us that a real-world Java web application is simpler than, well, anything. ;-)

Execjs, libreadline, libsqlite3, lib-postgres, (older)nokogiri etc. Quite a lot of the headers keep being a problem. Not major, often a mere ddg+apt-get away, but still annoying. It does get problematic when you have multiple apps depend on different libreadlines, for example.

And I'm not often deploying java, but at least the runtime and tooling has become omnipresent. The latest deploys were a mere "move all files there, and restart the service". With Rails you at least need to rebuild the gems.

Often java-services are an `apt get jenkins` away. There is no equivalent apt-get gitlab or apt-get mastodon that just sets up the stack entirely. But that is probably due to their popularity more than technology.

Tongue in cheek, but honestly, that's probably debatable.

Under the hood, Java apps (at least in my experience) are Eldritch horrors with hundreds of beans/proxies/servlets/God knows what, needless layers upon layers of abstractions and dangerous amounts of reflection and dynamic behavior, all to launch and initialize a web application in a "simple" manner. I've always seen the exact same horror, be it working with Spring Boot, Spring, JavaEE, Struts, or even something like Dropwizard (though it seemed a bit more sane). Only the microframeworks seemed decent in comparison, something like Vert.X (has a different paradigm, though), or Quarkus/Helidon.

But when it comes to running them... well, historically they were still a nightmare, if packaged with .war and relying on certain application servers/JDK versions being present. But packaging them as executable .jar files (think Spring Boot) makes them similarly easy to Go, at least as long as you have the JDK version that you need. You just drop the file in a folder and if you have some configuration (which your Go app would also need in one way or another), you can probably launch it.

> I have never had the sort of problems you say you're having trying to keep gems sorted.

Ruby suffers from the same problem that Python, Node, PHP and other non-statically compiled technologies suffer from - messing around with dependencies. If i develop on Ruby locally against version X.Y.Z, but only X.Y.W is available on the server due to Debian packaging older versions, then i'll run into problems because of the project refusing to build/run. I've also run into situations where building the native stuff (DB drivers in this case) will fail, for example, when libpq-fe.h headers were missing and pg couldn't been installed, so the gem native extension couldn't build. Also, on Windows, Ruby 2.7 downloaded the sqlite gem with 3 different trojans (Win64.Alien.eg, Win64.Alien.ef, Win32.Agent.xahigh) in the extensions (btreeinfo.dll, memstat.dll, memvfs.dll), as picked up by Kaspersky. No idea how that happened, or whether that could have been a false positive, but i didn't appreciate that much either.

That said, i have a folder that has about 112 images of all sorts of software breaking in various ways to date, and the number is only so low because i don't screenshot things on my work computer and not even every small instance of something breaking. In my experience, all of the technologies out there are bad in some ways, it's just about identifying and managing these tradeoffs towards whatever is suitable for your circumstances.

Just my 2 cents

Either you use RVM locally to develop with Ruby X.Y.W or you install RVM on the server to use Ruby X.Y.Z there. Of course if the OS on the server is too old or too new there could be some versions of Ruby that won't work. It happened to me because of versions of openssl.

That said, Docker is also oftentimes a poor option for local development, unless you're talking about external dependencies the app needs (e.g. MariaDB/PostgreSQL, RabbitMQ, Redis etc.), or maybe very specific setups that just won't be achievable otherwise (specific userland needed).

For example, when trying to run Ruby/PHP/Node/Python/Java apps in Docker containers, i've run into the following:

  - problems with file permissions
  - problems with file line endings
  - problems with Docker Desktop bind mounts (e.g. directory won't be mounted, path parsed incorrectly etc.)
  - problems with Docker Desktop bind mount performance
  - problems with networking (e.g. host.docker.internal sometimes refusing to route to non-containerized stuff on the host)
  - problems with run profiles (e.g. your IDE + Rails integration just won't work, at best you can just launch a different container/Compose stack with some other parameters)
  - problems with test runs and coverage (if you use the IDE integration, though that depends on the stack and how deeply your IDE is integrated, e.g. Java with IntelliJ IDEA)
  - problems with remote debugging, breakpoints, flame graphs etc. (most of those either don't work when the code is inside of a container, or need additional setup, e.g. remote debugging, where supported)

That said, I'm _really_ excited to get into all the goodness that Rails 7 has to offer. I really thought they lost their way with the whole webpacker debacle. I recently installed a fresh Rails 7 app and the word webpack doesn't even exist anymore! It's like a breath of fresh air to be done with that. Those were some dark days. (I'm exaggerating only a little)

There's really nothing I miss about Ruby for web dev and Elixir feels like a complete win to me these days. Ruby is still far nicer for unix-y scripting, though.

I meant something more like "of course there's a function for that with the name you'd expect".

This is not surprising: the Elixir folks were fairly high-visibility in the Ruby community before they did Elixir and Phoenix.

But most Rails programmers write Rails, not Ruby, and it's important to differentiate them. Ruby is a big playground. Rails is...not, by design. (This is why I do not like Rails, personally--if I am choosing to write code in Ruby, it's because I expect to need to get weird with it.)

Rails->Phoenix omits a lot of the stuff that makes Elixir cool, both in itself and in OTP, but that's not a bad thing and I do think the transfer isn't too onerous for somebody coming purely from Rails.

You're really exaggerating.

Roughly equivalent interfaces exist in Phoenix, and the mapping isn't quite 1:1 but it's close enough that the adjustment is an adjustment, I think, and not a re-learning.

I don't think we can compare languages based solely on the experience of juniors. I'm not saying it's irrelevant, it is, but it's only a part of what writing software is. In the end of the day you need the seniors and people who really get it to maintain the code and steer the ship in the right direction.

But I do agree that Rails is a kind of dialect of Ruby and that you can get pretty far with Rails without being great in Ruby (you still need to be decent though, no way of getting around that).

> Most Rails programmers I've worked with live in the controller and the view

Where is the business logic at? They also write services or something similar or stick it in the Model. Whatever isn't ActiveRecord will be pure Ruby - Rails has no opinions there.

Absolutely, and I would submit that this is most developers' experience up until a senior-ish role. Not title, you can have juniors doing this, but the pyramid of tool-users versus tool-makers.

> In the end of the day you need the seniors and people who really get it to maintain the code and steer the ship in the right direction.

Again agreed--but jeez, there are just so many fewer tool-makers at most places I've found myself. (This worked out great for me early in my career because there were tons of vacuums to start doing that work even in my first year out of college--building systems that other people could rely on to go faster, because nobody else was doing it!) Thinking about it, this impression perhaps sticks more because of the businesses I consulted for, which is perhaps where my perspective comes from--hiring uncritically 'til you're in a pit where you're paying the medium bucks to get somebody to come dig you out.

> Where is the business logic at? They also write services or something similar or stick it in the Model.

Found the guy who hasn't seen too many 10KLOC controllers in his time out there. ;)

You may be right that I am grim about this, but at the same time, reading your description makes me go "I haven't seen too many places that actually do-it-right". The truth's probably somewhere in the middle, and outliers exist on both sides. But yeah, I've absolutely been consulting for more than one company whose primary product was thin models, gargantuan controllers, and no service abstraction to speak of. They make a lot of money, too. (Fortunately for me, they brought me in for devops stuff, not "please save our megamonolith".)

I don't get why this is still happening. It's kinda common wisdom now to not let your controllers get fat. I"m not saying that fat models are great but as a community we kinda agree that most business logic stuff is easier tested on the model/service and should be placed there. CTO's/team leaders have to be real sloppy to let teams build code as you describe At the same time - being a consultant, don't you more often than not see codebases in bad state? I mean those are the ones that usually need consulting. So maybe you have an overly negative view on how Rails projects usually look.

I'm not following you here. Could you please explain your position a bit more?

The reason I'm confused by your commentary is that Phoenix makes extensive use of OTP and since it is just Elixir + macros, there is nothing at all limiting you from employing every Elixir (and Erlang, for that matter) feature you want to in your Phoenix apps.

At my current company, we use Phoenix and we use a lot of its bells and whistles. But we also have a lot of people who understand OTP--definitely way more than I do. I also see people ship Phoenix apps without getting deep into that when their needs are relatively small.

How do you think Phoenix could better take advantage of Elixir and OTP? IMO, it already is doing plenty to take advantage of what's cool about OTP, through channels and especially LiveView. I suppose Phoenix apps could use Mnesia instead of relational databases. But would the benefits actually outweigh the costs?

I'm considering migrating to Phoenix and my biggest concern is ease of deployment

Even 6 years ago when I started, you could just run it in production with `mix phx.server`, much like running `rails server` and use the exact same deployment patterns. Or you could build a release and copy it up to the server however you saw fit. At that time it was painful for some coming from different habits that involved pulling in config at run-time rather than compile time, but those patterns have also been supported since at least 2019.

Since then, the team has been continually making it easier.

Now there's a single command line to do a deploy to Fly and also one to build a docker container for you if that's what you want: https://twitter.com/chris_mccord/status/1468998944009166849

I generally use a VPS, but Render is a top-notch host if you want something managed. In many ways, I think it's like Heroku was before Salesforce bought them.

https://render.com/docs/deploy-phoenix

https://softwareengineeringdaily.com/2021/12/07/render-with-...

Here’s the docs for Phoenix deployments: https://render.com/docs/deploy-phoenix

A friend of mine who worked at an SEO agency once had a client who's company name was a misspelling of some noun that google would automatically fix which i found hilarious.

I would take a look at render.com. they support connected nodes and autodiscovery

I've never used Redis in any of my Elixir/Phoenix apps. In cases where I might need something like Redis to store state, I'd reach for a GenServer or maybe Mnesia instead. I've never had an issue with deployments using this strategy. Also, K8s and BEAM work together just fine using libcluster. I'm not a fan of K8s or Docker and don't use either one, but you easily can.

The benefit of BEAM is that it can surgically handle a vast array of failures without restarting the node. K8s, on the other hand, will take the pod down and restart it. In many cases, solving a problem with BEAM is like using tweezers to remove a splinter whereas K8s is more like using a sledgehammer.

And I see nothing wrong with having the BEAM run in Docker or K8s. There's some overlap in concepts, but they work at totally different layers. Using the BEAM is like having resilient microservices _inside_ your app with none of the downsides.

Also, higher performance translates to lower machine costs. Our server costs would easily be 4x if we ran rails. given that elixir is 80-90% as productive for writing code in the short term, its a fair deal. Over the long term, we've found elixir to be more maintainable since there's very little magic and the immutable data structures and lack of magic make it super easy to debug.

Thanks to the performance of our system, we've also managed to get by without setting up caching. (we'll probably need it eventually but our performance gains have come mostly though fine tuning our sql queries. Ecto gives us way better control over the output sql than active record or sequalize)

People who pay for those extra boxes? Not everybody has your budget (especially when they're building things themselves, on the side, so they aren't even paid).

Scaling vertically can take you a long long way before you actually have to start separating services.

I think it can be simple if you keep it simple :). In my book Deployment from Scratch I show people how to run Rails on a cheap VPS with a couple hundreds lines of Bash.

Just Ruby + chruby + Puma + systemd (and optional systemd socket activation).

I include a scripted demo (so you don't have to do it yourself) which demonstrates all of the this:

- Setting up a git-pust deployment for Ruby applications

- Using chruby to automate installing requested Ruby versions

- Configuring NGINX as a proxy using UNIX sockets

- Setting up systemd socket activation for graceful restarts

- Configuring NGINX to serve static assets

- Configuring NGINX to proxy WebSockets connections for Action Cable

- Automatic SSL/TLS certificates with Let's Encrypt

- Redirecting HTTP traffic and subdomains to main domain over HTTPS

- Running PostgreSQL and Redis on the same server

- Building a custom SELinux module

- Configuring firewall

- Setting up automatic weekly system update

- Setting up log rotation for NGINX and PostgreSQL logs and max limit for system log

- Doing application backups and restores

- Creating admin tasks

Although it sounds like a lot, the demo is reasonably small and clean so you can go through all the files in 2 hours.

I think people many times complicate what they don't need to complicate...

> a couple hundreds lines of Bash.

One or the other. It can't be both.

[1]: https://github.com/havenweb/haven/tree/master/deploymentscri...

And it's great!

Hatchbox takes the maintenance and setup out of DevOps for me!

Using Hatchbox makes it easier to treat servers like cattle and less like pets!

I'm not even using node very much these days. Serverless is what I would reach for for most professional applications these days.

Suppose you have been given 100,000 developers to work on a website, but it will have millions of customers and things cannot fail.

Now it is actually useful to be able to parcel out the database layer to a group of 1000, and they can split up each service into groups of 100 etc.

There are enough people to decide what conventions, logging, validation people ought to use and perhaps measure them. If all of them hack on the same Rails codebase they might be running into each other all the time.

Of course, for a solo developer, it makes sense to use Rails/Django/etc.

Now there is a little spot in between 2-99,999. It is possible that there is a team or two at this level. And I think that gear shift or pathway has not yet been discovered. Typically you start on rails, suspect you should switch and then switch way too late.

A pathway might look like this:

Django -> Django API + VueJs -> Redis + update database?

It probably depends on the challenges.

If someone has an idea, or something they have tried, it would be interesting to take a look so please share.

Edited for clarity.

The vast majority of businesses/projects are small, and will only have one or a small number of developers working on it. So Ruby on Rails (or PHP or Wordpress) is a suitable choice for maybe 80% of projects.

At the same time, a very small number of businesses require 100,000+ (or even just 1000+) developers. These businesses, because they have so many developers, employ the vast majority of professional developers. Thus, maybe only 20% of professional developers work on products that are typical of business needs.

This disparity explains why communities on Hacker News are typically so negative towards Ruby on Rails. It also explains part of what makes Ruby on Rails so remarkable. Even after all these years, it has a thriving community and has resisted the pull to become more "enterprisy." It's still a tool that is targeted at and well suited for a large swath of business cases, the vast majority of which will likely never need to migrate to something "better".

It’s good to start simple and only add complexity when it’s really needed. It’s unfortunate the current trend is to start with massive complexity.

He got fired.

This is exactly why, in 2004, I moved from Java (Struts/Spring/Websphere/XML/...) to Rails. History repeats but we never learn. Kudos to Rails for remaining relevant.

- slow to compile and run

- easy target for spaghetti code

- test suites get sluggish

- hard to upgrade

- won't scale well

- with no static typing, large scale collaboration can easily break things

- metaprogramming hell

P.S would have been happier if you had a better choice of words than "knocking myself out".

As the company scales they really need to break things into more services to distribute work and satiate work happening in other languages, but now that’s nearly impossible because of the mess they’ve created

Rails is great if your building a basic “dumb” web app, but rails folks seem to think it’s more than that and need to get their heads out of the clouds

Let me stop you right there. If you tell me Rails wasn't meeting your needs so you rewrote (X) in a different manner using Ruby + something else, I'd totally understand. Otherwise, all it tells me is that there was a tragic lack of deep Ruby knowledge, respect even, to begin with as the project unfolded. Often I muse on how in some ways it was unfortunate that Rails became such a darling in the startup community early on. It meant that a ton of people jumped into Rails web dev thinking they were writing "Rails". No, you're writing Ruby, and Rails just provides a nice set of base classes and some decent defaults and assumptions. If you end up with a giant wad of spaghetti mess, that's on you. It's totally feasible not to end up there, and plenty of projects end up in far better shape.

Really?

https://twitter.com/ShopifyEng/status/1465806691543531525

Github is in Rails. I mean, seems like it scales just fine.

It takes increasing the number of pods on your k8s which is how almost all teams deploy nowadays. Bigger amazon bills yes but the devops overhead isn't really larger in Rails,

I knew a few people at university, who were pretty hyped about it. That was in 2008, I think. They used it at some companies where they jobbed and found it too limiting.

I was building customized stuff with bare PHP in those days and found it quite flexible. Later, the Rails hype came to PHP, but I already left for Node.js and never looked back.

Like anything else, Rails is good for some things & terrible at others. Rails tries to minimize developer time, at the cost of computing time (Ruby is not a fast language), and focuses on CRUD-type applications. Whether or not that's a good match depends on what you're trying to do.

I saw this as a sign that the time of these fully fledged frameworks was over and devs were craving modularity.

But, yes, I saw a bunch of recent projects done with Rails. Forem is built with it, for example.

To cut a long story short, Active Record (especially at the time) wasn't super keen on generating super good SQL that would do as much as possible in database, and Ruby 1.9 arrived with stable ordered hashes. Then someone implemented basic ActiveRecord wrapper on top of key/value store (I think it started with Berkeley DB or similar), and from there NoSQL was off to the races as "new hip thing"[1]

[1] Meanwhile IBM IMS has been selling pretty well for last 40 years at the time ;)

At some point, I gave up on even trying to use any Node ORM's because they were clunky and offered no value. It was easier to just drop straight to a knex session and write the queries natively.

Though I would also consider their exodus to be a good thing for Rails ;)

I guess by now Rails could also be considered boring technology.

It is tiring to keep track of latest fads, specially those that don't happen to stick around, so only move when dust settles down.

Yeah one might lose business opportunities from not being first mover, on the other hand, there are business opportunities to port back the surviving projects into classical development stacks.

For me, still much better than dealing with management.

There's no need to chase the latest Node JS shenanigans. It feels like it's all we talk about on HN but a huge, huge number of developers are out there happily coding away in Java, C# and all the rest. Even within Node world the majority are using the exact same React stack (the homogeneity of which is the reason for using it in the first place).

IMO Ruby has faded because it never found its niche. Java and C# thrive in the corporate world. Python already exists as a dynamically typed server language. As much as people complain, writing server-side web stuff in JS does make sense given it's what you're using in the client. So where does Ruby go?

Same thing with Rails/Laravel/etc...people don't choose it because PHP or because Ruby , etc...the language doesn't matter at all, the tooling does matter it all.

https://twitter.com/logicmason/status/1371255029412233218

In the web server side, helping you build web apps faster. That has been it's niche (with Rails) and it's still doing well there.

Ruby isn't new; it's only slightly younger than Python, and brought things to the table. Rails and Django are roughly the same age (and Django started off more as a CMS). That Python exists isn't an argument against Ruby. Lots of other languages exist, each with their own cost/benefit tradeoffs.

> [...] writing server-side web stuff in JS does make sense given it's what you're using in the client.

I don't understand the argument. Matching client/server language is a different goal than matching task/language, and the task isn't the only driving factor.

There's some crossover in that the language is the same, which means you could standardize on JS developers, but the ecosystems are not the same, and the required overall skillset is different. In wee shops the tradeoff may be worth it, but at scale, the benefit is oversold.

Well JS is forced upon us on the browser and not everyone is tremendously happy about it. It has improved, but still. Plenty of folks out there will seek other languages, we will never all fully agree on what's good on the backend.

BTW - are we positive JS will have a monopoly on browsers 5-10 years from now? (I'm talking about WebAssembly). Cos if the monopoly is gone JS will die a violent death I think.

Yes.

I will assume your question arose from the exuberance of youth.

"Good enough" + inertia = Yes

It's not enough to be better; it has to be so much better, existing infrastructure and training must become moot. And preferably an obvious best to avoid balkanization.

See: Fortune 500 and their continued reliance on COBOL and mainframes rather than opting for replacement and retraining costs.

Going with the masses (e.g Java/Node/Python) is the easy thing to do. It feels "safe". But who knows where Python/Node will be 10 years from now, people are restless and will come up with new things (Deno?) and it could be they will start declining. Java is pretty much unavoidable though.

As an ML engineer it’s really hard to justify spending your time in a language you will be handicapped by instead of just using python

Sounds like bike-shedding to me. Eg. You start with solving trivial problems instead of the hard problems.

Usually the first thing that is decided is what framework to use, often by a non technical person that herd from a friend that is was good. Rather then having the engineers sit down and think about what are the biggest challenges, and what is the best framework suited to solve the hardest problems.

This video was recently linked in other thread on HN, but I feel this comment really resonates with the main conclusion: https://www.youtube.com/watch?v=pW-SOdj4Kkk

[^] Mostly data engineering at work, and low level os/system dev in my spare time.

Could you let me know what your job title is - I am looking for something similar where I would like to work on some semi technical stuff and at the same time manage developers but not sure what title I am supposed to look for.

Given the popularity of rails at the time, if node.js was unproductive like you say it is, it would have never caught on.

The reality is that express was considerably simpler and easier than rails and that's what made many of the internet companies we have today even possible.

After the sit I seen with my own eyes(not internet stories) I am convinced that there are packages like "red" that defines the color RED="#ff0000" and a package colors that depends on all colors, and probably a few big dev tools that depends on the colors one, this node/npm ecosystem is crazy but we are paid to maintain and fix other people stupidity.

EDIT forgot to mention why stuff gets complex after time, you hit cases where you need to get a new version of package A , but A needs say a new node version or some new other shit, but you have a package B that will fail on the new node version with some stupid error, and package B was abandoned or the fix is to upgrade to a new major version... also you will notice that your packages are now abandoned and might have security issues when you inherit some old project , it is a big mess.

I guess I should be thankful re-inventing the wheel on all of these aspects in node has given me hundreds if not thousands of hours in compensation.

Never understimate the power of fad, tech pop culture, hype, and trying a shiny new thing...

Exhibit A here is the JS language itself. It became popular not because it was the best language, just the most ubiquitous one. You could read the 25 years since as trying to turn it into a solid language.

Or look at the rise of PHP. It started out as a tool for making a Personal Home Page, which is a fine use case and was certainly a popular one in the mid/late 1990s. It eventually turned into the foundation for one of the world's largest companies. But nobody can argue it was a particularly good language. E.g. https://eev.ee/blog/2012/04/09/php-a-fractal-of-bad-design/

Same thing with Java, really. Bunch of interesting ideas, some of which worked out and some didn't. Even at launch a lot of the things that have turned out to be problems were criticized. They say that Java is the new COBOL, and that seems pretty fair to me.

I could go on all day. The most popular technologies are not always the best. That applies to movies and music and pretty much everything. And personally, I've accepted it. It's part of how the world works. The trick for those of us who like "better" things is to figure out how to get them to become mainstream.

The easier deployment story for PHP3 meant you saw massive expansion at the bottom of the market as cheap webhosts were able to offer very locked-down accounts for peanuts, which meant a generation of developers came along whose only experience was with PHP. They didn't have anything to compare to and nobody was really offering them anything else at anything like the same scale. The virtualisation revolution hadn't happened yet, so these were all shared accounts on physical boxes, and they couldn't install their own stuff either.

It's just not reasonable to say PHP was "better" except in a very limited, and largely accidental sense. It just so happened that that one advantage was enough to catapult it into first place.

It’s just me building this and I have no one in my circle that’s familiar with spring.