* Wow, an O(n^2) require algo. I guess this really drives home that we shouldn't make basic assumptions about the quality of the libs we use without actually looking at them. Great catch.
* For all of you waiting for this to be pulled into Ruby master. Why not just patch your copy in the meantime?
Wow, amazing find. I've been struggling with this issue for many months, and I just figured it was rails 3.0 fault for being so big and bloated. Now we find out it's ruby implementation. I really hope this patch makes it in.
I've patched my local copy. Feels like I have doubled my computer speed! Seriously though, I can't believe the fix was so easy, and the ruby core hasn't done something about this yet.
The last major external fix to ruby was with the REE people, and that still hasn't made it back into MRI. So I'm a bit worried this patch will get rejected and the problem won't be fixed.
tenderlove is going to follow up on this. The issue is for metal-like responders, not for full-fledged apps. We've lost a couple of ms on the low-end due to the stack issue.
I've always thought it would be nice if you could cause Ruby to dump its heap and symbol table to a file, which could then be loaded by other instances of Ruby. So, for example, you could run Ruby, load all of the Rails files, and then save everything, so the next time you want to run Rails, you just have to load one file instead of hitting the file system 2000 times.
You could probably also do this by running a Rails instance as a service, and then every time you want to run Rails, you tell the service to fork, and use the forked thread. I think this is how Passenger works actually?
You could probably also do this by running a Rails instance as a service, and then every time you want to run Rails, you tell the service to fork, and use the forked thread.
FWIW, this is how Spork works, a tool people have been using a lot more recently in order to have fast(er) tests on Rails 3. Spork boots up an environment then waits for RSpec (or whatever) to hit it over DRb and then it forks off for that run.
One day in the early days of computing, General Electric had a problem with their computer. All of their engineers took a look at the problem. Although each was wise, they were unable to understand the complexity of the machinery and repair the error. A call was made to the retired engineer who had helped in the original set up of the machine.
The retired engineer walked around the machine for a few minutes, just looking it over, not touching anything. After a few minutes, He took out a piece of chalk, walked over and placed a large X on one particular part of the machine. He then said' "Tap it here with a hammer, just once."
After the one tap, the computer roared back to life and began working!
A few days later, GE received an invoice from the retired engineer for $10,000! This was a lot of money in those days, so they returned it to the engineer and asked that he itemize his invoice.
A few days later, they received an itemized bill which read:
Given the roots in Perl, which has the %INC hash, it's surprising that Ruby started with an array for loaded modules. Can anyone shed some light on this design decision? Surely there was a reason at some point -- and maybe there still is.
If you're not using Bundler and have a simple dependency chain, you can get further improvement from symlinking your gems into a single directory (see https://gist.github.com/975509 ).
This patch certainly improved my load time tremendously, but the core of the problem still lies with the way rubygems and bundler dump all directories of gems in the load path. The promise of $LOAD_PATH is that all the directories in it will be tried -- the most bang-for-buck optimization is thus minimizing its size.
If rubygems did this, everything would be much better.
There is one problem though, and it is that some libs are bad citizens and install stuff outside of lib|bin and depend on it (haml's VERSION file is an example of this), and that sucks.
Rubinius uses the exact same technique as is detailed here, namely backing an Array with a Hash (LookupTable in the Rubinius case). We added this a few years ago to speed up require.
I haven't tested this myself, but I can guess that they will load slower in development. In development asset requests go through the asset pipeline now instead of being directly served up through the HTTP server.
Not really for application specific images. However, vendor, lib, and engine images will be copied into public in production. Keeps you from manually having to keep somebody elses asset dependencies updated in your app.
Sorry for the snide comment, but if you want to really speed up your Rails, switch to Go, it will compile from scratch and start up your project faster than the ruby interpreter starts up.
I used rails in a few projects, and performance was so painful even on pretty good hardware that I said never again. Go on the other hand is pure pleasure.
This is a bit like saying, "If you want to get better gas mileage out of your car, walk." Switching to Go would be throwing the baby out with the bathwater. If you like Go, that's good, but people who are using Ruby and Rails presumably like those technologies. It's unlikely that somebody would choose Rails based solely on a mistaken assumption that it has the fastest possible runtime.
He reduced the example in the post to the core of the issue, which is the time complexity O(n) for each load (thus O(n^2) overall), and his replacing the existing data structure with what seems (I didn't look at the patch for very long) like a hash[1] which should be O(1) for each load (O(n) overall). A tradeoff of speed for some additional memory.
There is quite a bit going on in module loading. I think reducing it to 'ass clowns' is a bit unwarranted.
[1]: publicly exposed as an array, but a hash internally for actual loading and checking to see if loaded already.
* Wow, an O(n^2) require algo. I guess this really drives home that we shouldn't make basic assumptions about the quality of the libs we use without actually looking at them. Great catch.
* For all of you waiting for this to be pulled into Ruby master. Why not just patch your copy in the meantime?