Well, in Python world we did just like what OP described: use uWSGI to respawn a new worker after certain number of requests.
Edit: alright, I got downvoted for speaking a fact. Don't get me wrong, OP's article is great, but not everyone has control over the underlying code, nor has enough time to tinker into runtime nuances.
To confess I have maintained >100K LoC Python/Ruby projects, yet the respawn way is the most effective and easiest mitigation method available and I think it's likely to continue to be in a very long time.
Lipstick on a pig. Exactly what the author calls out:
"Sadly, most Ruby devs out there simply employ monit , inspeqtor or unicorn worker killers. This allows you to move along and do more important work, tidily sweeping the problem snugly under the carpet."
The author describes a mindset to solve the underlying problem. Endemic to any language.
"I'm not a real programmer. I throw together things until it works then I move on. The real programmers will say "Yeah it works but you're leaking memory everywhere. Perhaps we should fix that." I’ll just restart Apache every 10 requests."
I mean, I'd be first person to tell you that I am a scripter at heart, but I would never, ever say "I don't care about this [computer science] crap at all." I feel like my job is to defer to the many programmers that are smarter than I am. Including the parser and compiler and linker etc
These quotes aren't terrifying they help explain why something such as PHP is so successful. The general philosophy was, and still is building something that spits html and can connect to a database in a few lines of spaghetti code. Granted there is a movement in PHP that is as religious regarding TDD,OOP as in the Java world, they don't represent the majority of PHP devs.
PHP execution model satisfies 2 crowds : hosting service providers that can provide cheap hosting to the second group of people, hobbyists that just want to install wordpress or write a "dynamic" page.
The later will never care about OOP,TDD,FP,Agile, and stuff like that. In PHP you begin with writing HTML then you put the PHP code. not the other way around,no matter how hard "advanced users" try to deny that fact, PHP scripts begin with a TAG,it is,first and foremost, a templating language that generates HTML files.
The sad thing is that it could have been done a better way. But serious language designers were too pedantic and arrogant to write simple solutions. Rasmus lazyness paid off ...
> PHP execution model satisfies 2 crowds : hosting service providers that can provide cheap hosting to the second group of people, hobbyists that just want to install wordpress or write a "dynamic" page.
And untold tens of thousands of companies using PHP to drive their corporate internet presence.
"We have things like protected properties. We have abstract methods. We have all this stuff that your computer science teacher told you you should be using. I don't care about this crap at all."
I kinda agree with that sentiment. There's a lot of unnecessary stuff going on in CS. We had extreme OOP and I fear extreme FP will spread into the mainstream. The mission of programming language designers should be to help programmers do their jobs better, not design something elegant/interesting/powerful/expressive/[buzzword]. Those things are all good, but only in the service of programs with fewer bugs and shorter time-to-market.
> The mission of programming language designers should be to help programmers do their jobs better, not design something elegant/interesting/powerful/expressive/[buzzword]. Those things are all good, but only in the service of programs with fewer bugs and shorter time-to-market.
That may be true for programming language designers, but I'm afraid you misunderstand what CS is all about. Hint: it's not about time-to-market. The job of CS is to explore the theory -- precisely how to do something in interesting/elegant/expressive ways. Its job is to explore the theoretical and practical underpinnings of computing; to provide proofs; to explore the abstract and formal systems behind software. CS is not necessarily directly applicable to enterprise software (though of course CS research does have practical applications; and of course there is overlap between theoretical and practical computing).
A lot of cognitive dissonance comes from people who look at CS thinking it's what they need in order to write software.
This is ridiculous. I think you are seriously missing the point of PL research. Furthermore, I can not even imagine what your definition of "helpful" is when you say not ".../powerful/expressive/...". I honestly can't even argue with you if you do not think power or expressivity is beneficial.
PL is not a buzzword-oriented culture. No (serious?) language designer creates a language with the intent of making it harder for programmers to do their jobs. (Brainfuck and other Turing traps excluded.) That said, being "helpful" can be interpreted in many different ways. And of course, every PL designer does interpret it differently!
OOP took off because it was a revolutionary way of structuring your code so it could be easily used, reused, and maintained. The point was to help teams be more efficient!
Functional Programming is so worthwhile first off due to immutable types. This allows highly parallelizable code. You don't have to worry about race conditions because your objects are read-only! The emphasis on types also allows better ability to prove correctness of a program before run-time. Finally, all FP languages (that I can think of) are highly expressive. This means that you can write the same program in (OCaml, Haskell, CL, Clojure) in a shorter amount of time than you could in Java/C#/Obj-C.
If you still think there is "a lot of unnecessary stuff going on in CS", I'd really like to hear what you think is necessary and we'll go from there. I already discussed your "fewer bugs and shorter time-to-market" points.
> I kinda agree with that sentiment. There's a lot of unnecessary stuff going on in CS. We had extreme OOP and I fear extreme FP will spread into the mainstream. The mission of programming language designers should be to help programmers do their jobs better, not design something elegant/interesting/powerful/expressive/[buzzword]. Those things are all good, but only in the service of programs with fewer bugs and shorter time-to-market.
I'm not sure where you're going with that. I have stared at enough medium- to large-sized codebases to realize that an excellent way to "fewer bugs" is to protect programmers against themselves and restrict their proven ability to shoot themselves in the foot (especially in the absence of code review and/or competent technical leads). Shorter time-to-market is actually a different property. PHP or Rails got popular because they make it easy to crank out a web application in a relatively small amount of code. However, this often doesn't translate into "fewer bugs" when the codebase gets big.
I don't know what "extreme FP" mean, but there is a night and day difference working with referential transparency and immutable data structures in terms of cognitive load and ability to change parts of the system with confidence (for the record, I'm also mystified about "extreme OOP").
PHP is (admittedly slowly) moving in a positive direction because of community effort, Rasmus' opinion shouldn't terrify anyone since he's just one vote of many.
I'm of the mindset from 1995-2005, computer science and software engineering didn't exist while the web matured. PHP was major player in this time, as was Perl. Perl could have "won" just the same as ColdFusion, or VBScript.
It took a decade for these web programming communities to introduce and/or build on ideas from 1958. Long before the web.
PHP is better, however Rasmus' philosophy (while ok at the time) permeates PHP's design and it hurts. It hurts a lot.
> Perl could have "won" just the same as ColdFusion, or VBScript.
Perl was far more widespread early on (96-99) than PHP was, but it was quite painful to deal with compared to PHP. PHP made web stuff dead simple, and that's why it 'won'. CF and VBS never really stood a chance because of the extra costs involved. iHTML, HTML/OS and others were all vying for a foot in the door at this stage, but all were second-run players charging money for something that 99% of hosts were never going to pay for, and therefore devs wouldn't get exposed to those tools.
The funny thing is, despite being originally inferior in everything but simplicity compared to Perl, PHP has added feature upon feature (usually poorly thought-tout) over the years, and nowadays looks like a poorly-designed Perl with a terrible standard library.
And an absolutely enormous installed base as well as a huge amount of code available for use. It's funny how it succeeded so well in spite of being a fairly crappy language.
It really illustrates how useful a quick way to get started with the language is. PHP could provide something useful to the author and others (web content) almost immediately, while Perl required you to understand CGI (to some degree) first. In some respects, PHP's lack of features was beneficial, as each missing feature was something people didn't have to learn about, even if tangentially. No packages or namespaces? A poor module ecosystem develops, but devs didn't need to think about how it worked for the most part.
I suppose the thing is that Perl is a general-purpose programming language that was not really optimized for the Web whereas PHP started out with no ambitions other than being a Web templating language, which made it easier to get started with.
I think this is pretty reasonable depending on how severe the issue is. You should have some monitoring and alerting on this. If this is happening once a month under extreme circumstances, its probably safe to let things continue like that with minimal investigation. But these are typically issues that will hurt you when scale starts to matter. I've personally experienced that. Ignore an issue that you know is important, but seems like it can wait relating to these kind of leaks, and a few months later you're in panic mode trying to hotpatch.
Sadly many ruby, python, and php devs will just use process supervisors as a bandaid to the problem. I understand that some people who aren't really programmers need to run a few scripts sometimes. For them supervisors make sense. Your average shared hosting environment running wordpress with some random custom plugins will probably benefit from a max of 10 or 100 requests per child.
For anyone who's job description involves writing software and owning it's performance and reliability, process supervisors are a crutch. A crutch to be used only until the memory leak is found and a patch submitted to the relevant project.
A former colleague used to refer to the concept of software entropy. Like physical entropy, it tends to increase. It's both natural and inevitable. But as engineers we sometimes, for brief windows of time, have the ability to impose order on chaos. To locally decrease entropy.
I've seen the end result of relying on process supervisors to work around frequent crashes and memory leaks. The result is that you end up with ruby processes with multigigabyte heaps, resulting in a stack that can only run a few dozen requests/s even on the beefiest of hardware.
To be clear, I'm not saying don't ever use process supervisors. It's just that if they are saving your bacon multiple times a day something's wrong.
I'm also not claiming that inefficient software will cause your company to fail. It's perfectly possible that your buffaloak architecture will keep running until your company dies of natural causes, but I believe that we should have enough professional pride to fight entropy, even if we're the only ones who know what we did.
That's one of the things I like about the Java ecosystem. The tooling is very mature. Heap dump analysis is easy and being able to connect to a live JVM, either using JMX/remoting, Thermostat or Oracle's Java Mission Control, makes it almost painless and easy, almost, to look at your JVM internals and see what bottlenecks you might be hitting.
I don't know how to get the same kind of information on Ruby, Python or node.js but I'm pretty sure it might involve some gdb and debug symbols. No idea.
"technically". Almost any comment containing this word is not helpful. So is this, we don't need a definition debate of memory leak every time it comes up for dynamic languages.
Memory leaks are technically when you allocate memory, then lose the reference to it without freeing it, thus leaving the memory permanently allocated without a way to free it. Outside of a serious VM bug, you won't leak memory in a GC'd language; instead, you can leak references (leaving objects alive and uncollectable even after you've finished using them, because you've left a reference to the object laying around somewhere without intending to). The memory is still reclaimable, if you can eliminate the references keeping the object from being GC'd.
This really is just pedantry though, because the net effect is "the memory usage chart keeps going up and to the right" in both cases. "Memory leak" is a fine enough term for what's happening in this case.
They are both memory leaks. Unneeded memory is not being reclaimed. It's not even pedantry. It's just incorrectly applying one type of problem to the whole range of problems captured by the term.
Relax. It's not pedantry. If I click on a link titled "Memory leak in Ruby" I'm concerned the GC has a problem and I should avoid some particular method call or something else because the GC is currently having trouble collecting something in particular. That would be a memory leak.
The Full Article is talking about handing over data with references to live objects to some library that doesn't let go of those references, it seems. That is a space leak and can and will happen anywhere in any language regardless of whether you're using a perfect GC or not.
So the difference is this:
Ruby has a memory leak -> not my fault, I'll just stop using the offending method that leaks memory, thanks article!
Someone caused ruby to space leak -> duh, it's your fault, no need to write an article about it, you should be letting your references be GC'ed if you don't need them anymore, learn that memory is a scarce resource, etc.
Totally different problems. I was very confused reading the comments here because people are afraid of being called a pedant. Folks, our trade comes with a jargon. If you don't learn it you won't know what you don't know.
Saying simply "unneeded memory is not being reclaimed" is not constructive. Why is it not being reclaimed? Because the reason matters to programmers, we invented different names for those occurrences.
You can have space leaks without memory leaks (if you build up big enough thunks/delayed computations/promises or otherwise run out of heap space) and you can have memory leaks without space leaks (malloc 1 byte and lose its pointer).
It's OK if you don't understand the difference yet - you'll get it eventually - but there is a big difference.
>It's OK if you don't understand the difference yet
Oh I understand the difference. But losing track of memory and accidentally keeping memory in play are both called memory leaks. I would still call it a memory leak if it's due to cyclic references or stale entries in a hash table or other container.
> Considering that each entry could involve a large number of AR objects memory leakage was very high.
Ah. I thought I was going crazy. Memory leaks in a managed language (i.e. one with a GC) are solely the fault of the GC implementer and cannot be fixed by the language user.
Edit: alright, I got downvoted for speaking a fact. Don't get me wrong, OP's article is great, but not everyone has control over the underlying code, nor has enough time to tinker into runtime nuances.
To confess I have maintained >100K LoC Python/Ruby projects, yet the respawn way is the most effective and easiest mitigation method available and I think it's likely to continue to be in a very long time.