However, it's also a failure - in part, due to Brian and his attitude towards contributors. See this twitter conversation: https://twitter.com/the_zenspider/status/547527644535726080 He's been grinding on Rubyspec for years, bless him, but I think there's a reason why he was unable to rally the community behind his effort, both in terms of gathering more contributors and in making it "official" in terms of the language spec.

JRuby, currently the only non-MRI ruby implementation that you can seriously consider for production use, runs the MRI test suite against JRuby. RubySpec is far from The Only Solution, though Brian would like you to think it is.

you're right, but I remember learning quit a bit of ruby by working through RubySpec back in the day. and I remember liking how well structured, logical and organized the suite was, very exemplary. so I'm sorry to see it go. I have no perspective on Brian or the mri devs so I can't say anything about the situation around it, I just liked the code and spec and found it very well done.

It would certainly be nice to have such a ruby specification and test suite that was endorsed by ruby community at large (unlike what I saw in those example MRI tests).

Looks like he seems to be burned out - maybe he just wants something in return for his time investment e.g. a paid job or some other gratification in return for his work.

Btdt - you put a lot of work into something, people/companies start benefit from it but they don't return anything important (monetary compensation is not a bad thing -it allows one to pay the bills/go on holiday/get a life).

We've seen this "pattern" a lot in the OSS world, but saying a different attitude would "fix everything" is a lie. We (as in the Ruby community) need to crowd-fund independent developers who are not paid by companies for their OSS contribution IMHO.

Are there concrete shortcomings when using Rubinius in production, or is it just not battle-tested yet?

It's a promising project, but as far as "no GIL, true concurrency, mature GC" implementations go, JRuby delivers the goods.

Edit: For kicks, I'm trying to boot my primary app with RBX right now. RTLK threw some "id for nil" exceptions due to some initializer stuff, so I disabled that, and now, trying to run my test suite, after 2 minutes of sitting there doing who-knows-what, it finally attempts to boot my app and dies with a "Missing constant" error despite the fact that I put in debug code to ensure that the file containing said module is a) loaded, b) executing properly, and c) that said constant actually exists in the module tree.

This code boots and runs just fine under MRI 1.9, 2.0, 2.1, and JRuby 1.7. I might be able to get it running with some more work and explicit load path management, but I'm off to a party for now, so I'll poke at it a bit when I get back.

If Rocher and Laforge had come clean about how they turned the RI into the language itself, the backlash might have blown over quickly, but instead they led developers along for many years afterwards, not changing the spec to dormant until April 2012. Projects other than Grails who've tried to build atop Groovy have had to risk the ref impl changing in breaking ways between versions. The most spectacular incident was when Groovy++, an experimental static compiler built by Alex Tkachman that hooked via annotations into Groovy's AST, had to drop back down from Groovy 1.8 to 1.7 in 2011, and my own side project was also affected by the change. It turned out Rocher and Laforge had secretly employed a mate to extend Groovy with the exact same static type-checking and compilation functionality as Groovy++ and were obviously trying to shake us off.

Unlike Ruby, Groovy only has one other implementation, GrooScript, built by Jorge Franco, which generates JavaScript from Groovy syntax. When the developers of the most used implementation of a language want to protect their control, it certainly does hurt the ecosystem, turning it into an "echo system".

Matz and Charles have helped to set the tone for the community. While I appreciate Brian's passion, it sounds like he needs to check his ego. No matter how smart we are, we can always learn things from other people. Without the collaboration of others neither JRuby or Ruby would be what they are today which is why they are successful.

The small number of mature Ruby runtimes is a bad sign.

"Diverging from a formal spec" is one thing. As it is, many versions of MRI has diverged from how the core team thinks it works. If MRI is "the spec", then Ruby changes from revision to revision as the test coverage is nowhere good enough to prevent regressions or changes in behaviour.

Basically: Nobody knows Ruby.

Frankly, I very much hope that another Ruby implementations overtake MRI sufficiently in quality to out compete it enough to shift the initiative of definining the language to a more responsible team.

I really wish it was:

"I set up a server that runs ruby spec on ruby-head daily and automatically reports spec failures to ruby-bugs"

So many companies are making big bucks off Ruby, yet so little are willing to fork out a bit of money and time to make Ruby better.

I get that "the implementation is the standard" is frustrating for people who want to make an alternate implementation, but why should that obligate the original developers to accept some third party's definition of what their project should be?

And presumably the MRI devs would say that by coding MRI you're defining Ruby. What gives RubySpecs the authority to say they are the canonical definition of Ruby and everyone should follow it, when the people who created Ruby disagree?

They most certainly don't, do either of those things. You can still have incorrect, incomplete specs even with an executable test suite. You only get those properties through constant work.

The idea behind TDD is to do that as you develop the application, but even then you're not automatically saved from incorrectness and incompleteness.

The only way to really do it is the way RubySpecs intended, create a third place for the specification of the language to live, that any implementation can use to measure its compliance against. If anyone running any implementation on any platform finds a problem and traces it to the implementation, then a new test case can get written and all implementations can then fix the bug assuming it's present. Code quality goes up for everyone.

Most other applications don't need to do it this way, they're intended for an install base of one. But since Ruby has multiple implementations intended to be able to run under multiple platforms on multiple architectures, a living collection of specs that ensures that each combination still has the same behavior would seem to be absolutely essential for one's sanity if one cares about the quality of the language. Each implementation then only has to maintain platform-specific test suites. It's ultimately less work.

But just like TDD/BDD, it requires organizational buy-in to really get the benefits. Everyone working on Ruby needs to be involved in maintaining the test suite or there's no way you can consider the suite canonical. If you can't consider the suite canonical, there's no point in having it.

I develop on MRI at present, but at some point I'm going to switch to either JRuby or Rubinius. If Brian actually follows through on his goals to implement even better ways of specifying Ruby than a test suite, I'll probably switch to Rubinius.

I think something like RubySpec is fantastic, and that it's really too bad that, perhaps only because of personality conflicts with the maintainer, it hasn't been successful at standardizing behavior across ruby implementations.

It's just such a better outcome, nobody needs to change workflows its just that information gets reported upstream earlier in a much more useful time.

Heroku does sponsor Matz, Nobu, and Koichi. We hired them and give them a full time salary to work on Ruby.We don't tell them what to work on (i.e. we don't dictate what features or projects get shipped in what versions) it's more like corporate sponsorship. GitHub hired tmm1. Beyond that CRuby has a host of other non-sponsored contributors that contribute code and doc patches as well as set up tooling. You don't need corporate sponsorship for that, you need a passion and some time. You, in fact, could be the very person that sets up this automated system.

I find the attitude of '<company x> should sponsor <thing y>' a bit misguided. While I agree that companies who profit from a OSS should give forwards to OSS, individual contributors are ultimately they only way progress gets made. If you or someone reading this works for a company making money off of Ruby do that thing! Tell your boss you can't deploy on friday afternoon and have to fix a bug in the Ruby codebase. Ship the project on company time, and then boom...your company just sponsored that thing. It's like magic!

Anywhoo, yes finding out what things are blocking the MRI team from adopting a tool/technology and working around them can be extremely valuable. For example Matz has agreed to move the codebase development to github. This would make contributions a bit easier, however there is a blocker. There are a ton of SVN bots and tooling written around the current workflow. Right now the Ruby core team wants to spend time focusing on pumping out C code to fix bugs, improve performance, and progress the language forwards. They don't want to go off into the woods on tooling. These types of projects could be hugely impactful to the team and don't require C knowledge so most Ruby devs may be able to help. I encourage you and others to reach out to the core team to ask them what tooling projects they need and how we can help them. Understanding why someone isn't using a tool/technology is a really good start.

tldr; Good idea, let's talk to the core team about ways to help with tooling.

And even if there wasn't, @brixen has repeatedly pointed out how the MRI authors have chosen to ignore the RubySpec project entirely. That's really sad.

Note that I'm not trying to criticize this position, rather, I'm pointing out that the core Ruby team seems to have different goals than, say, the community around Java or C#.

Cucumber provides better reporting than JUnit/TestNG. For web testing you have watir. For SOAP services you have savon which IMO is much easier than SOAPUI. For RESTful web services you just make direct calls. Then there is sikuli for ad hoc GUIs.

Long story short (and hope my facts are right! not swearing by this, was just some googling...), I had to optimize some old code for an RoR client I freelance for and wanted to add in some more advanced ORM (still fairly simple though, mapping entity results from a stored procedure query). All the solutions I saw for this involved kindof hacking the db connector to execute some raw queries. I was curious as to why such a basic feature wouldn't be natively supported, so eventually I found some forum posts asking why it isnt (it may be by now) and all the responses were in the vein of: "That is not the Rails way and therefore the core team chooses not to support those types of features".

I know people think Java is too bulky & sprawling but after 6 months with JPA, SpringData, Spring JDBC etc. I was kindof left with the feeling "Man, there is a bit of a learning curve to pick up the overarching concepts of the Java ecosystem and its design paradigms at first, but once you are over that hump you can do more, easier with any ORM lib than with ActiveRecord". I know there is another Ruby ORM gaining traction (DataMapper) but it just seems that the opinionated nature of Ruby/Rails core dev teams makes it doomed & dangerous for enterprise use (and indeed maybe its not even their goal).

The Java approach is bulky and design-by-committee for sure but their userbase seems to demand that the fullest possible spectrum of features be supported, and then the library developers try to provide their recommended approach for new projects. Also the specs expand pretty quickly... had to use JPA 2.0 for some projects and found myself constantly frustrated because in the advancement to 2.1 there seemed to be a landslide of awesomeness added to support missing advanced database features.

I guess its one of those things where now it feels like a breath of fresh air thinking "I have more power in my pinky than....". It's a shame too cuz I still get calls from companies in a lurch desperate for RoR devs but philosophically I just can't bring myself to go back in that direction after seeing how unscary & mind-bogglingly powerful the enterprise langs have become. In part thanks to RoR, I'm sure! Wouldn't mind using it for front-end but all that stuff is so interchangeable, the syntax diffs are barely noticed in the development process. I think really the only front-end techs with noticeable differences in feel are the ones with advanced data-binding/component libraries.

Maybe there are some mind-blowing gems out there now though? Who knows....

I got excited by seeing how passionate and excited people like patio11 were about Rails. But then seeing how Rails prefers insecure defaults and had exploits because they were essentially eval'ing user posted data really hampered my enthusiasm.

Seeing articles like this reinforces the idea that Ruby is cowboy programming, just making stuff up as they go along. Like PHP but with class.

Developers in the RoR community have a tendency to say that "Rails is very good for one set of problems (Basecamp), and using it for anything else is problematic". This has been the answer for every attempt that tried to introduce some worthwhile idea to the Ruby community, be it OO design, TDD or something more "enterprisey" like hexagonal architecture. It's not the Rails Way™, move on.

I've heard developers whom I respect (ie. Avdi Grimm, Sandi Metz) shrug this inherent limitation of Rails off, and I find it really weird. It's akin to Stockholm syndrome really.

However, Rails (and specifically ActiveRecord) has a very specific design philosophy (see http://david.heinemeierhansson.com/2012/rails-is-omakase.htm... ) which does not include support for stored procedures (because business logic belongs in the app, not the database).

Of course, there are ways to make stored procedures work with Rails (see https://github.com/leopoldodonnell/uses-stored-procedures ) but they are likely to end in tears, since you are working against the grain of the framework.

The basic argument for Rails is not about support for or against any specific underlying technology (if you can reach Facebook-scale on PHP, then you can make anything work), it is that using a mature and well-crafted framework maximizes developer productivity.

For most startups, as well as most enterprises, that is the critical resource.

The main problem with Ruby in this kind of setting is that these internal tools are extremely hard to maintain because of backwards compatibility issues--which there are plenty due to lack of spec, organic language development etc. The effort to upgrade the Ruby version is much greater than doing incremental hacks to support some half-day feature, so the codebase stays pinned to the 2007 Ruby release, we can't use new gems, have to live with old bugs or missing features, etc. And with every new incremental change the project gets harder to upgrade.

The Java ecosystem has done a lot better in this regard. Scala is not very good either, we have had a couple Scala projects with version lock-in effect as well.

i apologize, i must have been unclear because it seems that you (& some of the commenters below) are dissecting some of my statements quite literally whereas i put a lot of disclaimery type of statements to try to indicate that this is a combo of personal opinion / hazy memory (if the commenter below ever sees this he can also now receive my apology about mentioning datamapper as an ORM gaining traction, that must be a memory from years ago when i still did RoR). :D

I am aware that RoR has been successfully adopted many places, but i guess part of what i failed to explain here (that thankfully foobarian touches upon a bit) is that i believe that part of the power of "a spec" in a programming context is to maintain a solid core of functionality, that then also paves the way for expanded functionality in the future based on extensions of a similar style of syntax.

I know Ruby & Rails teams are not one and the same, I just have many memories of experiences in that community where library evolution is not incremental, does not expand functionality, deprecates as a rival lib rises in popularity.

In Java you see 2 things -- slow spec evolution & incremental expansion of features. It takes time (I know the fast-moving lang communities hate this) but I find it makes for much more robust codebases that survive upgrade processes with wayyyyyy less re-factoring/re-writing, and in type even some libraries that had their own syntax start to embrace/support the spec (Hibernate implements JPA now...).

So.... I suppose, yes, it is a personal matter perhaps. Some companies have used Rails successfully. But in freelance it has kindof bitten me (the overhead of dealing with security vulnerabilities / gem upgrades alone is astronomically more than I've dealt with in upgrading Java projects). I understand that a competent team can handle this & enterprises can pour money into making it work, but as a programmer who often works alone & hates forced unplanned maintenance, it is a tough pill to swallow.

Also, maybe it is because I am dealing with a lot of enterprise/legacy code at times. Sure, maybe business logic shouldn't go in the database. But if someone supplies me a library of highly complex Oracle functions & tells me i need to wrap these into services ASAP, am I really going to begin an immediate re-write? I would prefer that my framework is just able to wrap these functions until a re-write can be scheduled (if the codebase isn't retired before then...). Also in the enterprise community I don't find as much of a desire to have ALL logic in the app layer... often devs feel that as long as a stored procedure / view / whatever returns a meaningful coherent entity that is useful in the business logic layer, they don't really care where/how the query was aggregated.

I also understand anything can be made to scale & work for an enterprise... but it takes enterprise-level support!! I am more a champion of the rogue programmer trying to build an enterprise with a few lines of code a day. ;)

Best, john

He thinks that DattaMapper is trending, when it's clearly not(!!) and doesn't know Sequel (another popular actively developed ruby ORM). Hence it's easy to deduce that he doesn't have a clue about the landscape he is talking about.

Yet he is bashing Ruby as if it's hard to write a module with a couple of complex, custom-made raw SQL queries.

... blah! ...

It would also show leadership from MRI to encourage a semi-independent Ruby self-test framework that can run on any Rubyish interpreter. (Granted RS has RBX bias, but it would show leadership to coordinate common infrastructure.)

Since RS exists, seems like a good idea to go with that for full up integration testing, but each Ruby should still unit test it's own, low-level bits. (Is MRI doing that at least?)

And why would/should they when there are quite literally dozens (or possibly hundreds or even thousands) of developers who will happily work on "making Ruby better" for no cost to said companies?

A large percentage of software developers will quite happily hack away at projects for nothing in return simply because they enjoy it (and would, quite incorrectly, claim that as compensation itself), in order to gain notoriety, boost their own ego, or any of a number of related reasons.

But why should the Ruby developers work to fix 'bugs' against an unsupported 3rd party specification?

This seems telling.

Does anyone have any concrete information about why Matz and the Ruby team are opposed to using RubySpec then?

Has there been any progress on creating an ISO spec?

EDIT: It seems Ruby has a published ISO spec since April 2012 - http://www.iso.org/iso/iso_catalogue/catalogue_tc/catalogue_...

Just read the actual code of the MRI tests.

Java bugs need to stay there because if they were fixed then it would be a backward incompatible change.

I don't get it... the Java way seems pretty right to me. That said, I will allow that they might have been a little bit too rigid about not allowing breaking changes. At the very least this policy has resulted in Java evolving more slowly than it might have otherwise. But I don't necessarily find that to be unacceptable, all things told.

In my experience the Java way is 'oh, a bug, well, that's a bug defined in the spec so we can't fix it now, we'll make a note for the next version'...5 years pass.

The 'I don't want to touch working code' argument is another strange Javaism. If you expect your code to not rot your going to have to constantly maintain it anyway.

Java's language spec exists, I'll give it that. Well designed? It was when it was created but, it has evolved very poorly. Look at C# as an example of what a similar language designed at a similar time could become while still being 'Enterprise'.

Apparently, he and some of the key devs don't like design by committee or much in the way of bureaucracy.

It's troublesome to conflate the bulk of that proposal with the reception of RubySpec itself. It may be that some of the Ruby folks would've been receptive to a more limited proposal like "all released versions of Ruby should pass RubySpec", but we don't live in the universe where that happened.

Even then, telling a group of language developers that they should commit tests into what is effectively your "pet project" is a big ask.

I think that's a very disappointing point of view, and I say that as a someone who used Ruby in the past. In light of this I can say it will not be my go-to language in the future.

I feel like after a certain point of community uptake, a project surpasses any single developer and becomes a part of that community in and of itself. I understand wanting to maintain control of your pet project, but for community projects I feel like there should be community control. That said, you should have the freedom and the right to do whatever you want with your pet project. Maybe the solution is for the community to fork so they can make the changes they see fit.

Given the way the MRI devs and Matz in particular have responded, perhaps it's time for the community to fork Ruby if they don't like the way Matz insists on remaining the dictator. It's his project and he can do what he wants with it, but we don't have to use it.

@yukihiro_matz: @senthilnayagam I am not in charge of testing. But as far as I understand it has been communication problems. Blaming no use.

@senthilnayagam: . @yukihiro_matz when merb could merge with rails, rubyspec shpuld merge with MRI & become official reference for all Ruby implementations

https://bugs.ruby-lang.org/issues/7549#note-2

1. Matz is, and always will be BDFL. 2. The core Ruby team wants to talk in terms of C and MRI being the Ruby spec, not a third party/written RubySpec

Then again, I don't care about Python. I do care about Ruby.

http://www.iso.org/iso/iso_catalogue/catalogue_tc/catalogue_...

Aw crap, I have to buy this doc to see it?

But the ISO document is mostly irrelevant, if we talk about Ruby implementations like CRuby, JRuby, or Rubinius.

Why is RubySpec the special sauce that is so special that the devs not using it is horrible?

Given that, the onus is on the MRI developers to demonstrate why they are actively avoiding the use of a tool that could improve the stability of their language and environment.

Nobody questions RubySpec as a project. But there are many nontechnical reasons why it was never adopted wholeheartedly by ruby-core.

This is a _huge_ stretch. It's hardly surprising that the suite of tests MRI is running against aren't failing against a _released_ version.

Assuming TFA isn't actively lying[0]: because it fixes the issues outlined which allow (amongst other things) cross-implementation and cross-version uses, because it tests behaviours not tested by the MRI suite anyway and because it's there as an extensive test suite for the language.

Oh, and because according to TFA it found regressions in just about every release it's existed for. Regressions which could thus have been caught before the release was cut.

[0] and I don't see any reason to assume otherwise, quite the opposite.

Reading this blog post it sounds like the developer of RubySpec did this:

1. Decided Ruby needed more formal specification 2. Made their own formal specification 3. Kept telling the core developers to use it 4. Is giving up after years of the core developers not bending to the way he thinks that Ruby should work

The whole thing just read as kind of arrogant to me. Why should the ruby developers have to follow his specification? The fact that there is a bug that causes a segfault only indicates there is a bug, I don't see it as an indictment of the core language process.

Instead of saying "you should be using my test", you could commit mew tests and patches to upstream. If some of the existing tests or two large or don't test only one function, why not submit a patch to fix those issues. That doesn't mean the entire test week needs to be thrown out and the language respecified.

This seemed more like someone trying to force themselves into a position and giving up after a few years.

RubySpec isn't a formal specification in any way, shape or form. RubySpec is an implementation test suite, and for MRI a regression suite. It uses language constructs and expects a behaviour A based on Ruby version B (which is itself assumed to match MRI version B).

RubySpec isn't somebody thinking up his own definition of Ruby (let alone doing so formally), it's somebody encoding MRI behaviour into a test suite because he needed that to code his own Ruby implementation.

> Why should the ruby developers have to follow his specification?

There's nothing to follow, there's a test suite to run. A test suite which encodes existing MRI behaviour. Why should ruby developers run it? Because it catches bugs is a pretty good reason.

> The fact that there is a bug that causes a segfault only indicates there is a bug, I don't see it as an indictment of the core language process.

How is segfaulting on existing code not an indictment of the core process when just running an existing test suite would have caught it?

> Instead of saying "you should be using my test", you could commit mew tests and patches to upstream.

Have you missed the laundry sheet of issues in the existing MRI testing system?

E.g. large parts of the 1.8.x series behaved differently from what most people thought it did in terms of the bootstrapping of inheritance of core classes. I wrote a blog post about Ruby behaviour based on introspecting the 1.8.6 interpreter that some people insisted was wrong because it worked differently than intended. Yet the behaviour persisted for years until it silently changed at some point (I don't know which revision).

In that case it was no big deal since the reason the real behaviour was pretty much unknown was that nobody depended on the behaviour, but from what I've seen this is fairly common with MRI.

As a "spec" it is one of the most unstable environments I've worked with. Though I love Ruby the language, MRI is a problem as the default implementation and atrocious as a "spec".

I frankly hope one of the other implementations gains enough traction to force the core team to the "negotiating table" over a proper specification.

Arguably it's no different than something breaking in your current Ruby program, and after spending a bunch of time debugging you realize it is a problem in MRI. If that happens, you would file a bug against MRI. RubySpec just makes it more apparent.

https://github.com/ruby/ruby/tree/trunk/test/ruby

2015-01-02 14:45:30 brixen raise your hand if you have implemented Ruby and used MRI's tests

Glad he didn't get things to go his way. He cries too much. If you don't like some projects go write your fucking own.