Honestly, just the fact that rubyspec exposes a segfault in a new release is more than reason enough that they should be using it. It is de facto a more complete test suite than the ad hoc one they've been making.
Given that, the onus is on the MRI developers to demonstrate why they are actively avoiding the use of a tool that could improve the stability of their language and environment.
They have used and contributed to RubySpec in the past, but many (like me) were turned off by the maintainers' attitudes toward contributors and lack of respect. For example, see the zenspider link elsewhere in this thread.
Nobody questions RubySpec as a project. But there are many nontechnical reasons why it was never adopted wholeheartedly by ruby-core.
> Why is RubySpec the special sauce that is so special that the devs not using it is horrible?
Assuming TFA isn't actively lying[0]: because it fixes the issues outlined which allow (amongst other things) cross-implementation and cross-version uses, because it tests behaviours not tested by the MRI suite anyway and because it's there as an extensive test suite for the language.
Oh, and because according to TFA it found regressions in just about every release it's existed for. Regressions which could thus have been caught before the release was cut.
[0] and I don't see any reason to assume otherwise, quite the opposite.
I wouldn't say TFA is lying but it is presenting only part of the truth. MRI does have an actively developed test suite, which they run in CI and which we use to test compatibility in JRuby. MRI has contributed to RubySpec in the past and many contributors I know stopped (like I did) for nontechnical reasons. They are not ignoring it, but it is a hard project to collaborate on given the attitudes of the project lead that should be evident in this post.
I see what you're saying, but this isn't about using any one person's pet project. This discussion would be moot if MRI Devs were using a different competing third-party Ruby-Spec like tool, but they aren't. They wrote their own tools which contain documented bugs and flaws which they are not interested in fixing, all the while a much better tool exists that they refuse to use.
That's basically how I read this. It seems that Ruby is specified de facto by what MRI does.
Reading this blog post it sounds like the developer of RubySpec did this:
1. Decided Ruby needed more formal specification
2. Made their own formal specification
3. Kept telling the core developers to use it
4. Is giving up after years of the core developers not bending to the way he thinks that Ruby should work
The whole thing just read as kind of arrogant to me. Why should the ruby developers have to follow his specification? The fact that there is a bug that causes a segfault only indicates there is a bug, I don't see it as an indictment of the core language process.
Instead of saying "you should be using my test", you could commit mew tests and patches to upstream. If some of the existing tests or two large or don't test only one function, why not submit a patch to fix those issues. That doesn't mean the entire test week needs to be thrown out and the language respecified.
This seemed more like someone trying to force themselves into a position and giving up after a few years.
> 1. Decided Ruby needed more formal specification 2. Made their own formal specification
RubySpec isn't a formal specification in any way, shape or form. RubySpec is an implementation test suite, and for MRI a regression suite. It uses language constructs and expects a behaviour A based on Ruby version B (which is itself assumed to match MRI version B).
RubySpec isn't somebody thinking up his own definition of Ruby (let alone doing so formally), it's somebody encoding MRI behaviour into a test suite because he needed that to code his own Ruby implementation.
> Why should the ruby developers have to follow his specification?
There's nothing to follow, there's a test suite to run. A test suite which encodes existing MRI behaviour. Why should ruby developers run it? Because it catches bugs is a pretty good reason.
> The fact that there is a bug that causes a segfault only indicates there is a bug, I don't see it as an indictment of the core language process.
How is segfaulting on existing code not an indictment of the core process when just running an existing test suite would have caught it?
> Instead of saying "you should be using my test", you could commit mew tests and patches to upstream.
Have you missed the laundry sheet of issues in the existing MRI testing system?
The problem with "MRI as spec" is that nobody really knows how the language is defined.
E.g. large parts of the 1.8.x series behaved differently from what most people thought it did in terms of the bootstrapping of inheritance of core classes. I wrote a blog post about Ruby behaviour based on introspecting the 1.8.6 interpreter that some people insisted was wrong because it worked differently than intended. Yet the behaviour persisted for years until it silently changed at some point (I don't know which revision).
In that case it was no big deal since the reason the real behaviour was pretty much unknown was that nobody depended on the behaviour, but from what I've seen this is fairly common with MRI.
As a "spec" it is one of the most unstable environments I've worked with. Though I love Ruby the language, MRI is a problem as the default implementation and atrocious as a "spec".
I frankly hope one of the other implementations gains enough traction to force the core team to the "negotiating table" over a proper specification.
RubySpec is nothing more than Ruby code doing things that worked in the previous version. Whenever it breaks, that is a regression in MRI.
Arguably it's no different than something breaking in your current Ruby program, and after spending a bunch of time debugging you realize it is a problem in MRI. If that happens, you would file a bug against MRI. RubySpec just makes it more apparent.
