Good idea, agreed. There's a partial attempt at that in the JS implementation's test suite, albeit written into the test code itself; that'd probably be a decent enough starting point for a non-comprehensive approach.
At some point a full-blown test harness will be useful (i.e. to compare implementations and make sure they have equivalent behavior, for e.g. randomized or sequential tests). Haven't gotten that far yet :)
A test vector file for implementers would be nice (something like what cbor provides) so all possible edge cases can be checked for.