I have a system I built that compiles TS types to JSON schema, which then validates data coming into my endpoints, this way I am typesafe at compile time (Typescript API) but if someone hits my REST endpoint w/o using my library, I still get runtime goodness.

The number of different ways that JSON schema can be programmatically generated and therefore expressed is a bit high, different tools generate very different JSON Schemas.

Also the error messages JSON schema gives back are kind of trash, then again the JSON Schema on one of our endpoints is over 200KB in size.

Having a runtime parser (which can consequently express things impossible with TypeScript types alone, like "a positive integer" rather than being satisfied with any instance of `number`) from which types are inferred, is a needed mindset shift made easy with Zod.

I use JSON Schemas for request validation and response serialisation (eg: [1]) in Fastify, derived from Zod parsers via zod-to-json-schema [2]. Some of Zod's runtime validation does not translate to JSON Schema (typically transforms), so YMMV. But this gives a good runtime glue with the static typing of request handlers.

[1] https://github.com/SocialGouv/e2esdk/blob/beta/packages/serv...

[2] https://github.com/StefanTerdell/zod-to-json-schema

Typebox just creates JSON Schema objects at runtime and projects them into the type system with `Static<T>`. In so doing, you simultaneously create schema and types and the process of doing so is pleasant--you can just hand a TObject to Fastify as a validator object and you're done. Plus, with a Typebox type provider, it infers down to your handler.

JSON Schema is the lingua franca; to me, working in it, rather than converting to it, is a much easier proposition.

In the Zod world, if you need JSON Schema to e.g. communicate to the outside world, you can extract it from the Zod schema with https://github.com/StefanTerdell/zod-to-json-schema -- but if you don't, you don't.

JSON Schema is kind of underpowered for real validation, so if you limit yourself to it, then you'll just have a another round of validation immediately after. TypeBox' s CreateType seems to be the same idea, it cannot be expressed in just JSON Schema.

[1]: https://github.com/fabien0102/ts-to-zod

Also if you start with objects you can express runtime conditions that are not possible with types, for example maximum sizes.

Zod is quite a bit more general than JSON schema because it can express transformations not just validations.

(Or, requires an extra build step to generate that. It's beyond Typescript itself.)

Although in practice the mapped types to do this do slow down the typing service non trivially.

1. Fancy combo of build and compile time generics generated all of our libraries 2. Tooling ran over return values from the generic definitions and created the schema

I rewrote it and now everything is defined in JSON files and those JSON files are ran through a code generator that creates our exported libraries, and we still generate the schema based on the TS exports.

Having everything defined in JSON then allowed us to write tooling on top of the JSON to make changes to our libraries.

Protobuf v3 is horrible, had to start using it recently. The type system is so anemic, it is a joke how hard it is to model things in it.

JSON schema is more powerful than TS, and TS is orders of magnitude more powerful than what can be expressed in PB.

The original generics code was super cool, and obscenely succinct, but it wasn't amenable to being auto generated.

As an extra, our frontend uses the same shared TS code for data transfer objects, this way there's an extra level of type safety at the API.

I’m invested in it. I’m using it to provide implementation-specific validation of requests to/from a third party API.

I wish there was a good macOS editor or IDEA plugin for it with autocomplete etc. The static generators from examples are obscure, ugly, minimal, and can’t account for variations. It isn’t pleasant to write, it’s tedious and slow.

Nevertheless I’d rather write API validation this way, in a document, than in code.

Ass-dragging on 3.1 support aside, the code-generation tools out there are, in my experience, trash anyway. It's a massive hodgepodge of Java-based tools with incomplete and redundant documentation repositories all over the place, and widely varying output quality... when the tools work at all. I appreciate people giving their time to open source, but on the other hand I wasted so much time trying to make the broken tools work that I could have written my own code generator from scratch (which is what I ended up deciding to do after weeks of dicking around with OpenAPI Generator).

Anyway, I was just wondering how you're using JSON Schema for API work (since you didn't mention OpenAPI).

Going into it, I thought we'd be on a well traveled path, but that was far from the case...

Typescript has some decent code generation packages (although fragmented) and request validation can be hacked via AJV... But tooling around C# was halfbaked at best.

In the end, I didn't really feel like OpenApi provided the API "contract" I had originally hoped for.

If I could start over, I would probably use something like gRPC for internal service communication (would maybe still use OpenAPI for any public API)

Using OpenAPI and Stoplight was very useful for thinking the API through and how the whole thing could work, but the code-generation aspect was a total bust. And talking to some new colleagues later who had some knowledge of it, I found that I wasn't alone in my opinion that the tooling is trash.

Even worse, the prevailing opinion on the ecosystem is so poor that it might even be a professional liability to propose (or admit to) using it. The implication was that it (or at least the code-generation facilities for it) is by and for people who don't know what they're doing. Oof.

Half tempted to just start building out OA3.1 stuff to replace those services

And I don't even know Python. I figured it'd be easier to learn it and use it for text processing than wasting any more time trying to make the sprawling Java "tools" work.

The OpenAPI Generator uses Mustache templates and a plug-in-style design to handle all the different languages and outputs targeting different packages. I was never able to find a succinct document explaining exactly how the processing steps worked or even a catalog of data elements that the generator extracted from your OpenAPI spec document (YAML or JSON). So while there's a lot of talk about creating a custom template, or, if that's not enough, a custom generator... the documentation to do so is very incomplete. The only example I found all discussed making a generator for documentation and not code. Again, not to sound unappreciative, but... that's lame.

And I say that as a conscious XML user. We are going in circles. At least XML has comments.

That said, there are plenty of examples. JSONPath, JQ, Jolt...

I am not claiming that this is not useful. Just amused to see the wheel go around.

For a simple, most frequently used subset of jq a more modest, non-Turing-complete language of transforming strings could be created, so that for many useful cases one just need to provide such a string. Strictly for JSON -> JSON.

XPath really started to show the warts here - it leveraged namespaces as an axis, where your XPath could utilize namespaces which were defined at a particular element in the XSLT. However, this went against generalized tooling, as you had to understand which attributes or text were paths in order to know if namespaces had semantic value beyond the XML syntax itself.

IMHO, Canonical XML and XML Infosets were an effort to ret-con the new behavior back in - that namespaces and prefixes were not just used to serialize XML arbitrarily, but were semantically important parts of the XML itself. But at this point XML was too large to evolve - even XML 1.1 had relatively little market uptake.

On top of that, you had issues with how e.g. Microsoft Internet Explorer had divergent behavior for XSLT from other browsers/the spec.

<{http://my.example.com/ns/fruit}citrus> ... </{http://my.example.com/ns/fruit}citrus>

Sure, it's textual bloat, but XML was already verbose and already compressed well with gzip, and it's a lot simpler to process.

The problem with the xmlns:alias="url" rewriting thing was that 1) most things didn't bother implementing it in full, and ended up e.g. relying on the name of the chosen alias 2) it had a special case of changing the current default namespace, which caused even more bad implementations.

I guess you could do something like that with redis but at the time it was magic.

JSON Schema is something that address stability guarantees for JSON - where JSON was invented to be easy to change so it skips heavy XML stuff that was giving stability guarantees.

Now people involved in JSON Schema have problem with decision on stability guarantees - if they skip those or keep it backwards compatible...

because one thing is for sure, json schema and openapi are not really nicely readable, even if written as yaml

Then I can use the same schema in the backend to validate the data, both sent in via the form and directly with the application/json content-type. It's a pretty smooth flow, and reduces a lot of redundancy.

[0] my biggest gripe is it's not well defined what to do with multipleOf when the number isn't an exact integer.

To be more specific, since I wasn't in GP... It's not clear what you should do when for example you want to see if 0.3f is a "multiple of" 0.1f (due to the whole 0.30000000000004 thing)

If the implementation wants to account for some floating point errors I think that's fine too. If they want to codify it though, maybe add an epsilon param so the user can specify how close they want it to be.

Having said that most implementations don't use decimal floating points to represent these.

The point is JSONschema could take a stand and say for example, "this filter will always fail when the operand is not the number representation of an exact integer".

Whether your JSON parser will preserve the precision correctly is another story. For JavaScript/ECMAScript, you'll need to use a library.

   1 == "1"; // true
   1n == "1"; // true
   1n == "1n"; // false

For example, given multipleOf: 0.3, and an input of 0.9, 0.9/0.3 is 3, which is an integer, so would be accepted. A value of 0.8 would be rejected.

And maybe schema URIs weren't exactly the best idea. Not a fan how many libraries make me register a schema with some a fake URL instead of just feeding them the schema document and simply not caring where it came from at all. But since it's already there - OK, fine, it's a minor nuisance.

Otherwise, it just works.

P.S. I have no idea about the relation to OpenAPI/Swagger/RESTful APIs/whatever. I use vanilla JSON Schema as a convenient "cross-platform" DSL for JSON-serializable data structure validation, and I think it does excellent job at this. Would love to see it staying in that scope.

Keeping things compatible is essentially always preferable. Making breaking changes is in isolation a bad thing but can be worth it for what you get out of it.

Is there a wishlist of "if only we could break compatibility, we'd do X/Y/Z"?

> The specifications that have been published to date provide no stability guarantees, but we will be adding explicit guarantees with the next publication. However, it seems disingenuous to promise "no breaking changes" while including breaking changes.

Not really, this seems pretty par for the course of any project hitting a 1.0 release - announcing that things are now stable. Perhaps that's a better framing, you wouldn't release something saying no breaking changes while containing breaking changes, but releasing something that has breaking changes as a stable thing makes a lot of sense.

I have not worked for an organization with public APIs. Not a single one of the APIs I work on is completely stable. Figuring out what our APIs do requires techniques from archeology, anthropology, and sociology.

At one point I did, but then discovered RAML[0] and it subsumed the value of what JSON Schema provides as well as being easier to work with than OpenAPI[1]. Also, generating JSON Schema from RAML definitions has proven to be a fairly straightforward process.

The usual caveats apply... Your mileage may vary, my experiences do not speak for any others, my opinion does not detract from the value of JSON Schema, etc.

0 - https://github.com/raml-org/raml-spec/blob/master/versions/r...

1 - https://swagger.io/specification/

What I didn't like was that it doesn't infer something is an object and instead you have "type:object" repeated a lot. Also properties rather than props.

So I'm not sure if my feedback is valid but, I sure hope that the jsonschema crate follows the spec! Otherwise I'll never use jsonschema but instead something-not-exactly-jsonschema. In other words.. you better not break anything.

pg_jsonschema matches that support with the caveat that it doesn't support loading documents over http since that would be risky behavior inside a database

disclaimer: I'm the author of the pg_jsonschema

I also found this site useful - https://cswr.github.io/JsonSchema/spec/grammar/ .

Can you elaborate please? I particularly like their well formedness concept.

I seem to recall other issues, but can't find them now. We may have raised issues which were resolved, but I don't recall the specifics.

Also lack of inheritance support. (For example I want a way to specify that my json object should be deserialized as Dog not as Animal.)

   (dog){
      "ears": [(ear){"side": "right"}, (ear){"side": "left"}]
      {% this is a comment %}
   }

(This is only half sarcasm)

EDIT: Even Better, make comments a subtree by having a special type tag, like #. this way, (#,dog){...} can stay in the tree, but is ignored.

Use Typescript, then so long as it barks, you can treat it like a dog!

And if it knows how to bark and meow, you can treat it as a dog or a cat at your leisure. :-D

Here's Microsoft's workaround: https://devblogs.microsoft.com/dotnet/system-text-json-in-do...

    "properties": {
      "cat": {
        "$ref": "#/$defs/catType"
      }
      "dog": {
        "$ref": "#/$defs/dogType"
      }
    },
    "oneOf": [
      {
        "required": ["cat"]
      },
      {
        "required": ["dog"]
      },
    ]

    {
      "dog": {
        "name": "Charlie"
      }
    }

[1]: https://json-schema.org/understanding-json-schema/reference/...

[2]: https://json-schema.org/understanding-json-schema/reference/...

https://en.wikipedia.org/wiki/Tagged_union

I mean how MS does it, yeah, you have a tagged union, look at the tag to determine what it is.

Heck even networking packets work this way, UDP or TCP, look at the Protocol Field on the IP packet to see which it is.

We've used { "__type": "DOG" } as metadata flags in GraphQL and JSON API's for a long time.

I rarely see an API return anything other than an ISO8601 UTC datetime string anymore other than Xero's horrific .NET(?) string " /Date(1326480130277+1300)/"

Use custom deserializers for that.

Date type reminds about some network-specific types which used to be in older version of JSON Schema (like IPv4 address?). Today we can use a string with regexp for an approximation. Do we have a mechanism in JSON Schema to define, mmm, dialects?..

Deserialization of a number as an integer, not a double, could be a similar problem. We can agree that we (try to) deserialize to most abstract class (?) or to use external explicit indicators of the class.

Since we have tooling[0] that validates requests and responses at runtime, the clients can be absolutely sure of what they receive (we through 500 if the server attempts to respond with an undocumented respond) And the server is also sure about the shape of the requests. This allows us to validate everything at compile time too, generating typescript types for both client and server.

And since we have similar tooling regarding our data stores (typescript types for sql queries) most of the time if there is a bug, the code would simply not compile - pretty nifty!

[0] - https://github.com/ovotech/laminar

In most cases where I want to do some validation on JSON I find that I usually have a class/struct/object that represents the payload and I want to unpack JSON into it (or dump that class to JSON). Ultimately, there are already nice tools to do this (eg; marshmallow on the python side). So unless I'm crossing language boundaries writing a separate jsonschema and using that is more work and I have to keep it up to date.

And in the cases where I want to cross language boundaries at this point there's less and less of a compelling case to go with JSON and not eg; flatbuffers/protobuf/thrift/capnproto since you're writing a schema anyways.

In theory this is already supported - ish. Because unknown keys at any level of the spec are ignored… but how does one know that those keys will always be unknown? The specification should do something like say certain keys, say, any with a period in the key name, will never be used by the core specification as it evolves. It’s a bit unclear from the spec at https://json-schema.org/draft/2020-12/json-schema-core.html#... IMO. This is why this thread is important!

The application I'm working on predates FastAPI, so I'm using CherryPy and modified their JSON tool to call the parse_obj() and json() on the model where needed.

Any language with a half-decent type system can represent that at the type level. And certainly any language worth knowing can trivially check whether an integer is a multiple of integer X or a string matches regexp R, which is what grandparent was getting at.

If you add events, they may not have FKs, you just add them to the table while generating PKs on the fly. This is useful in many cases. There could be other approaches with different cases, I suspect.

NB: It's actually already possible, since the type system is already implemented, it's just that many features of the language are not yet available.

No, but I don’t do much backend stuff, anymore.

I’m totally anal about Quality, so I want to do things like jsonSchema. I used to deliberately publish APIs in both JSON and XML, so that I could use XML Schema to validate the data, but JSON, to actually use it.

XML Schema is a huge PItA. I don’t like auto-generated Schema (see “anal,” above), so I tend to hand-tune (or write code to dynamically generate) my Schema.

The main reason I don’t use jsonSchema, is that I don’t have a real use for it, these days.

I mostly have internal APIs (proprietary backends), so there’s no need. I do have one backend that I recently wrote, that is public, and I may consider adding a published schema to it.

Yes. In the past I used to use Schematron; it's quite fun and extremely straightforward, although it's better described as a set of rules / tests than a tool to describe the structure of a document.

https://en.wikipedia.org/wiki/Schematron

A Schematron equivalent in JS would be cool. Maybe it already exists?

If I say {"type": "object"}, now my schema will accept any valid object. If I add {"type": "object", "properties": {"foo": "string"}}, now if there is a property "foo", it must have type string. I can still have whatever other properties I want of whatever type.

https://github.com/chapmajs/dynamic_dns

My main interest in using JSON Schema in the above project was security related: this service sits on the public Internet, by nature I cannot restrict the sources that connect to it (road warrior type systems couldn't send DNS updates!). Having a strict schema is another layer of sanitization on what one nowadays must assume is a malicious source.

Making sure there aren't major breakages from one version to the next sure would be nice, yes. We hit some snags as we attempted to upgrade from I believe 4 to 7, especially because we'd have to deploy native apps into the wild to use the newer version, and getting native mobile apps deployed everywhere quickly is an exercise in futility.

So being able to be confident that draft X will work with draft X+1 would be pretty excellent.

For libraries that build/offer specific JSON outputs (maybe like an online editor that dumps out a JSON file on export), I would also expect that product would come with a client library with Typescript types that would give type safe access to the JSON data.

Maybe JSON schema is useful for RESTful resources to provide a payload definition of responses? And I guess consumers could then generate client definitions from the JSON schema? It seems weird to do that at runtime, so I'm guessing it would also be a compile step to generate clients from JSON schema? Or is there an intentional runtime use case?

Are there popular APIs/libraries/etc that use JSON Schema? I don't see a "used by" section on this site which could help folks understand where this sits in the modern software development industry.

- OpenAPI/Swagger (1), which is the industry standard for REST API. Used by Kubernetes among others for resource descriptions - OpenRPC (json-rpc standard) - Most YAML based configuration files.

GraphQL doesn't quite supersede JSONSchema because it doesn't deal with validation. (Something like Cue (https://cuelang.org/) might)

Type systems are not typically cross-language, which is where JSONSchema tends to be used a lot.

There are indeed a lot of openapi-based client generators. This one, for example, has been around for quite some time: https://github.com/OpenAPITools/openapi-generator

(1) Kinda. OpenAPI schema and JSONSchema are largely intersecting sets but there are bits in both that aren't present in the other.

The problem with using it directly was that the validators did not produce very usable output. At one point I found myself using it to validate big tables of AirTable-hosted data that were sent into a system as JSON blobs. Identifying the error in a field, and displaying that error to a user so they could correct it, was quite possible, but was a hack in reverse engineering a specific validator system and parsing and mapping its descriptions about where the error was. This was at its worst when you had to have a subschema that was oneOf a few different things: communicating back what was wrong was incredibly painful, as there was basically no metadata coming back that could point you at the problem, just “this object has to be one of a few things, but wasn’t.” In my case I was able to add a second pass of validation if that error came back, validating a specific subschema depending on what type of thingy had been submitted exactly, then mapping that back to the original input fields, before translating those into hyperlinks where the operator could take action.

It was further compounded by a library that referenced any shared sub-schemas by loading the URI. I had to write my own system to load them up at application boot instead, and lock down the load-over-network capability for sanity’s and safety’s sake.

I didn’t see things that were much better when I attempted to work with it in Ruby, either.

The best way to use JSON Schema in a Python API project, in the end, seemed to be to use FastAPI to output the OpenAPI schema and check that in as an artifact (and test it being up to date in the unit tests) so that diffs in the API were obvious at code review.

(I have the opposite working life to you: I use lots of languages but none are based on JS. So I don't feel qualified to say this, but I don't see how a mish mash of hand written client libraries is a substitute even if you do only work in Typescript. What if I write my own JSON data at the application level? Do I have to write my own mini Typescript library to encode and decode it? It's going to be less concise than specifying the format in a schema.)

It's mildly infuriating. I've taken to using Cuelang[0] to write my own validators based on their specifications. (Using Cue because JSON isn't the only data format I have to support.)

I wish there were an easier way to take some documentation and generate JSON Schema from it. I can take the sample JSON in the docs and generate one, but those samples don't usually contain all the edge cases that the systems complain about, so it's not super useful.

[0] https://cuelang.org/

https://github.com/sshine/library-recommendations/blob/main/...

My current impression is that JSON Schema is nicer in theory than in practice.

Additionally, the value is a "schema" as opposed to a boolean, meaning you can apply a subschema to any additional fields. Useful if you for instance want to make sure any additional fields start with a specific prefix.

Booleans are valid schemas. It's subschemas all the way down! I wrote an article around this topic: https://json-schema.org/blog/posts/applicability-json-schema...

[1]: https://json-schema.org/understanding-json-schema/reference/...

If you want to have comments, consider using YAML for definitions where possible as it is a superset of JSON[0]:

  YAML is also a superset of JSON, so JSON files are valid in YAML.

I cannot say the below holds for all YAML libraries, but can say the below works.

Assume there is a file named "foo.yaml" which has the following content:

  {
    # this is a comment
    "hello" : "world",
    array : [
      # first number
      1,
  
      # second number
      2
      ]
  }

  ruby -ryaml -rjson -e 'puts JSON.pretty_generate(YAML.load(ARGF))' < foo.yaml

  {
    "hello": "world",
    "array": [
      1,
      2
    ]
  }

    cue export foo.yaml --out json

    cue export cue: - --out json < file.json

Regarding the ability to add comments to JSON, it can be done in a cumbersome manner, such as adding a field to an object called "comment", but it is a hack since this wont work for arrays anyway.

JSON is not a data structure. JSON is a data language. A computer program is a data structure and doesn't need comments, a programming language is a format for a human to read, write, and interact with automated tooling, whoss content is a program, but for the human uses, comments are important.

Similarly for data languages.

As is often the case, good domain knowledge can help to choose which features are "real" and can stand and which are more doubtful and have a higher chance to fall out of favor later.

Personally, I'd like to see it become officially supported. We have work to do!

[1] https://www.umami.recipes [2] https://schema.org/Recipe [3] https://github.com/google/schema-dts

Part of my motivation to build the tool was to learn more about JSONschema.

like you would expect that people realize "additionalProperties" are additional to "properties" but pretty much any doc or code gen tooling I have used has some problems with getting that right (various problem not always the same)

I mainly seem to have no leapfrogged the need for this

There was a time when the JSON parsers across different platforms were so finicky that schemas seemed like a solution, but the parsers got better at not breaking on unexpected data types or JSON structures

I'm also usually able to do system design to return a smaller subset of data, with consistent data types

Or someone else's JSON or JSON output has its own documentation that already tells what something is and how it should be parsed