If the API that the author is building is a REST API then the response for a non-existing resource is 404.

In case of REST the main idea is that if you try to GET a resource by ID then you assume that the resource exists. Thus if it does not exists => 404.

It does not matter too much which part of the URL is the one that is causing the URL to be wrong.

Thus `api/v11/employees/1` and `/api/v1/employees/100` both are wrong resources.

In the first case, asking for `/api/v11/employees/1` is not a search or find to see if there is a V11 and if inside there is employee 1. Building an URL is an intentional choise that includes assumptions, like there is an 'api', that has a 'v11' and inside there are employees and one should have the ID 1.

The same goes for later case with employee ID 100. If you ask for an employee with an ID that means you know (or assume that you know) that employee exists. Thus if it does not it should be very level clear => 404.

In both cases responding with 200 means something like "I kind understand that you want an URL that is similar with some that I have so it is almost okish".

But in REST this is not the case. It is like you are serving some static folders and you want to get the file 100 under /api/v1/employee and that file does not exists.

Nobody is stoppping anyone to add response body to a 404 to indicate which nested resource is invalid. That can be added as a debug message for the developer for example.

Of course this is IMHO and I am only addressing REST API. If the API is not supposed to be REST then do whatever you want but make sure you document it well and be consistent.

Yes, `api/v1/employees/100` should return a 404, because that path represents the location of a specific entity and that entity does not exist.

Just as the author thinks it's clunky to return HTTP error codes representing application errors, I think it's clunky to apply application logic to HTTP method semantics. GET, POST, and DELETE were designed as instructions telling a web server how to handle static-ish content, and it shows in their design. Why would a GET have a body? It's a request for the server to return a specific file. This leads to breaking REST standards - for example, search endpoints that logically are GET calls (they return entities), but are implemented as POST methods because the search criteria is complex enough that you wanted a request body. Or bulk delete calls that are similarly implemented as POST methods.

REST works best in a "rules are meant to be broken" manner in my opinion. It's not a bad system (which is why it is so common), but mixing application logic with HTTP transport logic does lead to oddities.

There have been some attempts to extend the list of http verbs to include something that fits those use cases more naturally - SEARCH and QUERY. QUERY got more traction, if I remember correctly, but I haven't heard much about it in a while.

https://www.w3.org/2012/ldp/wiki/Proposal_for_HTTP_QUERY_Ver...

That’s probably part of the reason the more general one proposed for HTTP is “QUERY” and not “SEARCH”. Also, WebDAV, in following its apparent design philosophy of “more is more”, has multiple different GET-with-(required or optional)-body methods with different purposes: PROPFIND and REPORT as well as SEARCH.

A GET request to that path should return the current state of the resource at the path. As you note, the current state of that entity is that it does not exist.

Let's look at the spec:

>200 OK - The request has succeeded. The information returned with the response is dependent on the method used in the request, for example:

> GET an entity corresponding to the requested resource is sent in the response;

Did the request succeed?

Yes, we found the current state of the entity so we return 200

What do we return?

"An entity corresponding to the requested resource". In this case however we want to represent an entity that doesn't exist.

There is no resource at the path.

> As you note, the current state of that entity is that it does not exist.

A thing which does not exist does not have state. Existence (and even moreso non-existence) isn’t a state, existence is logically presupposed in any description of state.

Then please explain how you can write and I can understand the phrase "A thing which does not exist".

Who said you shouldn't? The shorthand is convenient. But it doesn't mean what other statements with the same shape mean. A thing which isn't doesn't have a state or representation.

Many online REST discussions focus too much on shallow linguistic analogies with verbs/nouns.

If you were doing a static website serving from a file system and said the same thing about a filename as a path component, you'd be seen as mad. But there is no difference when there is an id for a potential employee than a name for a potential file: if no employee/file exists when the server looks for the id/name, the correct result is 404 Not Found.

The id isn't the resource, it's part of the path to the resource (and if it is the resource in that URL, you need a different scheme for when you need to find the employee pointed to by the id and not info about the id.)

The content of the file `/api/v1/employees/100` need to be defined off-band somehow since you cant actually put an employee on disk.

moreover part of REST is that you are not responding with static files by path, but with representations of entities; the question then is whether the entity is the employee or the id.

I agree that `/api/v1/employees/100` is not the best name for the id entity, something like `/api/v1/employee-id-status/100` might be more descriptive.

In "true" REST, if you have a "search" endpoint, POSTing to it should create to a resource, which could be a reference to the list of results. The created resource should have its own endpoint (something like `/search/results/:query_id`), and the client could then cache it.

If you don't want to do that, but still want to say you are really following REST, you could have an endpoint to represent your resources (`/products`) and use GET with filter parameters in the query string (`/products?search=my+search+term`)

This is _specifically_ about HTTP APIs. REST is not a synonym for HTTP but there are much better resources out there that rant on about the important of hypertext and URL support etc.

This is mostly about the perspective of consumers.

>It does not matter too much which part of the URL is the one that is causing the URL to be wrong.

Your consumer disagrees, they'd like to know if their URL was fat fingered or if a record was missing. My argument is 404 is inappropriate because the web service exists, but the record doesn't.

> Thus `api/v11/employees/1` and `/api/v1/employees/100` both are wrong resources.

I can't say this is wrong, but it doesn't feel right. `/api/v11` straight up resolves to nowhere. Maybe this an instance where Gone is better than Not Found?

> Nobody is stoppping anyone to add response body to a 404 to indicate which nested resource is invalid. That can be added as a debug message for the developer for example.

100% agreed. It's just a thought I had while debating with my team.

I posted this here for this exact kind of feedback :D good points raised

So I will focus on HTTP then.

Say you install an nginx/apache and then have a static structure where you have the profiles of employees saved as PDFs directly on the disk.

Then you do `GET /public/v1/employees/1.pdf` and it works returns 200 with the content. Then you do `GET /public/v11/employees/1.pdf` in this case all servers will return 404. The same goes for `GET /public/v1/employees/100.pdf` will still be 404. What if someone asks for `GET /public/v1/employeea/1.pdf` the server will again respond with 404.

Then I go and I implement an webapp to replace that. I plan to keep the URLs the same but now there is an app that will return the .PDF as a datastream or file.

For me, I don't see any reason why to change the behaviour of the URLs just because I replaced a static app with a dynamic web app. Any HTTP server will respond in the same way thus the current one should respond the same.

Responding like this has a compatibility (let's say) reason behind that is not a personal nor related specifically to my project.

It’s worth being redundantly explicit that this does not extend to all cases. There are cases where a 204 is warranted. But I’m roughly convinced that it may not be as ubiquitous as I thought. Very rad.

>Then you do `GET /public/v1/employees/1.pdf` and it works returns 200 with the content. Then you do `GET /public/v11/employees/1.pdf` in this case all servers will return 404.

Which is a sensible default because the most nginx/apache can conclude is that the resource does not exist on the server. However, if we know that this server is the canonical record for these pdfs we can conclude that it doesn't exist if it's not on the server. So now we know the state (it doesn't exist) and can return its representation.

What does 200 means with regards to existence/non-existence.

I think 200 means something exists and 404 is the representation of non-existence.

You think (please correct me if I am wrong) that the existence or non-existence information should be in the body.

Actually I think the underlying (and more important point) is about how valuable the existence/non-existence information is for the client => how quick the client should have feedback about this?

I think existence is a very important information and thus should be a first class citizen of the data representation. Thus I want it in the status code on the same level with the body itself.

If you put it in the body then that means for me an extra step to parse the body and then see what is there. So then the existence/non-existence is on the second level.

In your case with responding with 200 + body then the 200 status becomes irrelevant and I always need to parse the body => time is lost to access the _first_ important information that should then guard my business logic to parse or not the body.

While in my case (using 200 and 404 status) the client receiving 404 knows directly (without any parsing of the body) that the request was not successful in retrieving existing data.

But that's not what the spec says:

>200 OK - The request has succeeded. The information returned with the response is dependent on the method used in the request, for example:

> GET an entity corresponding to the requested resource is sent in the response;

>The 404 (Not Found) status code indicates that the origin server did not find a current representation for the target resource or is not willing to disclose that one exists.

404 is not the representation of non-existence. It's the representation of not found. Something can be "not found" for many more reasons than non-existence. Which ultimately causes the person integrating your API much consternation because they have no idea if it's a "Everything worked" 404 or "My DNS is borked" 404 or "Your server's routing is borked" 404 or half a dozen other possibilities. Sure, you might add further information to your 404 response but that means you can't have generic 404=bad monitoring. Plus causes headaches for people that are working in systems that do assume 404=bad.

200 means that the request has succeeded. And in these cases it has. You requested a representation of employee 100 and you're getting one (it doesn't exist).

Even if you disagree with the word smithing the latter is far far easier to work with.

Originally, anything with a path was meant to simulate a directory tree of static files. We build it dynamically because that's easier to maintain. But making it look and act the same by returning 404s is historically correct.

Of course things evolve and move on. You're free to do as you wish. But to me you're making a bizarrely arbitrary distinction about what part of the application is allowed to return a 404 (routing code in the framework) and what aren't (your own code). Or did you not realize that a framework like Django isn't actually part of the webserver?

Your article is entitled "I've been abusing HTTP status codes" ... but... you're not "abusing" them, you're "not using" them for your APIs. (Or, said another way, you're leaving them to their normal usage for HTTP servers.)

Thus -- as REST is /the/ canonical "hijack HTTP status codes to mean something clever" paradigm -- your article is /entirely/ in context of REST even if you avoided mentioning it.

...

Anyway - I'm entirely with you on the foolishness of using 404 to mean both "your URL is messed up" and "I couldn't find the resource you wanted".

It's doubly not. The REST Architectural style is (1) protocol neutral, rather than specific to HTTP and (2) emphasizes using the underlying protocol, whatever “as is”.

> Anyway - I'm entirely with you on the foolishness of using 404 to mean both "your URL is messed up" and "I couldn't find the resource you wanted".

But those are literally the same thing. A URL/URI is a “Uniform Resource Locator/Identifier”.

“I don't have a matching resource” is a 404 (unless you are distinguishing “I had a matching resource but you missed it and it's not coming back”, which is 410.) While you might use a body message to distinguish “I would never expect to have a resource with that shape URL” from “I have resources with URLs shaped like that, but not that particular URL”, both are within the usual, RFC-defined meaning of the 404 status code.

If you hit an endpoint and get a 404... did you do it wrong? Is my documentation outdated?

Even better: What recourse do you have? how do you figure out the answer?

Your only recourse is to email me. Send me cURL commands and screenshots and sit on your thumbs until I write back.

IMHO the REST folk were blinded by the existing 404 normalcy set up by web servers.

...

personally I think OP's idea isn't great. I think returning 200 and making me parse the response and hoping it's consistent between services is too much work compared to the simplicity of the HTTP response.

Instead, I'd change the default from 404 to 501.

    HTTP 501 - not implemented (URI is not working)
    HTTP 404 - resource not found (Joe doesn't exist in db)

So instead of

    GET /api/v1/employee/100
    Accept: application/json

    GetEmployee /1000
    X-API-VERSION: 1
    Accept: application/json

If making an RPC and restricting yourself to the known HTTP methods, the closest is

    GET /api/v1/employee?id=100
    Accept: application/json

Sure, you might want information in addition to that provided by the status code. And, again, rather than reinventing the wheel with some ad hoc mechanism, you can follow the HTTP spec for a solution: almost all HTTP status codes support a response body to communicate additional detail.

> Instead, I'd change the default from 404 to 501.

5xx errors indicate server problems, not request problems. If you wanted a different status code for “that path isn't structured in a way I understand” vs “I understand how I would look up something with that path but can't find it“, 400 or 421 for the former would be better than anything that is not a 4xx since they each (1) are in the correct class and reports a client error, and (2) have a definition which arguably fits the scenario, even if 404 arguably fits better.

The main "issue" is that 404 is the normalized response for web servers when an endpoint doesn't exist. So it feels like one is breaking the established paradigm by using something else, but I think it's absolutely worth doing.

Certainly a bigger fan of defaulting to returning a 409 than making my API consumers parse all my response bodies.

Oof, that's a hell of a good point. So much for that plan lol

> Anyway - I'm entirely with you on the foolishness of using 404 to mean both "your URL is messed up" and "I couldn't find the resource you wanted". Seems like, for REST, you'd want to return a 400 (malformed request) or something if your URL was borked rather than overloading 404.

Yup, that's the headache I'm trying to muddle my way through.

Really it's less "this is how to build APIs" and more "have you considered your consumer when you return data?". But I think even in that context your point stands better.

Back to the drawing board it seems.

At least I can generate more content now :D

This is most appropriate when the URL does not make sense with the current state of the server, but other future operations could (in theory) change the server state such that it does make sense. This might make sense if you have a user-extensible data model, where some kind of relationship is being mapped into the path structure and you want to signal that this relationship is not currently known to the query system, but _could be_ in the future.

Now, you are faced with a decision for when this ephemeral status is a semantic conflict, when it is simply a resource not found, or when it is a forbidden request for the current user.

The last is subtle and depends on other security posture. Do you want to tell your user "this is possible/available for a sufficiently privileged user, just not for you" or do you want to avoid leaking information about higher privilege roles? This is similar to the debate about whether a login UX should tell you whether you have an invalid user id versus password or just say login failed without leaking more information to a potential attacker.

Why?

How often does this really come up?

Who is typing in URLs like this manually?

If you're typo'ing it in code, are you not doing any kind of validation/testing against any kind of spec that can catch this?

Why is it up to the actual webservice returning a 404 to catch these kind of typo errors?

And I'm not saying I disagree with the argument -- I fully get the argument that was made, but practically the fact that you're caring about it suggests you're missing other components in your stack. You're producing a URL request which is outside of the spec of legal URLs for the webservice. You can validate that before you ever make a real web request against a real server.

You can’t ignore the http verbs, so your article doesn’t make sens. You also shouldn’t ignore status codes.

You’ll also might get in trouble with caching.

You can easily use status codes, and provide all the detail + status field in the response. It makes consuming a lot easier

Gone means a resource identified by a URI existed, it no longer exists, and that resource (not the URI, necessarily) will never be available again.

The main idea of the article is exactly that he does not agree that http error codes be used for application level errors (as REST principles recommend).

I think at this point that ship has sailed. Most HTTP based API will use http error codes in various ways. I would be surprised to not receive a 404 when requesting a resource ID that does not exist for example when consuming a new API.

If you were talking about TCP/IP, then perhaps - because much of the internet's infrastructure is hard-coded (even burned into silicon) to use current standards. But application semantics aren't frozen in stone by any means.

Servers and clients built atop HTTP are still being written. Why not adopt better patterns that apply properly separate semantics for each layer?

I liked the REST ideas when they came out, particularly because they provided a much better and simpler alternative to SOAP. But I think improving patterns where we can is always a good idea.

Either way, a proper api doc/spec should make either approach a non-issue.

Personnally, I've switched to graphql where it makes sense. Application level error codes and handled on the graphql layer, not on the HTTP layer, so you could say I've adopted that approach!

This is actually more than a little scary.

If you're configuring the API client, and typo the base URL, your next automation run could decide that no resources exist anymore, and delete all kinds of things from your database.

I would really recommend designing APIs that communicate more than just the 404, and writing clients that check that extra part, to differentiate between "something is wrong" and "you requested item of type X from api Y, and that does not exist". If the client got a response that doesn't explicitly state X and Y, it should assume misconfiguration and give a more general error.

The response body could be used for this distinction, if you want to stick with 404.

The problem is that this error then overlaps with server path routing issues, DNS problems, and general network issues. Even if it's logically correct it makes dealing with your API annoying.

>Nobody is stoppping anyone to add response body to a 404 to indicate which nested resource is invalid.

But then we've lost standardization which is the whole point of the error codes to begin with.

Returning 200 and then using response body to denote missing resource is no different. So you have to choose, either 404 can be invalid path and missing employee; or 200 can be valid data or missing employee. Personally I would prefer the former as 200/OK then indicates success.

404 Not found is not exactly correct. I argue that the server found a representation of the object with that representation being "It does not exist".

If you try to request a file `/public/pdfs/100.pdf` and that does not exists what does the server respond? 404

What the server responds if you try `/public/dpdfs/1.pdf`? Still 404 as that path does not exists on the local storage.

What is the difference for a client if 100.pdf should be an actually file or a data stream served from a web framework? There should be no difference.

Choosing to behave when building a dynamic app the same way as the static helps a lot with integrating with multiple other services (eg. caching, observability ...)

>The 200 (OK) status code indicates that the request has succeeded. The payload sent in a 200 response depends on the request method. For the methods defined by this specification, the intended meaning of the payload can be summarized as:

>GET a representation of the target resource;

>The 404 (Not Found) status code indicates that the origin server did not find a current representation for the target resource or is not willing to disclose that one exists.

If you to give the client the representation of the target resource (i.e. it doesn't exist) then send 200 and a body indicating it doesn't exist.

If you want to tell the client you couldn't find a representation for the target resource then send 404

Let's say that you have a nested resource/path that will return devices owned by the employee so you will have this path:

`GET /api/v1/employees/1/devices/1` and this might return the first device owned by the employee.

Now what do you think the following request should return:

`GET /api/v1/employees/100/devices/1` where the employee with ID 100 does not exists but there is a device with the ID 1 owned by some other employee?

or

`GET /api/v1/employees/100/devices/1000` where both the employee with id 100 and device with id 1000 does not exists?

What do you think the response should be? Still 200?

So you're asking for the device with id 1 owned by employee 100. The answer is that the device exists but is not owned by employee 100 because there's no employee 100. So return 200 plus however you want to represent "the device exists but is not owned by employee 100 because there's no employee 100".

>`GET /api/v1/employees/100/devices/1000` where both the employee with id 100 and device with id 1000 does not exists?

Same as above but subbing in "because employee 100 and device 1000 don't exist" as appropriate

Nonexistence is not a representation of the target resource. A resource that does not exist does not have a representation.

It means that the server did not find a current representation of the resource and that the server accepts responsibility for providing an authoritative answer to whether it exists (if the latter is not the case, the most correct response is 421.) Aside from that and the combination of being unwilling to provide a default representation, not having a representation consistent with the Accept header provided by the client, and preferring not to distinguish this case from non-existence with a 406 response (which, like the situations to which 421 applies, is an edge case), the reason for a resource not being found is overwhelmingly that it does not, in fact, exist.

It is true that there are some other things that a 404 might mean, but “does not exist” is not only within the space of things covered by “Not Found”, it is by far the most common reason for it.

The problem is that this error then overlaps with server path routing issues, DNS problems, and general network issues. Even if it's logically correct it makes dealing with your API annoying.

As others have mentioned, you can return anything you want in the body of a 404 to clarify. However the other issues you all mentioned should really be in the 500s with errors.

This isn't how the world works though. Much development happens on COTS platforms or internal frameworks or whatever. Many of those will have generic error handling, logging, alerting, etc. based off the response code.

Your pedantically correct 404 with a clarifying body ruins my day (really weeks) because now I have to chase whoever built those things to fix my problem. I am annoyed and curse your name.

One topic the article didn't even touch was flagging bad parameters in the request which didn't make sense on the application level. HTTP has 422 for that. I commonly write things like

    validate_my_param($c->req->params->{format})
       or $c->detach(HTTP => 422, ['Invalid format']);

I've used that for years and been quite happy with the downstream results.

Edit: I should elaborate on the downstream results.

On the javascript side, the ajax methods often contain separate success and failure callbacks. You often need to handle the failure callback to ensure the user knows that something broke. If you also have an error path in the success callback, it clutters the code.

    ajax({
      success(data, ...) {
        if (data.wasActuallyAnError) {
          // lines
          // of code
          // to handle
          // the error
        }
        else useTheData(data);
      },
      error(response, ...) {
        // lines
        // of code
        // to handle
        // the error
      }
    })

    ajax({
      success(data, ...) {
        useTheData(data);
      },
      error(response, ...) {
        // lines
        // of code
        // to handle
        // the error
      }
    })

Oh sure. I'll just call up the CTO and tell them to scrap the product they spent $10 million on because it's not pedantically correct in parsing HTML codes.

Yes, and then parse the appropriate response payload, in the context of the response code. Thats how eg. you get validation errors for a submitted form that just failed submission. If you're just assuming that everything !=200 is an error, it is your own assumption, not a framework shortcoming. As an example, most of the complex API systems I've developed would actually ignore completely the response code (so, basically the opposite of what you assumed), because the responses need to have further context information in case of errors, such as application-specific error code, context-specific error messages (eg. form field errors) or just translation-aware detailed error messages.

On a vaguely related note, most people working with HTTP API implementations seems to forget that GET requests can in fact have a request body. Most high-level clients/libraries will assume you won't use it, but it doesn't mean you can't use it.

No webserver will ever return a 404 for any of those cases.

Why, in all fairness, we always knew. It's an overloading of an existing technology, a bit hacky as it is, and stuff like GRPC are the answer to this, but still the simplicity of HTTP and REST are the demise of standardization.

REST is an architectural style, not a standard.

> It's an overloading of an existing technology, a bit hacky as it is

REST is not tied to a particular technology or protocol, and is not “overloading” whatever protocol(s) it is used with; the architectural style specifies using the subset of the available features of the underlying protocols that corresponds to REST semantics consistent with the specifications of those protocols.

There is nothing in the protocol that mandates only 2xx status codes are parsable. Instead, the Content-Type pinpoints to the client what kind of content the body has, regardless the status code. And the content-type will most probably be what the client asked (if not, then the server announces that the response content is not subject to negotiation, something that you see rarely nowadays..)

In general I think this post reflects one of the mentalities that appeared around 10 years ago regarding HTTP APIs (another was the extreme hypermedia-driven APIs). But I really think nowadays we can do much better, we can deliver a practical API that works 99% of the times.

By the way in the latest HTTP RFC (RFC9110) status code 422 is finally officially defined (previously was part of webdav extention), which means that the war 422 vs 400 for validation and other semantic errors is over and we have a clear winner. But 200 vs 404 for something that was not found? Well that was has ended long ago I think..

Who is “they/REST”, and where have “they/REST” aligned on anything. All REST says is “use the underlying protocol without messing with its semantics, but only the parts that correspond to REST semantics”.

If the resource exists and is accurately represented by no data, then 204 fits for a gET, but that’s a different case than where there is no matching resource to the URL. 204 is mostly for modifying actions (e.g., PUT, DELETE) that don’t need to return more than the fact of success.

I think a kinder reading of this point is "a 2xx response means you can parse the thing you were expecting to parse"

Often this it to get around the inability for some cloud-based load balancers to accept 5XX status codes as healthy responses. Take AWS ELB/ALB, for example. There are conditions under which a target instance knows the underlying issue isn't related to its own health, such as during planned maintenance, upgrades, or database failures. In these situations, it would be desirable to return 503 and configure ELB/ALB to accept that as a healthy response. Since AWS won't let you do that, some applications will just return an empty or bogus 200 response during upgrades or maintenance.

But then it would be both, wouldn't it?

I know I have a JSON payload representing a domain response because of the 2xx response code AND the Content-Type header?

If a resource isn't found for any reason, 404, 410, or 451 is the correct response. If you want to clarify why it's not found, that should be included in the response body. Don't return 200 while simultaneously reporting an error--that's just bad form. 2XX means everything is good, 4XX means problem on my end, 5XX means problem on the API's end. It's an easy way to tell at a glance who's likely at fault. Yes, status codes are always going to be ambiguous, but that's why there are response bodies alongside them. If the Content-Type header is something you recognize, you can at least attempt to automate that disambiguation process.

Nitpick, but 421 should be on this list, although the circumstances where you would need this should be extremely rare.

It doesn’t though.

Even with an Accept header on the request, it is not impermissible for a server to return a 2xx result with a different format indicated by the Content-Type header if it cannot, for whatever reason, return a result in a requested format (it may also, consistent with the HTTP spec, return a 406 Not Acceptable or, if it wants to be cagey about a resource representation existing if there is none matching the Accept header, 404 instead.)

If you want to know whether and how you can safely parse the response, the HTTP compliant way is to read the Content-Type header. Otherwise, you are relying on assumptions (which may be valid for out-of-band, non-HTTP reasons) about behavior that are outside of the spec.

Did I mention that httpClient, by default, doesn't let you get 404 error bodies?

But ultimately, it is all just ideas on 'how neat it would it be to use codes!' when in practice it is so so much better to just drop it and use the codes for more literal values. Imagining a users/{X} as a 404 for invalid 'X' is fun..but like..the server actually defines it as something like /users/:userId and it isn't actually an invalid route and will not be caught by the default 404 handling. It's a conceptual dance.

Why would a http client not let you get http response bodies for statuses that usually send bodies? I could understand it for a 201, and definitely for a 204, but for a 404 it just seems like bad design of the client.

Unfortunately, the way 422 is written implies that the sent body/content has error(s) and not the header. It's close, but I still feel that for GET requests it's wrong.

> The 422 (Unprocessable Entity) status code means the server understands the content type of the request entity (hence a 415(Unsupported Media Type) status code is inappropriate), and the syntax of the request entity is correct (thus a 400 (Bad Request) status code is inappropriate) but was unable to process the contained instructions. For example, this error condition may occur if an XML request body contains well-formed (i.e., syntactically correct), but semantically erroneous, XML instructions.

https://datatracker.ietf.org/doc/html/rfc4918#section-11.2

Additionally, bodies are allowed on GET requests by the standard though are not commonly used because of bad middleboxes. However, many GET requests include query params or other parts of the url to be parsed, and its completely reasonable interpretation of the standard to return 422 if those are not well-formed according to application rules.

"HTTP messages often transfer a complete or partial representation as the message "content": a stream of octets sent after the header section, as delineated by the message framing." - RFC 9110 (https://www.rfc-editor.org/rfc/rfc9110.html#name-content)

Earlier standards even used the "body" and "content" in the same context:

"The presence of a message body in a request is signaled by a Content-Length or Transfer-Encoding header field. Request message framing is independent of method semantics, even if the method does not define any use for a message body. ... When a message does not have a Transfer-Encoding header field, a Content-Length header field can provide the anticipated size, as a decimal number of octets, for a potential payload body. For messages that do include a payload body, the Content-Length field-value provides the framing information necessary for determining where the body (and message) ends." - RFC 7230 (https://www.rfc-editor.org/rfc/rfc7230#section-3.3)

This is probably due to historical reasons - MIME (email standard) uses "content", the original draft uses "body".

> and the syntax of the request entity is correct (thus a 400 (Bad Request) status code is inappropriate) but was unable to process the contained instructions.

Emphasis mine

Reason being that any intermediary step in the chain of resolving your request can return the response. If your load balancer has fallen over? You'll get a 500 with a plaintext body. Don't parse that as JSON.

(Technically, any intermediary step might also return you a 2xx with a non-parseable body, but that scenario is far, far more rare... Mostly see it in completely misconfigured servers where instead of getting the service you're trying to talk to you get the general "Set up your site" landing page).

You should never assume format based on status code at all! You should detect it based on the Content-Type header.

> You'll get a 500 with a plaintext body. Don't parse that as JSON.

Any intermediary which returns plain text with an application/json Content-Type is badly, badly broken.

In approximately a decade of working on JavaScript and TypeScript UI code, I can count on one hand the number of RPC handlers I've seen that inspect Content-Type in the success codepath.

... for that matter, I can count on one hand the number of RPC handlers I've see nthat inspect Content-Type in the failure codepath also. The common behavior is to dump body to text, log error, and recover.

There’s absolutely something in the protocol, and you should absolutely use the Content-Type header to determine whether there is a body and whether it is in a parseable format irrespective of the status code, except for the minority of status codes defined as not having a body (e.g., 204 & 205.)

I think my overly defensive view of this for real-life code is that error states are inherently situations where the normal code contracts break down and that I must make fewer assumptions about the response, for example that they are well-formed or even match the requested content-type.

The number of times that I've encountered a JSON API that can suddenly return HTML during error states or outages is too damn high. So unless you give me a 2xx I'm not going to assume I got something intelligible.

I think that you should always assume that an HTTP response is a well-formed HTTP response (otherwise you can't even trust that the 404 itself is correct); and you should never assume that the received MIME type is the same as the ones you indicated you accepted; you should always check the Content-type header.

You should detail the error further in the 404 response, and you can say "this entire path prefix could not be found" or just "the path prefix is fine, but this specific employee could not be found". The spec does not prevent you from informing that only the representation of that specific resource is not found. [1]

Using 200 for that is a bit of a cop out, you lose all of the semantics of the response and it's all pushed to out-of-band definitions (like your API's documentation). It could just as easily be argued in the article's example that every single 4xx error is just a 200 with a '{"result": false, ...}' body.

---

[1] - https://datatracker.ietf.org/doc/html/rfc7231#section-6.5.4

Returning a 200 is counter-productive because now any sort of logging, metric system, or middleware is going to be ineffective unless you write some custom code to parse out the body and decode whatever custom error resource the developer decides to invent. And in that case why are you even using HTTP as a transport protocol in the first place if you're going to violate 30+ years of precedent.

My highest scoring StackOverflow answer [1] (and my most controversial) is on exactly this topic and _no one_ in any competing answer have given _any_ sensible explanation of their reasoning. I've genuinely never understood their thought process, despite trying really, *really* hard. This explains it a bit, even though I totally disagree.

There are other tools for helping differentiate different varieties of 404 or 401. There are status codes, there's _nothing_ preventing you from returning a payload with explanation or overriding the Reason Phrase. If your HTTP client is too fragile to handle a very common usage pattern for HTTP status codes then pick a different one.

---

[1] - https://stackoverflow.com/questions/11746894/what-is-the-pro...

I've said it in other responses, but maybe I should have been specific that I was referring to HTTP RPC (explicitly not citing REST for a reason).

Perhaps a combination of both is best, but in my experience a nice clean break between these 2 layers at the protocol level helps clients simplify their handling of responses.

2xx: Go ahead and parse (another commenter pointed out Content-Type. That one always gets me)

4xx/5xx: Something bad happened and the request wasn't processed by the server, either because it blew up (5xx) or because you did something _very_ wrong (4xx)

The objective is clarity to the caller.

It's a little obtuse but I think it works well with that objective in mind.

The query portion of a URL is, in fact, part of the URL, so that is encoding the employee id in the URL, and REST doesn’t care about how URLs are constructed, in the first place (in fact, excessive concerns about out-of-band knowledge about how URLs are constructed mostly are a sign of not doing HATEOAS, in contravention of the REST architectural style.)

And instead of "if its not rest" I meant (in the context of the article) "if it's HTTP RPC" because the author clarified that they were talking about RPC rather than REST.

Remote Procedure Calls typically identify "endpoints" such that a specification of the endpoints could be compiled into a host language's function-calls. The logical way to do that with HTTP is to have a path be the function, and HTTP parameters (whether url or posted) be the parameters of the function.

Using a URL that has both a resource ID encoded in the path and also query parameters is a mixture of REST and RPC. Since the example given in the article does not show parameters, I think it is reasonable to call it REST, as many other commenters here have inferred. The author claiming it is RPC doesn't jive with my expectations for RPC.

Maybe I'm just complaining about calling it RPC. If it's neither RPC nor REST then it's just some random mismash of HTTP-ish stuff, and I don't think that's a particularly strong argument for "things should be done like this".

Query params tell me something _about_ how I want the data filtered, displayed, or grouped.

Route params _identify_ the object or collection I'm retrieving.

Well... in REST they do :-)

HTTP is designed to give access to a tree of files. The REST design says "lets fully embrace the idea that we are representing our data and program state as a virtual tree of files." You can embrace or reject that to various degrees, but the closer a design is to a virtual tree of files, the more "RESTful" I say it is.

In a RPC ("procedure call") paradigm, you have a collection of named procedures, each of which takes parameters/input and returns a result/output. Anything that isn't the procedure name is a parameter to the procedure. Some are mandatory parameters, and some are optional parameters, but in the most straightforward implementation of "HTTP RPC", the path indicates which procedure, and parameters to the procedure should be a HTTP parameters (or request body). Also note that in this case the HTTP status '404' would indicate the presence or absence of the RPC endpoint, like the author wanted, and the procedure would be expected to return 200 for a within-spec return value, or possibly 422 or 5xx to represent an application-level exception to be propagated to the caller.

There's nothing stopping us from mixing the paradigms, and in fact nobody is forcing anyone to use any paradigm at all, since we can literally do whatever we want with the HTTP path, uri, method, and headers. But when the original article claims to be doing RPC and not REST, I think that is just sowing confusion onto the topic.

404: it doesn't exist 204: it exists and has no data to parse.

While I agree with you, I sometimes misuse 200 OK even for actual errors. Most times because whatever internal library I've used to do the requests have made it hard to parse the error data if it happens (just throws a general exception instead when it encounters a 4XX or 5XX). Or sometimes because the request failing is somewhat expected, but the client library then insists on doing lots of error handling when it fails (logging, alarms, circuit breakers and whatnot), polluting everything.

There is such a thing as software engineering, but that isn't what most of us do (nor is it what we should be doing, IMO).

NASA needs software engineers. Low-level critical systems need them, so the G-MAFIA has a few.

For most business purposes, though, the iterative exploratory "let's get a UI out there and see how it works" is exactly what we need.

It's a useful, cost-effective strategy when lives aren't on the line, and it isn't engineering.

(My current job title is "Software Alchemist")

Because honestly the answer is, as long as you are consistent, it really doesn't matter if you use status codes or not.

You can and probably should provide a payload for further processing in some cases, but that's an orthogonal concern.

You are also writing a HTTP server, which provides resources, expects certain formats, provides certain formats, tells intermediate layers and clients about what to cache for how long, says whether a new resource has been created or _will be_ created. You might be redirecting or you might be telling a client that they need to retry a request because there has been a conflict. Your response might be processed by different layers, some of which only care about the status codes, while others will inspect the payload to make further decisions. The list goes on.

Status codes are there so you can do all of these things and more in a standardized fashion. Even just from a human consumption perspective it is useful to just look at the code and immediately have an intuition of what happened.

Also many of the status codes make no sense at all if an application server doesn't provide them. A reverse proxy doesn't know whether there has been a transactional issue on the application server, so it cannot say 409 without that information. A caching layer doesn't know about the domain model so it cannot say whether a specific resource can be cached forever.

In the context of RPC, is it still impractical? It feels clearer to me, but I might just be continuing my own confusion now

If you are defining a bespoke RPC protocol that incidentally uses some parts of HTTP without much concern about the spec but only concern with what existing infrastructure will do with requests, you have…lots of freedom to design things however you want.

OTOH, a lot of work has gone into handling almost every conceivable aspect of information interchange, within a REST style (because REST was both derived from the architecture of the Web and used in updating HTTP for HTTP/1.1) in the HTTP standards (there are some gaps still, like that addressed by the draft QUERY [0] method), why reinvent the wheel, unless you are specifically dealing with use cases squarely within the gaps?

[0] https://www.ietf.org/archive/id/draft-ietf-httpbis-safe-meth...

In any case, consistency is key. Personally I think it is easier to lean on standards in order to be consistent. Which is why I made some examples of status codes that can only be formed by an application server. I think "how do I avoid surprises, workarounds and inconsistencies" is a good question to make decisions around what we're discussing here.

Then Twilio's obesrvability code is broken. Requesting a non-existent resource is not a serious error if resources are requested by hand-constructed URLs.

- random clients on the Internet poking around at stuff, which you have no control over and causes no harm so should not alert

- an error in the links or code of the pages you ship to clients, which will cause a consistent degraded user experience and you should be notified about.

In practice, the best way to solve this I'm aware of is not via observation of server-emitted 404s at all... It's via a back-channel for clients to report errors (possibly authenticated, so randos on the web can't fudge your stats) and then tie alerting to that back channel. So you don't track 404s to /foo, but you do alert on your clients screaming at you that they got a 404 trying to access /foooo.

Of course, this solution requires your end-users to enable JavaScript.

For 5xx errors you should typically monitor against a baseline of 0.

4xx class errors you want to monitor for substantial deviation from previous averages. This is a good indication if you broke something, vs client behaviour changed. Remember, monitoring isn't always going to "give you the answer", as much as alert you to a possible issue so you can investigate further.

Moreso, in a SaaS application, monitoring for deviations in 4xx rate by client is also helpful. An alert on a single client is likely not a system-level issue, but a devaition across all clients likely is.

In another case, I had a site that used http basic auth. An xhr api request returned an expected 403, which resulted in the browser suddenly concluding the basic auth (which was unrelated to the api call) was invalid and the user needed to be reprompted for credentials again.

Both of these could be argued as browser problems, but that's the point: browsers (and observability tools and many other things) think they know what a given http status means. Using http statuses for app-level errors often breaks that.

That sounds like it was a mistake on the part of the api. The browser should only prompt for credentials if the response headers include a WWW-Authenticate header, but that header should only be included with a 401 response (at least according to MDN).

https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/WW...

Yea, because they've all been built on top of the same faulty assumption. This isn't a counter argument to the OP in any way and I'm kind of surprised you're not the only one bringing this up as some kind of gotcha.

Obviously platforms would need to be handle and treat http and application layers differently. This is as much a given as the fact you'd have to update all your client code to wrap and unwrap the application layer on top of the http layer.

The observability of your application would actually increase as you'd be able to differentiate between an increase in application failure modes from an increase in http failure modes. That's actually really valuable.

Why not ping the monitoring when you encounter the business error?

From that perspective, a stream of HTTP errors could indicate a more specific kind of infrastructure failure vs just clients sending garbage.

Happy to admit I'm wrong though, it's why I posted it here, see what others think

But if we relax the “hypertext” presumption a bit and consider HTTP a protocol for the transfer of resources, surely an API can be implemented such that using response codes is sensible and useful. REST and SOAP for CRUD operations can still work just fine, and response codes can still make sense in the context of the API itself, even if the content is JSON describing business logic state.

It’s a messy world, so we have to be liberal in what we accept, but using status codes is not inherently abuse. Any ambiguity might just be a symptom of leaky abstractions in the API.

In this philosophy you would not suppose how employees are named and would ask for a list of valid employees URLs to a URL previously returned by `/api/v1/`.

In this way you only ever call URLs that the api provided you with, so that you are not expected to find out by trial and error whether `/api/v1/employees/100` exists or not; the api itself has already told you this and a 404 is the response to trying to build your own URLs by hand.

But the path does exist. The employee record represented by that path doesn't exist.

A request with a path that reaches a server, is looked up in the routing table or filesystem, resulting in no resources found, is expected to return "404 Not Found".

You can say the path "exists", but only as a label. All labels can exist. Only a subset of these are semantically valid resource locator paths, and only a further subset reference objects that exist.

Colloquially, https://news.ycombinator.com/this/path/does/not/exist (path "/this/path/does/not/exist") "does not exist" because we're talking about the requested resource.

Of course the path exists, I just wrote it and it's semantically valid.

That's not an interesting question though, and if API responses were this shallow -- just a "semantic correctness checker" -- then response codes would be useless.

Why? We all know what it means.

>Colloquially, https://news.ycombinator.com/this/path/does/not/exist (path "/this/path/does/not/exist") "does not exist" because we're talking about the requested resource.

That path is 404 - Not found and returns "Unknown"

https://news.ycombinator.com/item?id=52081900

Returns 200 - No such item.

In 10 years when we have that many threads it will return content.

As I've said in several other places it depends what you want to tell the client. 404 if you want to tell them you didn't find the resource. 200 if you want to tell them the resource doesn't exist.

> 404 if you want to tell them you didn't find the resource. 200 if you want to tell them the resource doesn't exist.

404 is a client error, like all 4xx codes. The requester can correct their request. It does not indicate that the server ~"tried to find but may have overlooked" a resource, it means that the server can authoritatively state that the resource is not findable.

This is essentially, "does not exist at this time". Resources that do not exist today, might exist in the future, or they might not. This is true for both /users/1000 and /some/other/path.

I interpret the RFC to support my belief:

  6.5.4.  404 Not Found

   The 404 (Not Found) status code indicates that the origin server did
   not find a current representation for the target resource or is not
   willing to disclose that one exists.  A 404 status code does not
   indicate whether this lack of representation is temporary or
   permanent; the 410 (Gone) status code is preferred over 404 if the
   origin server knows, presumably through some configurable means, that
   the condition is likely to be permanent.

Impressive given that it explicitly contradicts your point:

>the server can authoritatively state that the resource is not findable

>origin server did not find a current representation

Those two are not the same thing.

Those two phrases do not lexically parse in identical ways, but we should not interpret the RFC to be saying the alternative of "I really tried, but boy filesystems are hard and maybe the NFS mount disappeared, and well shucks, couldn't find it! 404!".

There is no defined equivalency between host1/one/path and host2/other/path, even if the resources returned could potentially be the same.

There is no retry formula for a 404 at one URL that might return 200 at another URL.

There is no defined meaning for the same resource at another location. HTTP 404 makes no attempt to comment on the uniqueness or commonness of a resource. No comment on whether that resource existed in the past, is expected to exist later, or in whose imagination it could conceivably exist in the future.

The HTTP response simply states whether the resource was found to exist at the specified location at the time of request. Any further conclusions drawn from a 404 are speculative.

But I also very much don't like playing the "if you expose your app over HTTP you should assimilate HTTP semantics and do a fuzzy lossy map of your application to HTTP verbs and HTTP status codes." I leave the HTTP semantics to the front end web server and live on top.

'Nothing handling this route' has no meaning, because routes have no meaning in a hypertext application. Clients should not generally be constructing URLs by hand; they should be receiving them from the servers they communicate with.

In the example in the article, an application dealing with employees should not be constructing URLs by appending employee IDs to strings; rather, every reference to an employee in the application should be to a URL rather than an ID. So when it requests a list of employees, it receives the equivalent of {/api/v1/employees/1, /api/v1/employees/2 … /api/v1/employees/N} rather than {1, 2 … N}.

> I also very much don't like playing the "if you expose your app over HTTP you should assimilate HTTP semantics and do a fuzzy lossy map of your application to HTTP verbs and HTTP status codes."

If you are building a hypertext application, then you should build a hypertext application. It's completely possible. Off the top of my head, protocols such as ACME (used by Let's Encrypt) are good examples to follow.

   users = http.client.get("https://yourapi.com/users")

   # Frobulate each user
   for user in users:
      user = http.client.post(user.frobulate_url)
      # While this is running somewhere on the other side of 
      # the world someone deletes one of the users. Oops.

I'm not sure I get this, every API doc is like "go to /users for the users and here's the methods we support, the payloads, and responses" If someone mistypes it and tried to get "/user" I want to send a 404 to be like "there is nothing here."

> In the example in the article, an application dealing with employees.

This is all very nice when your domain maps nicely to objects. My litmus tests for this is what would the semantics of mysql.net/query?db=mydb,q='select * from table;' look like.

* If the result is an empty record set should it return 404? Ew. I think it should be 200 with the response being `[]`.

* If you're not allowed to access a table should you get 403?

One other small gripe I have with this approach is that it reduces the visibility of a failing request in the application layer, which is useful for debugging. It's not uncommon for one of my apps to issue many requests to the app server and if everything returns a 200, I have to look through the various payloads to find the request that actually failed.

I guess my question is: is it really abusing HTTP statuses if it works well overall? I rarely find myself wondering whether a request failed in the network layer or the application layer, I could probably count on one hand the number of times this scenario bitten me in my career. I don't really see the point in throwing out something that works over semantics in an RFC.

If you absolutely needed separate status codes for the two cases, you probably could use 410 Gone or 501 Not Implemented for unrouteable paths and 404 Not Found for bad IDs.

For example, https://news.ycombinator.com/s.gif -> 200

But https://news.ycombinator.com/t.gif -> 404

I don't see why /api/v1/employees/3.1415926 shouldn't return a 404 either.

The standard is to return a 404 for a resource that is not found. The standard is not to return a 200.

An application which expects a 200 for a not-found resource is not expecting standard behaviour, and is dead wrong.

This is the conclusion I've come to. What the RFCs say the status codes mean doesn't matter anymore. There isn't anywhere near enough consensus on what they mean or how to treat them to base a system off of in the abstract.

What can specifically matter is how systems in the concrete will react to them. Do you have something that needs to be CDN'd? Then you'd better return status codes that make that work. Is it going to be proxied? Then you'd better return status codes that make that work. Is it never going to be either of those things and only directly accessed by internal users? Then use status codes in a way that works for them. Do you have a monitoring system? Use status codes in a way that works for those systems.

Do you have conflicting requirements as a result of those things? Bummer. Sucks to be you. Best get to figuring out what to do about that, but one thing I can tell you is that carefully consulting the RFCs about what status codes "mean" won't be much of a help in such a situation. Maybe a little. But not much.

To the extent you want to reply with a "but what about...", I said, if it works, use it. While it is the wild west, there are certainly some patterns. 200 vs. "a non 200" response certainly has patterns of meaning to it, and if playing into those patterns works for you, great! Do it. But if I encounter a case where it doesn't, I cry precisely zero tears and do what does work.

The time to moralistic and prissy about what status codes "really mean", if it ever really existed (to be honest, all there really ever was was "return one of the 5 or 6 codes the browsers understand when appropriate, and the search engines will follow the broswers"), is long gone. You can be wistful about how nice it could theoretically be if we just all agreed on what they all mean, but that's just not going to happen in a world where I can barely get people to agree that numbers shouldn't be strings in JSON, let alone precise details about what exactly a hundred error codes mean across such a diversity of systems and users.

I could even argue they are a failed element of the protocol. Such an argument would center around the futility of trying to represent such a rich variety of possibilities, along with such a rapidly changing variety of possibilities, into a single number. It's hopeless. You can't enumerate all the possible successes and failures of such a broad protocol like this, and it didn't help to try. It's just another instance of the "shared ontologies are fundamentally unscalable" problem.

But if you sit down and look at the RFC, and really try to understand what quite a lot of the codes are, they're just useless. Not necessarily to everyone, but they're useless to me, because many of them clearly involve some sort of not-universally-shared context and probably mean something very, very precise to somebody, but not me, not my API users, not the writers of the APIs I consume, etc.

Plus, you know, ultimately those codes were for documents, not for APIs anyhow, and that shows too, in some basic missing stuff for API calls.

I'd love it if this wasn't the case, absolutely. But it is, and yelling at people to "conform to this RFC harder" is just a waste of time on multiple levels.

I’ve personally integrated with almost 50 different SaaS APIs in the last 2 years.

The worst ones to work with were the ones returning errors in 200.

I don’t want to parse and write switch statements for your strings to understand whether it’s authentication, authorization, not found or any other error.

Now I have logic tied to strings you return and I can’t wait until someone decides to change the error messages they return. People rarely assume there’s an API contract in error messages.

I've had similar experiences.

Status codes are logical, machine-oriented things. At the very least, 4xx should denote client errors (that implicitly should not be retried without change) 5xx should denote server errors (where it could make sense for the client to retry).

I've integrated with APIs where 2xx can denote an error, so clients had to parse the body or headers of the response to determine the real status some other 'inventive' way. Please, don't do this.

But then what's the point of an API contract if it's not describing the returned data? What I'm arguing is the opinionated payload provides a lot of the same value. Or am I missing something?

The monitoring systems would naturally not be stoked to see 2xx codes containing errors, but I'd just ping the monitoring system out of my application server anyways. Not sure if that's better or worse though.

I've done like little to no platform engineering and I may be gravely underestimating the consequences of doing this, but it works well with prometheus.

Perhaps I am a fool though :) but how else would I find out if I didn't put my ideas out there :D

  But then what's the point of an API contract if it's not describing the returned
  data? What I'm arguing is the opinionated payload provides a lot of the same 
  value. Or am I missing something?

  You try employee 1, fantastic, it works!
  You try employee 100, not fantastic, it 404’d.
  Huh?
  Why do I get a 404 here? The path is clearly correct, otherwise employee 
  1 wouldn’t have worked either.
  “Ah”, you may be thinking “but it clearly means that the employee wasn’t found!”
  No, there’s nothing clear about that. If I were to call /api/v11/employees/1 
  I would get the exact same error. As an API consumer, all I want to do here 
  is raise my middle finger.
  But as an API producer, this results in a conundrum: What am I supposed to do then?

If for _some reason_ you want to give more info for requests to `/api/v11/*` or whatever other non-paths you want to handle, just serve a 400 response back and let your consumers figure out what they screwed up; but I argue you have more important things to do with your time.

Also, why are you worried about differentiating your server errors from client network failure? That's for the client app developer to handle. Don't worry about it. Your obligation begins and ends with a connection to your service.

  Opionated payloads should be mandatory
  Returning a 2xx code immediately tells the client that the HTTP response
  contains a payload that they can parse to determine the outcome of the 
  business/domain request. That is to say
    - client checks HTTP response is valid (2xx status)
    - client can confidently parse the response and make a domain oriented
      decision, as opposed to a techinical one
  This makes your client happy. Very, very happy. Using our above examples, 
  here is what we would see:

  My API is clean, easy to understand and easy to debug. A client no longer
  needs to send me a request to ask for clarity on an endpoint that sometimes
  returns a 200 and other times returns a 404.

In general, applications using REST should follow these semantics.

2XX - Request was understood and we found what you were looking for

4XX - Something was wrong with the request on the client side. The client should take some action before retrying.

5XX - Something was wrong with the server and you can retry this request sometime later and it may work.

2XX - Request was understood and we found what you were looking for

can be broken down into two parts:

1. Request was understood

2. We found what you were looking for.

This guy seems to be advocating that completing a search and there being no results is actually a successful request, and so should be responded to with 200.

The problem though is that the spec (RFC7231) requires a representation of the resource to be sent in response to a GET.

If there's no payload, then you can send a 204 instead, but this raises its own challenges - a 204 just means that the server is not sending content, not that no content was found (which is a subset of 'not sending content').

I think on balance, 404 is correct - the spec indicates that it should be used where there's no current representation, which I think is an accurate description of a failed search.

Adding to this, it is far easier for clients to parse response codes than bodies. TFA makes the following claim:

> Returning a 2xx code immediately tells the client that the HTTP response contains a payload that they can parse to determine the outcome of the business/domain request.

Which is wrong. Response codes say nothing about the body.

Parsing the response body requires much more work: determining the format of the response received by the client, parsing it, validating it, handling edge cases, and so forth.

Further, it is possible, however unlikely, that the client does not accept application/json. Now what? Now there is yet another problem you have to solve.

And don’t forget the server has to generate all this as well.

All of this could have been avoided simply by returning a simple 404.

Actually, that's not correct. A response code can say a fair bit about the body, especially about whether it exists or not, but in some cases also the nature of the body. For example, a 203 says that the body of this request is not the same as the body sent by the origin server. A 204 tells the user agent that there's no body. 400s and 500s SHOULD have a body which includes an explanation of the error.

I agree about 4xx bodies, generally. They indicate that the request was malformed or unsuccessful in some way. The requester can fix this problem.

I think a 5xx should always represent an error on the server side that the requester cannot fix. It is always a bug or system failure to return a 5xx, which should be fixed by the API operator. Therefore no details are required or desirable to return to the requester.

6.6

[...] Except when responding to a HEAD request, the server SHOULD send a representation containing an explanation of the error situation[...].

Unless the user can fix the problem, and you want them to try, an explanation of a 5xx error adds no value. Again, I don't think 5xx is ever a valid response from an API -- it's always a server-side error (code or infrastructure) that needs fixing, often with high priority.

In many environments (e.g. medical, financial, or other secure/private), providing details on server errors is strongly discouraged. Even when the data isn't sensitive, I think you're asking for trouble by broadcasting server-side problem details to users.

I think my original point stands though, that status codes do tell you about the body in some circumstances. Or SHOULD tell you something about the body at least :)

[0] https://www.rfc-editor.org/rfc/rfc2119.html

So yes, it is correct to say that response codes say nothing about the body.

404: Route not found vs 404: User ID not found seems like it resolves all ambiguity, no?

What if we changed `/some_photo.jpg` to `/photo?id=some_photo` and it didn't exist? 404? Okay, now what if we change `/photo?id=some_photo` to `/photo?id=100`? What if then change that to `/photo/100`? At which point does it no longer become okay for the request to be tied to the resource?

`/api/v11/employees/1` may be 404, and `/api/v1/employees/100` maybe 404 too, because neither of them are found. If anything, the problem is that HTTP status codes are limited and haven't really kept up with technology. We have a few additional codes, like with Cloudflare, but for the most part, there is no community project or standard for expanding HTTP Status Codes.

Perhaps there should be.

/api/v1/employees/100 - direct path to a resource. 404 if it doesn't exist.

/some_photo.jpg - direct path to a file/resource. 404 if it doesn't exist.

/photo/100 - direct path to resource: photo ID 100. 404 if it doesn't exist.

/photo?id=100 - search for a photo with ID "100". Return a code 200 with an empty results set if that ID is not found. (With optional application-specific error feedback in the response body.) 404 if /photo doesn't exist at all.

>The 404 (Not Found) status code indicates that the origin server did not find a current representation for the target resource or is not willing to disclose that one exists.

So when you say:

>yet the record doesn't exist, then the resource doesn't exist.

So you do have a current representation for the target resource. Namely, that it doesn't exist.

No.

> Namely, that it doesn’t exist.

That it isn’t a representation, it is a fact about the universe (not the resource, because existence is not a predicate) which is inconsistent with their being a representation. A resource must exist to have a representation.

Since a resource that does not exist cannot have any representation, it is therefore impossible for a representation to be found of such a resource. In fact, other than in the case where 421 is the correct response (“don’t ask me, I’m not responsible for that URL”) or where 406 is a more specific response than 404 (“I have the resource requested, but can’t produce a representation in the format you have requested, and don’t want to waste both of our time sending you a format you may not be able to use”), the only reason for a 404 is that the resource does not exist.

If you ask for an employee that doesn't exist, is it a failure? Or is it just a negative, but expected response?

Perhaps that's a stupid question though.

REST isn't RPC lite. If your mental model is that an HTTP GET is a function call, you think of 4xx's as exceptions, but that is not the correct mental model.

But that's a good point about the transport layer.

REST over HTTP is HTTP used according to spec, because the REST architectural style involves using the underlying protocol(s) per spec, but only the subset of such protocols that corresponds to REST semantics (but in HTTP’s case, since HTTP/1.1 is itself designed around REST principles and HTTP/2 and HTTP/3 maintain HTTP/1.1 semantics, that’s pretty much the whole spec.)

I think we need to be using HTTP error codes better at the API layer, not ditching them entirely. Building on this example, for an invalid user, how about an HTTP 204 and an empty response body? That gives us a clear "no such user exists" without having to parse JSON out first.

EDIT: for the record, I think that on some level, REST APIs themselves are an abuse of HTTP. JSON didn't even exist for much of the public Internet's first decade, and certainly the early work on HTTP never mentioned such a thing as a REST API. We're already using HTTP for so many things in so many ways that this is somewhat of a Lilliputian discussion.

Why would we handle not finding an employee differently from not finding an API version?

If we're saying employees/100 should return 200 because it "could" exist, but doesn't at the moment, then why would we ever return anything other than 200? Any route "could" come to exist at a future point.

If you want to do something the author suggests, just abandon REST pathing and do a service call like api/v1/getemployee?id=100

Then the endpoint api/v1/getemployee exists, there’s just no results for your query.

Now for those who are exploring your API with curl or are reading logs from their service they can see the human-legible information that may help them debug the error, while their application logic can continue to use `2xx` for success, `4xx` for my own failure and `5xx` for a server failure that I should retry after some back-off time, allowing it to do the most sensible thing it can in the situation, rather than explode in an uncontrolled fashion when an endpoint the developers only got successful 200 responses from during development suddenly encounters a 200 response with an error payload.

Not according to the spec, though you can for most status codes. (Consider, for example, 205.)

I’ll also point to other status codes like 400 Bad Request or 401 Unauthorized which give a clear reason for why something didn’t work as expected. Under the author’s framework all 4xx responses should be 200.

> The 400 (Bad Request) status code indicates that the server cannot or will not process the request due to something that is perceived to be a client error (e.g., malformed request syntax, invalid request message framing, or deceptive request routing).

The objective mostly was to try help form a way of reasoning about HTTP APIs from the consumer's perspective

> error 400 for all semantically-invalid requests

Counter point: Passing `a` to the example endpoint is not semantically-invalid from the perspective of HTTP, it's a perfectly valid URL. Nothing about that request is invalid, it's only invalid to the business layer, which has nothing to do with the HTTP layer. Put another way: The server _is_ processing the request, that's how it send back the "employee not found" record. I guess it depends where you want to draw the boundary of server.

Which is not to say you are wrong if you return a 400, but I suspect you might get a couple queries about it from consumers.

The objective was mostly about clarity. Maybe I'm being over-simplistic, but I like the idea of using protocol errors for protocol problems and domain errors for domain problems. Just keeps client-side logic cleaner in my experience

It's also why I raised the issue of where your "server boundary" is. I dunno how else to phrase it, but what I mean is the separation of domain logic and technical logic.

In my understanding, 400 denotes an HTTP request that doesn't comply with the RFC (missing headers, bad format etc) but I also don't disagree with what you're saying, because the user did send a bad request.

Of course, you could always do both :) Send the opinionated payload with a 400 error.

As long as your consumer can reason about what went wrong, then I guess that's in line with the objective I was trying to get across.

I think there is a correct answer: use the 404 to indicate that a requested resource does not exist, and use a 200 to return a requested resource that does exist; and use URLs to represent resources.

In other words, follow the HTTP RFCs.

And never, ever, ever EVER use a 200-series status code to return an error.

5XX - Server did something wrong

4XX - Client system did something wrong

2XX - Everything worked

IMHO, what's missing is a code for "All systems worked but the user did something wrong" for scenarios like looking up a non existent ID or if their CC details don't work.

https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/422

(+) This approach reduces ambiguity between failed routes and missing resources

(-) This approach adds ambiguity by making missing resources look like they exist

But what does matter is expectations. People naturally consume lots of HTTP APIs everywhere. And making things like everyone else's doing is important.

Everyone's using HTTP codes for their API. That's reality. So I think that it's better to stick to this format until we'll go some circle and get back to SOAP 3.0 or something.

If you would ask me, I'd prefer distinct set of statuses for APIs. So nginx will never return it, no matter what (mis)configuration happened. Right now if I misconfigure my reverse proxy, it'll return 404 for everything. This should be gateway error, but it's not and some clients will happily work thinking that everything not found which is not true. So something like 1200/1404/whatever for APIs and 200/404 for other things. And author's approach will not help, because it's equally easy to misconfigure nginx to return 200 with some garbage SPA HTML for every request. Well, at least there's hope that failing to parse it as JSON will signal to client that something's not quite right.

HTTP doesn't and shouldn't make a distinction between "this URL maps to a known route on my API backend but the ID doesn't exist on the database" and "this URL is not mapped on my API backend", this is leaking implementation details to the client.

This not "abusing" status codes, this is the expected behaviour for an HTTP server.

It is your API, it is your decision to decide if you have looked for the resource and were unable to find it, so return an error, or you have looked for the resource and found a placeholder that says there could be a resource here, but currently isn't and successfully return that placeholder.

To suggest people are wrong or abusing the protocol for not choosing to implement placeholders in their application is incorrect. To suggest people may benefit from implementing a placeholder instead of a 404 error is useful.

My personal experience is that the error status code here is much more useful than the message. It's a big red/orange flag that says don't do what you normally do with this resource. But if your API lends itself particularly to being queried for things that often don't exist then those red flags become noise that mask other things you really do want to flag as errors.

When you need to convince other people to use your API then those people will bring their own expectations how the API behaves and then you need to manage those expectations. If your API is esocentric on its design choices other people might find it unexpected, curse and move to do something else.

  /api/v1/employees/100 -> StatusCode: 200

  /api/v11/employees/1 -> StatusCode: 404

You can still include a payload with a 404 describing what is missing or why it wasn't found.

If I see a 2xx status code, I assume the request succeeded. OP's way forces an additional check, more code, more potential for bugs.

I understand what the owner of the API is trying to convey. "My service is up, and the URL itself exists" (status 200), but in the body "the data you requested doesn't exist" (result: true, error: "doesn't exist). If you are stubborn enough, you can even think you're right.

But by Odin's Beard this was one of the hardest APIs I had to work with. It doesn't help that most HTTP clients assume a sane* API expecting error codes based on the resources.

Note: I say sane, but within this context, this is of course subjective. For me, error codes corresponding to the resources is sane, but to OP this is clearly not the case.

Since this makes my firewall dialog pop every couple of seconds after I close it, I prefer to close the page without reading it.

Why not just serve it over 443 as well?

There's nothing wrong with that, and many implementors and applications have certainly taken the tack the author suggests -- i.e. use HTTP strictly as a simple transport layer rather than a representation of application state -- GraphQL is a great example.

But the rfc is pretty darn clear: "The 404 (Not Found) status code indicates that the origin server did not find a current representation for the target resource or is not willing to disclose that one exists."

I can't find a reasonable interpretation of this that somehow means if the server can't find the document you asked for, you should still return 200.

All the author disagrees with is mixing the http server layer and application client layer status codes in the same client response being given to users.

The only argument anyone should be having here is if they believe their api should speak HTTP to their client or if their api should speak through HTTP to their client.

Personally?

I think trying to shove business level response logic in HTTP (or REST for that matter) to be monkey butt level stupidity.

HTTP was built as a RESTful protocol to serving files up to web browsers. And it works great for that.

However, it was not designed to handle the 25 unique and user-readable or not-user-readable errors my one /authenticate route can throw back at the user at different times.

REST and HTTP by extension excels at simple CRUD or file serving operations. But it's just not suitable for these kinds of API's.

Here's an example:

If my HTTP speaking API needs to respond with an error, how should my client handle it?

How should I differentiate errors that should be displayed to users and errors that should be used only for debugging?

If you need to suggest that my client needs to parse out the errors or differentiate between server and api errors in order to be able to show these errors to users then you've sort of proved my point that HTTP isn't a suitable solution to this problem - you're already wrapping an extra non-HTTP protocol inside HTTP, you're just only doing it for errors instead of all application responses.

You know what happens next. If a client generates too much business errors, the load balancer tends to throw out a server, which doesn't bother the client in the slightest, except it has to login again for the next request. Keep this up for a few minutes, and it makes all but 1 servers disappear from the pool, and the only survivor to spend half its time negotiating connections.