> The purpose of testing is to document the code for the understanding of future developers, not to prove correctness.
Hmm. I've never seen tests with that goal in mind, except for behavioral tests that test the acceptance critera.
> as I am sure you are painfully aware if you have ever dealt with legacy forms of documentation [...] that quickly become out of date
I have, but allowing that to happen is a culture-issue, not something that is guaranteed to happen. When I open PRs to open source software, I always include a PR to the docs if it changes anything. At work, updating the docs is part of the default acceptance criteria and is usually the thing we do before writing any code, and goes through a PR process just like the code. But, we service enterprise customers, so we aren't going to be giving them code or tests to understand how to use our product.
> These are all mocks, ultimately.
This is a software field and there are specific words with specific meaning; trying to shoehorn things that aren't those things to generalize a meaning is acceptable when teaching. It isn't acceptable when working on those things. In other words, I would accept this if trying to explain the concept to a junior engineer, but not from a senior engineer to a senior engineer.
Then you've never seen a test, I guess. That is the only goal they can serve, fundamentally.
> I have, but allowing that to happen is a culture-issue, not something that is guaranteed to happen.
Mistakes are guaranteed to happen given enough output/time. No matter how hard you try, you are going to make a mistake at some point. It is the human condition. In the olden days one might use a proofreader to try and catch the mistakes, but with the advent of testing a computer can do the "proofreading" automatically, leaving the human effort to be pointless.
Maybe in the age of LLMs we can go back to writing documentation in "natural" language while still using machines to do the validation work, but then again if you write code you probably would prefer to read code. I know I would! The best language is the one you are already using. Having to read code documentation in English is a horrible user experience.
> This is a software field and there are specific words with specific meaning
Sure, but in this case you won't find any real difference in meaning across the vast array of words we try to use here. The desperate attempts to try and find new words is to broach silly soundbites like "mocks are a code smell", so that one can say "I'm not mocking, I'm flabbergasting!", even though it is the same thing...
> Then you've never seen a test, I guess. That [,not to prove correctness,] is the only goal they can serve, fundamentally.
I cannot wrap my head around this statement. It's literally in the name: "test" as in to prove something works... hopefully as designed.
> Mistakes are guaranteed to happen given enough output/time. No matter how hard you try, you are going to make a mistake at some point.
Yep, and they do. Its really easy to figure out which one is right: if the docs say that something happens, it happens. If the code doesn't do what the docs say, the code (and the tests) are wrong; and not the other way around.
> Having to read code documentation in English is a horrible user experience.
It's the difference between intention and action! I worked with a guy who opened PRs with totally empty descriptions. It was annoying. When I was reviewing his code, I had to first figure out his intention before I could understand why there was a PR in the first place. Was he fixing a bug, adding a new feature, or just writing code for the hell of it? ... nobody knew. Then, when you spotted a bug, you had to ask if it was a bug or on purpose, because you didn't know why the code was there in the first place.
Documentation is that living PR description. It doesn't just tell you WHAT exists, but WHY it exists, what purpose it serves, why that weird little line is the way it is, etc., etc.
> It's literally in the name: "test" as in to prove something works...
The documentation is what is under test. It proves that what is documented is true. It does not prove that the implementation works. This should be obvious. Consider the simplest case: A passing test may not even call upon the implementation.
I have most definitely seen that in the wild before! More times than I wish I had. This is why TDD urges you to write tests first, so that you can be sure that the test fails without implementation support. But TDD and testing are definitely not synonymous.
> Its really easy to figure out which one is right: if the docs say that something happens, it happens.
Under traditional forms of documentation, you don't have much choice but to defer to the implementation. With modern documentation that is tested, typically the documentation is placed above the implementation. Most organizations won't deploy their code until the documentation is proven to be true. The implementation may not work, but the documentation will hold.
> I worked with a guy who opened PRs with totally empty descriptions.
I'm not sure PRs fit the discussion. PRs document human processes, not code. Human processes will already typically be in English (or similar natural language), so in the same vein the best language is the one you are already using. That is not what we were talking about earlier; but, granted, does do a good job of solidifying the premise.
Hmm. I've never seen tests with that goal in mind, except for behavioral tests that test the acceptance critera.
> as I am sure you are painfully aware if you have ever dealt with legacy forms of documentation [...] that quickly become out of date
I have, but allowing that to happen is a culture-issue, not something that is guaranteed to happen. When I open PRs to open source software, I always include a PR to the docs if it changes anything. At work, updating the docs is part of the default acceptance criteria and is usually the thing we do before writing any code, and goes through a PR process just like the code. But, we service enterprise customers, so we aren't going to be giving them code or tests to understand how to use our product.
> These are all mocks, ultimately.
This is a software field and there are specific words with specific meaning; trying to shoehorn things that aren't those things to generalize a meaning is acceptable when teaching. It isn't acceptable when working on those things. In other words, I would accept this if trying to explain the concept to a junior engineer, but not from a senior engineer to a senior engineer.
> it is all the same at the end of the day.
No, not at all.