I'm just going to comment on one point of your comment:
> to be easily reviewable/checkable by coding teams.
I strongly disagree with this. I find Go code to be incredibly difficult to read. This is because of two main reasons.
The first is that the lack of expressive power in the language means that many simple algorithms get inlined into the code instead of using an abstraction with a simple and understandable name. I find that the first pass of the code is me reading over the lines (each of which is very simple and understandable) and virtually abstracting it into what the code actually does. I find that the interesting business logic is lost in all of the boilerplate.
out := []int{}
for _, v := range input {
number, err := fetchData(v)
if err != nil {
return nil, err
}
if math.Abs(float64(number)) <= 2 {
continue
}
out = append(out, v)
}
return out
As I said, every line in the Go is simple (except maybe for append, but generics can help with that). However the actual business logic is lost in the boilerplate. In the second example (Rust with one existing and available helper function) the boilerplate is much less and each line basically expresses a point in the business logic. There are really two bits of boilerplate here `filter_ok` instead of `filter` to handle the errors from `fetch` and the `collect` to turn the iterator into a collection (although maybe you could improve the code by returning an iterator instead of a collection and simplify this function in the process).
Secondly the "defaults are useful" idea is in my opinion the worst mistake the language made. They repeated The Billion Dollar Mistake from C. I have seen multiple expensive production issues as a result of it and it makes code review much harder because you need to check that something wasn't uninitialized or nil. It is absolutely amazing in Rust that I don't have to worry about this for most types (depends on your exact coding style, in the above example there is never a variable that isn't "complete").
So while Go may be quick to write. I think the understandably is deceiving. Yes, I can understand every line, but understanding the program/patch as a whole becomes much more difficult because of the lack of abstraction. Humans can only hold so much in our head, making abstraction a critical tool for understandable code. So while too much of the medicine can be worse that the disease I think Go aimed - and hit - far, far below the ideal abstraction level.
> So while Go may be quick to write. I think the understandably is deceiving. Yes, I can understand every line, but understanding the program/patch as a whole becomes much more difficult because of the lack of abstraction. Humans can only hold so much in our head, making abstraction a critical tool for understandable code. So while too much of the medicine can be worse that the disease I think Go aimed - and hit - far, far below the ideal abstraction level.
I think this is the fundamental point of contention. Go aims at abstraction at the package level. Each package exports a set of types and functions which are "magic" to outsiders and can be used by outsiders - an API, if you will. Rust seems to aim at abstraction at the line level - each line is an abstract "magic" representation of what it is meant to do.
In your Go code, the only pieces of line-level magic are `range` and arguably `append` (even though I would argue it is integral to the concept of slices). And of course the API magic of `fetchData` and `abs` which is in both versions.
On the other hand, Rust has .iter(), .map(), .filter_ok() and .collect(). So, while I think anyone could understand the Go code if you explained `range` to them, I do not understand the Rust code. Yes, I understand what it does, but I have no clue how it does it. What is the type of .iter()? Why can I map over it? Why can I filter a map?
But that's not the point of the Rust code. The Rust code expresses what should be done, not how it is to be done.
The way Rust deals with complexity is by offering tools which push that complexity into the type system, so you do not have to keep everything in your head. The way Go deals with complexity is by eliminating it and, when that is not possible, by making sure it does not cross API boundaries. In Go you do keep everything in your head.
> Go aims at abstraction at the package level. Each package exports a set of types and functions which are "magic" to outsiders and can be used by outsiders - an API, if you will. Rust seems to aim at abstraction at the line level - each line is an abstract "magic" representation of what it is meant to do.
Are line-level vs. package-level abstractions are necessarily mutually exclusive? One could argue that the functions Rust iterators expose are "'magic' to outsiders and can be used by outsiders --- an API, if you will".
> but I have no clue how it does it
The question here is whether it actually matters whether you know how the Rust code works. You don't necessarily need to know how `range` or `append` work; you just need to know what they do. Why should the Rust code be held to a different standard in this case?
> I think this is the fundamental point of contention.
I agree with your point. Part of Go was definitely the removal of unnecessary abstraction and complication. However my argument is that they went too far.
> Go aims at abstraction at the package level [...] Rust seems to aim at abstraction at the line level
I don't understand the difference in your mind vs package level or line level. For whatever is exposed as a package is surely intended to be used in a line elsewhere in the program.
> On the other hand, Rust has .iter(), .map(), .filter_ok() and .collect(). So, while I think anyone could understand the Go code if you explained `range` to them
If you explain range, continue, return and append then you could understand the Go snippet. I don't see how this is meaningfully different from explaining iter, map, filter_ok and collect. Sure, the former are language features while the latter are library features but that doesn't seem to be a meaningful difference when it comes to comprehension.
> Yes, I understand what it does, but I have no clue how it does it.
This is the whole point of my argument. You don't need to understand how it works. Much like you don't need to understand how continue or append work in go. That is the point of abstraction. You need to know what they do, not how they do it. In my opinion Go forces you to leave too much of this usually-irrelevant plumbing in the code, which distracts from the interesting bits.
> The way Go deals with complexity is by eliminating it
This is again my key point. I'm arguing that in most cases Go hasn't managed to eliminate the complexity. Maybe it got rid of a little, as your "map" loop doesn't need to be as generic and perfect as the Iterator::map in the standard library. But the complexity that is left is now scattered around your codebase, instead of organized and maintained in the standard library.
The intrinsic complexity has to live somewhere. And in Go I find a lot more lives inline in your code. In other languages I find it is much easier to move the repetitive, boilerplate elsewhere. And when this is done well, it makes the code much, much easier to read and modify as well as leading to more correct code on average.
> That's a Go feature.
I agree with that. But what I am trying to express that in my experience this is actually a flaw. It looks good at the beginning. But once you start reviewing code you start to see it break down. I think Go had a great idea, but based on my experience I don't think it worked out.
> Sure, the former are language features while the latter are library features but that doesn't seem to be a meaningful difference when it comes to comprehension.
Absolutely. The difference is that Go has a limited number of such features and once you have learnt them that's all you need to know, in that sense, and can understand any code base.
One thing which I have not expressed very well in my reply is what exactly I meant by "understanding what the code does". When you look at func fetchData(T1) (T2, error), it's easy to understand what it does: it fetches some data from T1 and returns it as T2, with the possibility of it failing, and returning an error. If you know what T1 and T2 are (which you should if you're inspecting that code), that's usually sufficient. You understand (almost) all of it's observable behavior, which is different from it's implementation details. Similarly, `abs` returns the absolute value of a number
`append` also has easy to understand observable behavior: it appends the elements starting at position len(slice) and reallocating it if necessary (generally, if the capacity is not big enough), but it's actual implementation is undoubtedly very complex. `range` is harder to explain, but rather intuitive when you get the hang of it.
Of course you also want to keep in mind the behavior of all the language primitives as well: operators, control flow etc. In Go, you have to keep all of these things in your head to understand what is happening in the code, but once you do you really understand it.
We can call all of these things: variables, language primitives, API functions etc. atoms of behavior. In Go, to understand a piece of code, you first have to understand what the observable behavior (but usually not the implementation) of all of the atoms in that code are, and then understand all of the interactions between those atoms that happen as a result of programmer instructions.
What I mean by line-level vs package-level abstraction is quite simple (maybe not the best names, but hey, I'll stick with them). With package-level abstraction, the atoms, as well as the interactions between them, remain conceptually easy to understand, but become more powerful as you move up the import tree. The observable behavior of an HTTPS GET is easy to understand, but very complex under the hood.
With line-level abstractions the atoms, and especially the interactions between them, become very complex. The programmer no longer "has to understand" the observable behavior of every single function he uses. Odd one-off mutators are preferred to inlining the mutation because it "makes the code more expressive" - in that it makes it look more like english, it makes it easier to understand what the programmer is trying to do. It does not, however, make it easier to understand what the programmer is actually doing, because the number of atoms - and their complexity - increases substantially. If you want to get a feel for this look at the explanation for any complex feature in C++ on cppreference.com. I must have read the page on rvalue references 20 times by now and I still don't grok it.
Of course, with line-level abstraction, the programmer doesn't need to constantly keep in mind 100% of the behavior of the atoms he's using, much less whoever's reading.
I can't tell you which one's better - probably both have their place - all I'm saying is that I, personally, can't work with C++/Rust/other languages in that style. I've tried to use them but I can't. C is easier to use - for me.
I'm honestly still confused what point you're trying to make.
Paragraphs 2 through 6 seem like they would apply to most, if not all, languages, even if the language is more complex. For example, here's the text with a few minor alterations:
> One thing which I have not expressed very well in my reply is what exactly I meant by "understanding what the code does". When you look at `fn fetch_data(input: T1) -> Result<T2, Error>`, it's easy to understand what it does: it fetches some data from T1 and returns it as T2, with the possibility of it failing, and returning an error. If you know what T1 and T2 are (which you should if you're inspecting that code), that's usually sufficient. You understand (almost) all of it's observable behavior, which is different from it's implementation details. Similarly, `abs` returns the absolute value of a number
> `Vec::push` also has easy to understand observable behavior: it appends the elements starting at position vec.len() and reallocating it if necessary (generally, if the capacity is not big enough), but it's actual implementation is undoubtedly very complex. `Vec::iter` is harder to explain, but rather intuitive when you get the hang of it.
> Of course you also want to keep in mind the behavior of all the language primitives as well: operators, control flow etc. In Rust, you have to keep all of these things in your head to understand what is happening in the code, but once you do you really understand it.
> We can call all of these things: variables, language primitives, API functions etc. atoms of behavior. In Rust, to understand a piece of code, you first have to understand what the observable behavior (but usually not the implementation) of all of the atoms in that code are, and then understand all of the interactions between those atoms that happen as a result of programmer instructions.
The details differ, but the overall points remain true, do they not?
Unfortunately, I'm still confused after your description of line-level vs. package-level abstraction. Just to make sure I'm understanding you correctly, you say an HTTPS GET is an example of a package-level abstraction, and an "odd one-off mutator" (presumably referring to a functional/stream-style thing like map()/filter()) is an example of a line-level abstraction?
If so, I'm afraid to say that I fail to see the distinction in terms of package-level vs. line-level abstraction, since what you said could apply equally well to both HTTPS GET and map()/filter(). For example, if a programmer uses a function for an HTTPS GET instead of inlining the function's implementation, you can say "using the function is preferred to inlining the request because it 'makes the code more expressive' --- in that it makes it look more like English, it makes it easier to understand what the programmer is trying to do. It does not, however, make it easier to understand what the programmer is actually doing..."
That's the nature of abstractions. You hide away how something is done in favor of what is being done.
> With line-level abstractions the atoms, and especially the interactions between them, become very complex. The programmer no longer "has to understand" the observable behavior of every single function he uses.
I believe your second sentence here is wrong. Of course the programmer needs to understand the observable behavior of the functions being used --- how else would they be sure that what they are writing is correct?
> Odd one-off mutators are preferred to inlining the mutation because it "makes the code more expressive"
I think you might misunderstand the purpose of functions like map() and filter() if you call them "odd one-off mutators". The same way that `httpsGet` might package the concept of executing an HTTPS GET request, map() and filter() package the concept of perform-operation-on-each-item-of-stream and select-stream-of-elements-matching-criteria.
> If you want to get a feel for this look at the explanation for any complex feature in C++ on cppreference.com. I must have read the page on rvalue references 20 times by now and I still don't grok it.
You need to be careful here; in this case, I don't think rvalue references are a great example because those aren't really meant to abstract away behavior; on the contrary, they introduce new behavior/state that did not exist before. It makes sense, then, that they add complexity.
In the end, to me it feels like "package-level" and "line-level" abstractions are two sides of the same coin. The functions exposed by a "package-level abstraction" become a "line-level abstraction" when used by a programmer in a different part of the code.
> I think this is the fundamental point of contention. Go aims at abstraction at the package level. ...
I think this is an excellent point. I have Go code which might make Rust fan apoplectic with its verbosity and basicness. But for me it is like magic every time it runs and get stuff done on any of my remote/local machine.
I made a similar point elsewhere about expression verbosity vs project verbosity. To me all the 3rd party dependencies, substantial number of outside tools and obscure setup files are a type of verbosity when I work on a project. Though I do not mind them if thats what is needed.
> to be easily reviewable/checkable by coding teams.
I strongly disagree with this. I find Go code to be incredibly difficult to read. This is because of two main reasons.
The first is that the lack of expressive power in the language means that many simple algorithms get inlined into the code instead of using an abstraction with a simple and understandable name. I find that the first pass of the code is me reading over the lines (each of which is very simple and understandable) and virtually abstracting it into what the code actually does. I find that the interesting business logic is lost in all of the boilerplate.
vs As I said, every line in the Go is simple (except maybe for append, but generics can help with that). However the actual business logic is lost in the boilerplate. In the second example (Rust with one existing and available helper function) the boilerplate is much less and each line basically expresses a point in the business logic. There are really two bits of boilerplate here `filter_ok` instead of `filter` to handle the errors from `fetch` and the `collect` to turn the iterator into a collection (although maybe you could improve the code by returning an iterator instead of a collection and simplify this function in the process).Secondly the "defaults are useful" idea is in my opinion the worst mistake the language made. They repeated The Billion Dollar Mistake from C. I have seen multiple expensive production issues as a result of it and it makes code review much harder because you need to check that something wasn't uninitialized or nil. It is absolutely amazing in Rust that I don't have to worry about this for most types (depends on your exact coding style, in the above example there is never a variable that isn't "complete").
So while Go may be quick to write. I think the understandably is deceiving. Yes, I can understand every line, but understanding the program/patch as a whole becomes much more difficult because of the lack of abstraction. Humans can only hold so much in our head, making abstraction a critical tool for understandable code. So while too much of the medicine can be worse that the disease I think Go aimed - and hit - far, far below the ideal abstraction level.