The issue is that declarative build systems ALWAYS need an escape hatch for the exceptions--and the exceptions grow over time.
We've been here before. It was called Make. And then it was called Ant. And then Maven. And Google still built their own system. And they will build another again.
Nobody ever learns the lesson that a build system is a program--period. Sorry to burst your bubble: there will be circular dependencies--you will have to deal with it. There will be two different C compilers on two different architectures. You will have to deal with it. Libraries will be in the wrong place--you will have to deal with it. These libraries come from the system but these libraries come from an artifact--you will have to deal with it.
The only way to deal with it is to have a genuine programming language underneath.
> The only way to deal with it is to have a genuine programming language underneath.
I disagree. Having a full programming language is great for the flexibility, but it causes lots of issues at scale. Restrictions in Bazel are critical for having a good performance on a large codebase, and for keeping the codebase maintainable.
With restrictions, we can provide strong guarantees (e.g. determinism, hermeticity), get better tooling, we can query the graph (without running the tools), we can track all accesses to files, etc. We can also make large-scale changes across the codebase.
Note also that Bazel language is not truly declarative, but it encourages a separation between the declarations (BUILD files) and the logic (bzl files).
> Restrictions in Bazel are critical for having a good performance on a large codebase, and for keeping the codebase maintainable.
Note the vocabulary--"restrictions". Your build system isn't solving a technical problem--it's solving a political one and trying to cloak it in technical terms.
We already have a problem. Your build system is now IN THE WAY if I'm not at scale. Any build system that makes people take notice of it is an a priori failure.
Thanks for posting this though. I've had a nagging irritation with so many of these "scalable" things, and this is the first time it has really coalesced that "scale" is almost always intertwined with "political control".
You say that like bazel hasn't already been proven out on a few billion LoC :). You're quite right that exceptions will exist - your job is to either fix them or squeeze them into the build system. Both are well trod paths. You're quite right that these edge cases aren't _easy_ - but in what system would they be?
> there will be circular dependencies--you will have to deal with it.
Add them both to the same compilation unit (cc_library, whatever). Or extract an ABI (e.g. Turbine) and compile against that.
> There will be two different C compilers on two different architectures. You will have to deal with it.
Poke around https://github.com/tensorflow/tensorflow/tree/master/third_p... . I see arm and x86, Windows and many variants of Linux, multiple versions of GCC, multiple versions of clang, multiple versions of cuda, python 2 and 3, and names I don't even recognize.
> Libraries will be in the wrong place--you will have to deal with it.
Just write the path to the file, or write a build rule that makes it visible in a standard location, or make it part of the platform definition, or use something like https://docs.bazel.build/versions/master/be/workspace.html#b... to alias it into your workspace. (Not recommended, but supported.)
> Libraries will be in the wrong place--you will have to deal with it.
I won't pretend bazel can express _everything_, but there's little you can't hack in _somehow_ with sufficient motivation, and moving towards the bazel-recommended patterns brings growing peace-of-mind (and faster builds).
(Disclaimer: Googler, work closely with the bazel team. I like bazel).
The issue is that declarative build systems ALWAYS need an escape hatch for the exceptions--and the exceptions grow over time.
We've been here before. It was called Make. And then it was called Ant. And then Maven. And Google still built their own system. And they will build another again.
Nobody ever learns the lesson that a build system is a program--period. Sorry to burst your bubble: there will be circular dependencies--you will have to deal with it. There will be two different C compilers on two different architectures. You will have to deal with it. Libraries will be in the wrong place--you will have to deal with it. These libraries come from the system but these libraries come from an artifact--you will have to deal with it.
The only way to deal with it is to have a genuine programming language underneath.