I work with TeX quite a lot and I can't even really understand intuitively the principles of the language design let alone the grammar. Like 15 years later and I'm still copy and pasting spells. I'd love to see an CST of my document and figure out why I'm writing the characters I'm writing!
My experience with TeX is a complete opposite. I feel like I can do pretty much anything in plain TeX, with a very good feel of what it is doing in its 'guts'. I wrote parsers in compilers in pure TeX (the latter was admittedly a silly academic exercise). It took a very thorough reading of 'The TeXbook' but after that I am constantly amazed at how deliberate and tasteful Knuth's choices were. TeX is simply ... elegant. I am not sure what 'modern' bells and whistles TeX lacks that everyone is so eager to see. CSS is a nightmare, even with recent additions, it seems clunky, most of its choices look like an afterthought.
It is also somewhat puzzling that same people who advocate for a better grammar would also push the idea of a better markup. So which is it: a better programming experience or a pure markup language that one wants? I feel TeX strikes a near perfect balance here. Most importantly, TeX has stayed nearly unchanged at its core for almost forty years! It is its greatest strength, not weakness. I cannot see what latest fads can improve in TeX
I'm far from an expert but I use LaTeX, and I distinctly remember there is some oddity such that you have to put an empty comment at the end of a specific line in the middle of some code that generates multi column figures, or else the layout gets messed up. That sort of thing just shouldn't be possible in anything that has the right to be called "elegant".
>"The % (between \end{subfigure} and \begin{subfigure} or minipage) is really important; not suppressing it will cause a spurious blank space to be added, the total length will surpass \textwidth and the figures will end up not side-by-side."
I think many of the ills of this type this can be traced, at least in part, to TeX's only-partial whitespace invariance.
The shifts in behavior with an added newline (or absence thereof) or a comment, as above, simply wouldn't happen if the language ignored whitespace outright.
Of course, requiring tags for newlines/paragraphs would make raw TeX as un-readable as raw HTML. It might be worth the trade for the predictability, but for the era in which TeX was written, one without IDEs or LyX, I suspect that Knuth made the right choice.
I am not going to argue about the tastes here but please consider the fact that TeX is a typesetting language. Ignoring whitespace altogether would make .tex files unreadable (as a comment below states). I am again puzzled by complaints about whitespace treatment in TeX (which has facilities for handling it, like \ignorespaces) at the same time having no objections to the choices made by, say, Python (which to me makes a totally wanton decision about such, also I am not suggesting that you personally are inconsistent in your opinions). Also, poorly written LaTeX macros in this case are not a reason to blame TeX itself.
I suspect that the vast majority of people who say they've used TeX have actually used LaTeX - and don't realize just how massively powerful TeX itself is. Much of the LaTeX macros are quite simple; TeX does the heavy lifting for so much of it.
An amazing thing about the TeX Book is that due to TeX's stability, it's mostly still current. It was written in a day when the graphical computer was a special thing that required a trip to the lab. So the stuff it has about how to run TeX is dated.
But once you know how to run TeX at your "site" (as they would have called it then) all the stuff on the language (that is, practically the whole book) is still good.
> one of the things that TeX can’t do particularly well is typesetting on a grid. This is something that people typesetting bibles really need to have. There are various hacks to try to make it happen, but they’re all horrible. In SILE, you can alter the behaviour of the typesetter and write a very short add-on package to enable grid typesetting."
> For two-sided prints with very thin paper, matching base lines would look much better. Especially in two-column documents it may be desirable to have baselines of adjacent lines at exactly the same height.
TeX usually tries to adjust the space between lines so that a single typeblock is more visually appealing. Unfortunately, doing this adjustment independently for side-by-side columns leads to output where the text lines don't line up, and that is way less visually appealing.
If you do a Google image search for [bible] and look at typical layouts used, I think this might be apparent.
Less obvious is that bibles are often printed on thin paper to fit a big document in a small space, so having things line up on one side of a leaf to the other is more important than in most other books.
Context-free languages are the standard for decades now. They haven't been in the very early days of computing due to a lack of formal education of programmers and because they come with a certain, but tiny, memory requirement.
What Knuth did with TeX was a violation of KISS. There's no good reason, but several downsides, to mix markup, code, and interpreter state like this.
I am not sure what you mean by context-free languages because literally none of the modern languages have context-free grammars. This includes HTML and XML although I would not call them programming languages per se. C++ grammar is undecidable. On the other hand, TeX's core typesetting language (i.e. not using its macro facilities) is context-free. Quite trivial in fact. Pascal, which is the language used to build TeX is context-free (even LL(1)), as well, which cannot be said about, say, C.
I would also be quite hesitant to claim that Knuth, who literally pioneered modern LR parsing theory (and deservedly got a Turing award for it) somehow lacked knowledge or awareness of cutting edge parser design techniques.
Finally, there is absolutely good reasons to design TeX as Knuth did. That TeX withstood the test of time for over three decades is a perfect testimony to that.
As someone who is intimidated by the thought of programming TeX, I think the following comment which asserts that it is badly designed seems unlikely to be coming out of nowhere. (MetaFont is apparently better than TeX)
I'll quote the key part here in case that link stops working.
> TeX has two programming systems, the "mouth" (which does macro expansion essentially) and the "stomach" (which typesets and does assignments). They run only loosely synchronised and on-demand.
> For programming purposes, they are a pairing of a blind and a lame system since the "stomach" is not able to make decisions based on the value of variables (conditionals only exist in the "mouth") and the "mouth" is not able to affect the value of variables and other state.
> While eTeX has added a bit of arithmetic facilities that can be operated in the mouth, as originally designed the mouth does not do arithmetic. There is a fishy hack for doing a given run-time specified number of iterations in the mouth that relies on the semantics of \romannumeral which converts, say, 11000 into mmmmmmmmmmm.
> Because of the synchronisation issues of mouth and stomach, there is considerable incentive to get some tasks done mouth-only. Due to the mouth being lame and suffering from dyscalculia, this is somewhat akin to programming a Turing machine in lambda calculus.
> TLDR: the programming paradigm of the TeX language is awful.
This description of TeX's process (originally by Knuth himself) is quite tongue in cheek but I fail to see why this makes TeX's programming paradigm 'awful'. Think of it this way: both C and C++ (even Rust has macros, which I heard is the new hotness) have a preprocessor, which is exactly what TeX's 'mouth' is. The fact that the language of the preprocessor looks very similar to the main language does not make it weird (Rust again...). 'Loosely synchronised and on demand' is a very uncharitable description of what is happening: would you call JavaScript paradigm awful just because events are asynchronously processed? One tricky part of TeX is output routines running asynchronously with the main paragraph typesetting algorithm. This is indeed a compromise made by Knuth simply because in his days the thought of keeping the whole book in memory (which is what a perfect page breaking algorithm would require) was pure science fiction. A mildly augmented design will fix this (indeed, some modern implementations offer exactly this) but the solution found by Knuth is still incredibly beautiful. Also, 'the mouth does not do arithmetic' is not true at all. While I am not a big fan of LaTeX, its developers are incredibly talented programmers and they have implemented a fully expandable (in TeX parlance, taking place entirely in its 'mouth') floating point library in TeX (it uses some eTeX extensions for efficiency but they can be avoided as well; I speak from experience, since I do not use eTeX).
Finally, please do not let your fear stop you from enjoying TeX: most users do not do any serious programming and its output is aesthetically superb. If one day you decide to write some tricky macros, there is a vast TeXlore that awaits.
I'll probably try SILE just because it does not support macro.
I personally think the adoption macro is the embodiment of non intuitive programming languages and why a modern programming languages, for example Rust is supporting macro is beyond me.
I'm not alone in this regard, D a modern successor of C and C++ does not support macro.
Hopefully one day Walter will write document on "Macro Considered Harmful" to enlighten the subject.
I respect your stance (although I disagree strongly) but you might consider something other than Sile, as Chapter 6 of Sile's manual is called 'SILE Macros and Commands'. Also from reading about D, his strongest objection to using macros is that they do not respect the scope. This is probably the main source of macros' strength as code generators. Again, I respect your aversion to macros, I am not sure I understand the reasons, though.
The SILE approach for macro is very limited and sane. I think that's purposely to make sure that it's simple to understand, intuitive and maintain. More complex activities or programming is delegated to Lua as mentioned in the Simon's TUGboat paper:
> SILE’s \define command provides an extremely restricted macro system for implementing simple tags, but you are deliberately forced to write anything more complex in Lua. (Maxim: Programming tasks should be done in programming languages!)
It's technically not macro as mentioned in the same page link:
>C preprocessing expressions follow a different syntax and have different semantic rules than the C language. The C preprocessor is technically a different language. Mixins are in the same language.
D shows that it's possible to be a very powerful and potent programming language for DSL, etc, without all the conventional macro abuse and misuse.
If anyone asked me how come Python has becoming very popular nowadays, the answer will be it's one of the most intuitive and user friendly programming languages ever designed, and that's mainly due to the fact that it does not support macro [1].
It's not a C preprocessor macro. Those are a very specific and limited subset of macros, and that is the main reason why they're criticized.
Broadly speaking, if something can generate parametrized chunks of AST on the go, it's a macro. Which is exactly what D mixins do.
As for Python, it doesn't really need macros because everything is runtime. But if you need it, compile() and eval() are there, and they can be abused much worse than any macro facility ever could.
It really depends on your definition of macro [1]. Modern programming languages like Scala, Julia, Nim, Rust etc are supporting hygienic macro.
According to D authors Mixin is not macro because Mixin still looks like D language while supporting macro would meant the inclusion of macro (hygienic or not), and depending on the implementation, most often than not will render the language unrecognizable (become non intuitive). Again if someone is purposely writing a DSL in D for generating a new language's AST then that's perfectly fine. Essentially the output is another programming language because it is the intentional product of the exercises but the original programming language is still intuitive.
I suppose you can have runtime macro like VBA but Python language designers refused to incorporate it due to issues as mentioned beforehand.
Surely hygienic macros are a subset of macros, as follows from their name?
I still don't see why D's mixins aren't hygienic macros. And the D documentation doesn't make that claim, either - it only highlights the difference with C preprocessor, not macros in general.
OTOH I can't think of anything in VB6/VBA that resembles macros, runtime or otherwise?
Mixin is not macro according to D authors. If you dig into Walter's past posts, perhaps you can find better explanations there than mine.
You can have macro in any runtime for example Ruby language, and that's the main reason for the maxim "Rails is not Ruby". That's also the reason Ruby is so powerful and to create a similar feat in Java will be close to impossible [2]. But at what cost? That's why Python now is much more popular even though Ruby is much more powerful. You can, however, create Rails clone in D language without all the macro nonsense [3].
> I work with TeX quite a lot and I can't even really understand intuitively the principles of the language design let alone the grammar.
Read the TeXbook. No, really, it is a beautifully written text. The program it describes is quite elegant. Most of the complexity comes from modern LaTeX packages, not from the core tool.
TikZ is the worst for this. I don't usually have any problem picking up esoteric programming languages, but whatever the hell is going in with TikZ/TeX will be forever beyond my comprehension.
Oh yes, I find TikZ very useful and it's an amazing piece of engineering. My point is more that I find it very difficult to understand how it works on a deep level.
