> TypeScript, Swift, Kotlin, and Scala take string interpolation to the furthest extreme of encouraging actual code being embedded inside strings. So to highlight a string, one must count curly brackets and maintain a stack of parser states.
Presumably this is also true in Python - IIRC the brace-delimited fields within f-strings may contain arbitrary expressions.
More generally, this must mean that the lexical grammar of those languages isn't regular. "Maintaining a stack" isn't part of a finite-state machine for a regular grammar - instead we're in the realm of pushdown automata and context-free grammars.
Is it even possible to support generalized string interpolation within a strictly regular lexical grammar?
Complicated interpolation can be lexed as a regular language if you treat strings as three separate lexical things, eg in JavaScript template literals there are,
`stuff${
}stuff${
}stuff`
so the ${ and } are extra closing and opening string delimiters, leaving the nesting to be handled by the parser.
You need a lexer hack so that the lexer does not treat } as the start of a string literal, except when the parser is inside an interpolation but all nested {} have been closed.
Presumably this is also true in Python - IIRC the brace-delimited fields within f-strings may contain arbitrary expressions.
More generally, this must mean that the lexical grammar of those languages isn't regular. "Maintaining a stack" isn't part of a finite-state machine for a regular grammar - instead we're in the realm of pushdown automata and context-free grammars.
Is it even possible to support generalized string interpolation within a strictly regular lexical grammar?