I'm confused why this would be a problem. UTF-8 and UTF-16 (the only two common unicode subsets) are a maximum of 4 bytes wide (and, most commonly, 2 in English text). The ASCII representation you gave is 6-bytes wide. I don't know of many ASCII unicode representations that have less bytewidth than their native Unicode representation.

Same goes for other characters such as \n, \0, \t, \r, etc. All half in native byte representation.

> Moment you have "Hi \uxxxx isn't the UTF nice?" you will probably have to allocate.

Depends on what you are doing with it. If you aren't displaying it (and typically you are not in a server application), you don't need to unescape it.

And this is indeed something that the C++ Glaze library supports, to allow for parsing into a string_view pointing into the original input buffer.

It’s just two pointers the current place to write and the current place to read, escapes are always more characters than they represent so there’s no danger of overwriting the read pointer. If you support compression this can become somewhat of and issue but you simply support a max block size which is usually defined by the compression algorithm anyway.

If you have a place to write, then it's not zero allocation. You did an allocation.

And usually if you want maximum performance, buffered read is the way to go, which means you need a write slab allocation.

> If you have a place to write, then it's not zero allocation. You did an allocation.

Where did that allocation happen? You can write into the buffer you're reading from, because the replacement data is shorter than the original data.

You have a read buffer and somewhere where you have to write to.

Even if we pretend that the read buffer is not allocating (plausible), you will have to allocate for the write source for the general case (think GiB or TiB of XML or JSON).

> You have a read buffer and somewhere where you have to write to.

The "somewhere you have to write to" is the same buffer you are reading from.

Not if you are doing buffered reads, where you replace slow file access with fast memory access. This buffer is cleared every X bytes processed.

Writing to it would be pointless because clears obliterate anything written; or inefficient because you are somehow offsetting clears, which would sabotage the buffered reading performance gains.

Maybe I missed it, but ITT we were talking about C buffers, not buffered reads.

I thought we were talking about high performance parsing. Of which buffered reads are one. Other is loading entire document into mutable memory, which also has limitations.

> Practically there is character escaping

The voice of experience appears. Upvoted.

It is conceivable to deal with escaping in-place, and thus remain zero-alloc. It's hideous to think about, but I'll bet someone has done it. Dreams are powerful things.