Hacker News new | past | comments | ask | show | jobs | submit login

It's part of the tokenizer. t contains data in the format of [char 0][char 1][base 64 encoded token id …], whereas the id is as long as the entry is smaller than 'b'. So it's ('++', `#m`), ('--', `%am`), ('*@', 'R<^1c'), and so on. '@' is used as a wildcard. As you can see this part parses operators.



Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: