Hacker News new | past | comments | ask | show | jobs | submit login

Yes, as long as you include punctuation in your definition of "word". I like to think of it like this:

A lexer takes a sequence of characters and produces a sequence of tokens. List in, list out. It just chunks runs of characters together.

A parser takes in a sequence of tokens and produces a tree. It produces a nested data structure from a flat input.




Right, a lexer is a mapping function. A parser is a fold.


A lexer is a fold as well - there are usually fewer elements in the output list than in the input list. A mapping function implies a 1:1 correspondence, because the recursion is abstracted out into map() and the map function can't accumulate any state during the computation. (At least in traditional FP technology; the Mapper in MapReduce isn't actually a true map function, it's more like an unfold.)


Ah true I am incorrect on the lexer as a map. Had I thought on that half a second more I would have saved myself some embarrassment. :)




Consider applying for YC's Spring batch! Applications are open till Feb 11.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: