> How did it work? Any interesting tweaks you can share?
For being such a naïve approach I think it works fantastically. It works better on some types of texts (newspaper articles, research papers) than others (poems, fiction). It was my first real programming project so the code has a certain sentimental meaning for me :)
The analysis code (https://github.com/peterldowns/bookshrink/blob/master/analys...) is very short and the comments include my thoughts on certain tweaks / approaches. Some quirks of the implementation: uses regex for sentence splitting (really!), doesn't perform any stemming, and weights proper nouns heavier than other words.
> BTW, I love the highlight-in-the-original-text option.
Thanks! The eventual goal was to build a tool that would make essay grading easier for teachers, although I never got around to it.
