Hacker News new | past | comments | ask | show | jobs | submit login

I am on mobile now, and can't provide a simple link, but it is given in Cover&Thomas "Elements of Information Theory", in the episode that discusses entropy of markov processes. I can find pages in google books but they won't zoom big enough to read...

IIRC, the proof requires the markov chain be irreducible, and extends to the general case by summing over the irreducible parts; and that entropy will stay the same or increase while converging to the stationary distribution over states.

(Although it has now been 20 years since I dealt with these things so I might be misremembering. Time to retread Cover&Thomas I guess...)




Cover & Thomas, 2nd Edition, Jul 2006, pg 81, section 4.4 - entropy rate of markov processes, I did not remember all the conditions needed for this to hold, please read if you are interesting.

[0] staff.ustc.edu.cn/~cgong821 /Wiley.Interscience.Elements.of.Information.Theory.Jul.2006.eBook-DDU.pdf seems to have a copy indexed by Google. I suspect it is not legitimate




Consider applying for YC's Spring batch! Applications are open till Feb 11.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: