According to Wikipedia, he owed the taxes (to Ohio, not the IRS) but was likely unaware of it.

https://en.wikipedia.org/wiki/Joe_the_Plumber#Tax_controvers...

It was Ohio Job&Family Services and the Highway Patrol which started trying to dig up dirt on him in retaliation for his criticism of Obama, not the IRS.