That is not a good measure of debt. Government debt is only meaningful as a fraction of GDP (large countries have more debt than small ones, e.g.). So in this case you want this graph:
http://research.stlouisfed.org/fred2/graph/?g=2TG
which is (Gross Federal Debt)/(Gross GDP), and was clearly decreasing.
The distinction between gross and public-held debt is also not very useful. The difference between them is government debt held by parts of the government, which is money that the government owes to itself. Public-held debt was decreasing even more:
http://research.stlouisfed.org/fred2/graph/?g=2TI
When it comes to talk about paying back the debt, that 2TG graph is misleading, though. Throughout the entire period of debt reduction starting in 1950, the government did not pay down its debt. Instead, the economy was growing.
So the graph is misleading in the sense that some people may draw the conclusion that government has used austerity measures, run surpluses, to reduce its debt. In fact, the government did not run surpluses during those periods in which the debt-to-GDP ratio was reduced so drastically. It ran deficits, which helped an already growing economy further along.
Implementing austerity is not going to start a sudden and drastic drop of debt-to-GDP measures in the current economic climate. If you honestly care about debt-to-GDP (why, btw?), you'd do better to stimulate the hell out of the economy.
The distinction between gross and public-held debt is also not very useful. The difference between them is government debt held by parts of the government, which is money that the government owes to itself. Public-held debt was decreasing even more: http://research.stlouisfed.org/fred2/graph/?g=2TI