Depends on how many points. It should scale with the amount of points the user has. A percentage based scheme seems fine to me. 0.1% your points for every downvote.
I'm not sure I'd like to spend six hundred points dealing with a single idiot; I'd have to develop a better battery of cut-and-paste denouncements instead. I'm not sure this would improve the overall signal-to-noise ratio.