Yeah, you can't just take the responses at face value.

Watching five subjects in a row get to one point and then get confused is important, but they might never mention it if you just ask them directly afterwards.

I've also observed talk between users that started out "oh, I hated it" - then they'll go on to praise the functional aspects - and it turns out they just didn't like the color scheme. Which is equally valid if it generates consistent responses.