It isn't the people going to the university that will be the problem: it will be trying to explain what you are doing to your parents, employers, or friends. I got my degree from "The College of Creative Studies", which has one of the top physics programs in the nation; this is understood by grad programs you talk to, but all those people external to the academic process just look at you as if you are insane, and went to some kind of lala-happy program.