Yeah, it's tricky. I remember it by breaking it into two pieces. "2>" means redirect stderr (file descriptor 2) to the following token. "&1" is the token for stdout (kinda like the address-of operator). So taken together, "2>&1" means redirect stderr to stdout.
Bash-only shortcuts like &> don't improve the situation :)