Fine, but investment bankers are still a small minority in finance. I lack the proper English terms for the different banking job descriptions, but bankers dealing with customers personally (loan approval etc., which for commercial clients can be pretty big), internal auditing, etc. have standard hours and, if they are doing a good job, a good career.
Those people that you're thinking of are local bank branch managers. They can have a decent middle-class career, but they are hardly the people you think of when you think "high-paying banker". PayScale.com lists their salary as being in the $42K-64K range:
