The problem with banking is the lack of regulation. See Canada as an example, we have 6 banks and no bailouts. The free-market does not work all the time and banks are not like other companies - as such, they should be heavily controlled by regulation to ensure stability and access to finance.
What specific regulations made the biggest difference?
The biggest factor I've read about is that Canada -- as a society and government -- never got on the "promote home ownership at all costs" bandwagon. There's no equivalent of Fannie and Freddie, and the traditional mortgage requirement of ~20% down remains. Thus Canada hasn't had the same sort of housing bubble, nor as many dubious mortgages, nor widespread mortgage securitization.