Actually both IE and Firefox now (I think) insert tabs to the left. I don't know why it became a 'thing' at some point, I dislike it enormously as well.
I like it that way. It allows me to think of the tabs like a stack. In any case, you can easily change the default, it's not like you use a new machine every week that needs to be configured again.
Check "Insert documents to the right of existing tabs".