Another good book is "Mathematical statistics with applications" by Wackerly et al. It's focus is more on the application of the toolkit from probability, but about half the book focuses on the background of probability theory. What's great is that by the end of the book you will have a great introductory perspective of both frequentist and bayesian statistics. There are some gaps in the book, but it's a great reference nevertheless.
