I'm guessing you're not married and don't have kids? In general, children don't have the right to decide that their parents' lifestyle isn't for them and leave.
But the bigger point is that the kind of mindset you're espousing ("It's my life, I'll do whatever I want without regard for my family") doesn't really work.
I agree, but just wanted to point out that the opposite approach (allowing the needs of your family to completely subjugate all of your own lifestyle/work dreams) is also an awful mistake that doesn't work either.
But the bigger point is that the kind of mindset you're espousing ("It's my life, I'll do whatever I want without regard for my family") doesn't really work.