I recently got a record player and have been amassing a small collection of my favourite albums. Including the reasons you gave, my excuse for buying/listening to records is I like having a physical representation of albums I like, and something that won't disappear because a contract expired with a streaming service.
Stick with it. I discovered some great music buying random stuff from the <$5 section. You'll meet fun people at the right record stores (if you're in Minneapolis, I have some suggestions). Every once in a while one of your LPs will turn into a nice investment piece. If you haven't tried repairing/restoring your own turntable, I recommend it. The engineering inside all the little mechanisms in an old turntable are fun to tinker with.
