Depends upon the heating system. A cheap air to air, single unit heat pump does not cost much but if you want anything more then it costs more. A lot of houses with a basic heat pump use it to reduce the amount of other heating used and pump hot air into a single area of the house, with additional electric heaters in other rooms and a water heater for hot water.
However to totally replace your entire heating needs with an all-in-one system is much more complicated than that. The heat pump I have alone in Scandinavia costs roughly $6-10k. That's powering water heating in every room plus hot water in the taps, zero additonal heating of air or water required. It's the size of bulky floor to ceiling fridge and does wonders for my electricity bill!
However to totally replace your entire heating needs with an all-in-one system is much more complicated than that. The heat pump I have alone in Scandinavia costs roughly $6-10k. That's powering water heating in every room plus hot water in the taps, zero additonal heating of air or water required. It's the size of bulky floor to ceiling fridge and does wonders for my electricity bill!