I think you should elaborate further on what your skill set/s are, so you can get specific feedback on stack, hosting and implementation.
To get started on hosting, it's hard to beat something like Digital Ocean; $5 or $10 / month and you can run all your test infrastructure (and if you feel like staying with them, push a button and scale as your app demands it). There are a few other good cheap options to get started, such as OVH, if you want a dedicated box; ramnode is interesting.
I can perform system admin work and what I consider basic scripting. Today's task for instance is downloading debian 7, installing, adding beautiful soup and playing with it to obtain data for statistical analysis related to sports.
My biggest issue is that I have not played on machines that are remote site and having to learn the current best practices. It has been about 13 years since I configured Apache.
Digital Ocean sounds like it is worth looking into.
It sounds like your biggest wall is going to be the actual app programming. Do you have any experience with Java or Objective-C? If learning those languages isn't likely in the cards, you still have a couple of other options, namely a HTML5 + javascript compiled app (this might get shit on here at HN, but depending on your app demands, it can definitely work fine; the html + javascript gets compiled to native code). These are worth looking at:
I've used all three. Sencha has a bit of a steep learning curve, but is nice. The Intel Framework is very easy to get started with; it's pretty fast for what it does; and so long as you're not trying to build the next Instagram, it'll work for a lot of different types of average apps. Using the Intel Framework, the scripting is similar to jquery, it fully supports javascript, and you can easily make ajax calls from the compiled app to a php / python / ruby back-end hosted somewhere else.
For a normal app, the systems side is pretty trivial. A modestly well configured $150 dedicated box will support 99% of all apps these days; and the bottom 50% of apps, based on usage, can be supported on a $20 Digital Ocean style account. If you're going to have a large active user base, then a very well run back-end is obviously that much more important. Today's powerful hardware provides a huge amount of slack on the back-end for a typical app, it's simply unlikely you're going to need to run at hyper efficiency.
I'll bookmark this thread, if you have questions I'd be happy to answer as many as I can here. It can seem like (is) a daunting challenge to build and launch an app.
It really depends on what the app does, if it is something that the UX will differ greatly between a mobile device and a desktop, then I would say build a completely separate mobile version of the site. However, if it is only subtle changes then just create a responsive app that looks good on all devices.
To get started on hosting, it's hard to beat something like Digital Ocean; $5 or $10 / month and you can run all your test infrastructure (and if you feel like staying with them, push a button and scale as your app demands it). There are a few other good cheap options to get started, such as OVH, if you want a dedicated box; ramnode is interesting.