I solve those problems by never using gem install. Actually I use gem install once, to install bundler in the @global gemset of every Ruby version I'm using. Then it's all Gemfiles.

I create a rvm gemset per project and I use bundle within it. Combined with the .ruby-version and the .ruby-gemset files this lets me switch between projects in zero time because I don't have to run bundle.

I deploy with bundle install --deployment (vendoring gems).