I have some questions based on previous experience many moons ago experience (3 or 4 yrs).
When I tried to port some of our platform to AppEngine I had some issues with various Java libraries (writing tmp files, opening sockets, etc). Once I sort of got past that and used some of Googles own APIs (which is sort of annoying that I had to couple my stuff) I had timeout issues. See we have to integrate with all these enterprise third parties (SOAP/REST) and these guys can be really slow.
We also couldn't use our own pub/sub (AMQP/RabbitMQ) so that was going to have to be ported as well.
Things must have changed because its hard for me to imagine Spotify being able to get all their needs met on a rather coupling platform.
* How does one write code for Google Cloud while being agnostic of Google Cloud?
* Maybe appengine supports AMQP?
* Or maybe the container engine fixes these problems? (ie need something more custom run it in a docker image).
Our current platform runs on Rackspace and Digital Ocean. These guys are IaaS so your basically doing DevOps which is a pain... but Rackspace does have kick ass customer service.
AppEngine is Google Cloud Platform's PaaS offering. Its the oldest product on Google Cloud Platform, but its not one of the ones mentioned that Spotify is using (the Spotify post doesn't talk much about the products, but Google's [0] does.)
The Google Cloud Platform services they are identified as using are: Cloud Storage, Compute Engine, Cloud Datastore, Cloud Bigtable, Direct Peering, Cloud VPN, Cloud Router, Cloud Pub/Sub, Cloud Dataflow, BigQuery, and Cloud Dataproc.
So, its not surprising that they might be doing something that Google AppEngine doesn't support particularly well.
I guess they are not entirely on Google Cloud but rather their data warehousing / big data stuff is now?/going? to be on GCP.
I don't really call big data processing "backend" but rather the microservices the backend. Its not clear to me if those are going to be ported over or if they are already.
Does spotify plan on porting everything over or just the heavy computing? A detailed techy case study I hope will follow soon as I am interested (the DevOps pain is very real).
> I guess they are not entirely on Google Cloud but rather their data warehousing / big data stuff is now?/going? to be on GCP.
I don't know why you would guess that: the Google announcement indicates that the migration has both a "services" track and a "data" track. And Spotify's announcement certainly seems to indicate that they are moving off of their own datacenters, onto Google Cloud Platform, and not in a limited way.
So... basically nothing is live yet? Are you from Google? Is anything live. I admit I got confused by the OP title: "Spotify moves its back end to Google Cloud".
Adding to what dragonwriter said, we now have an IaaS offering (Compute Engine) that launched publicly in 2013. You get raw virtual machines letting you be agnostic of Google Cloud's particular offerings.
If you're looking for "just run my app" but with a bit more customization / flexibility than App Engine (like running your own RabbitMQ) then yes, Container Engine (https://cloud.google.com/container-engine) is our hosted Kubernetes offering. You don't need to manage with the VMs yourself, just choose a cluster size and go.
Disclaimer: I work on Compute Engine, not really App Engine or Container Engine (but I'm familiar with them all).
When I tried to port some of our platform to AppEngine I had some issues with various Java libraries (writing tmp files, opening sockets, etc). Once I sort of got past that and used some of Googles own APIs (which is sort of annoying that I had to couple my stuff) I had timeout issues. See we have to integrate with all these enterprise third parties (SOAP/REST) and these guys can be really slow. We also couldn't use our own pub/sub (AMQP/RabbitMQ) so that was going to have to be ported as well.
Things must have changed because its hard for me to imagine Spotify being able to get all their needs met on a rather coupling platform.
* How does one write code for Google Cloud while being agnostic of Google Cloud?
* Maybe appengine supports AMQP?
* Or maybe the container engine fixes these problems? (ie need something more custom run it in a docker image).
Our current platform runs on Rackspace and Digital Ocean. These guys are IaaS so your basically doing DevOps which is a pain... but Rackspace does have kick ass customer service.