Can you tell me what specific challenges you've had with deploying non-HTTP services using Kubernetes? Many features are http-centric, but others seem fairly agnostic to me. What kind of non-http services are we talking about anyway?
Deploying them is easy. Figuring out the correct incantations to bind a load balancer or ingress port to redis/kafka/mysql/whatever and/or expose them in some way other than "let the ingress figure out the host from headers/SNI and map to some thing in the Service" is somewhat less than intuitive.
Sure, you could argue that you shouldn't expose backend services like that without an API endpoint at all, because one application->one namespace, or you could use a service mesh/consul to expose it to the things that need it, but it's an entirely different mindset from "I have a postgres cluster in public/private cloud"
Not OP and I don't do K8 in production, but I understand their complaint. You couldn't eg. deploy a game server easily.
They typical "web app" flow is Client -> ReverseProxy -> StatelessApp -> DatabaseWithState and back. The typically expectation is HTTP requests that aren't "long lived". Even running a database would be harder since you need to consider where the storage of the content is going (you'd want backups and snapshots etc) and how it'd get there. If the StatelessApp goes down (or you need more), you just add containers.
One example that wouldn't work well on k8 that I'm sure isn't very common is like a Minecraft server. It is a persistent TCP connection to a server that reads writes data to the FileSystem.
one big pain point is that k8s handle inbound (initiated by entity outside of cluster) and outbound connection (initiated by pod) differently and separately, it is difficult to using same public IP address for inbound dst and outbound source, such symmetric is needed for certain applications. I know there is work around for this case, e.g. only uses a single public IP for both inbound and outbound for whole cluster on aks, or use dedicate node, but these workarounds are just workarounds, all have issues.
