Lets say you have a micro-services app. Its got a bunch of containers that you’ve orchestrated out with Kubernetes. Deployments, Pods, Daemonsets all over the place. Autoscaling. You are happy. Now it comes time to implement that pesky ‘security’ step. You are a bit nervous, there’s no internal firewall, all the services listen on port 80, no encryption. All the passwords are hard-coded and in the global environment. No one would guess your l33t mysql password right? So you google ‘how is secur networx’. And you click I’m feeling lucky.
Good thing for you google was watching your previous searches and had the microphone on, so it not only corrected your txt-speak spelling but also selected Istio for you.
But suddenly you need to triple the capacity of your cluster. Lets take a look. Here’s kubectl top from my cluster. The lines in red are associated with the securing + auditing. See that last column? Seems we are using 8144MiB for monitoring the thing that is using 2259MiB. And don’t get me started on the CPU.
I said it before, the cloud doesn’t scale down.
Let’s look. istio-system + logging + monitoring == nearly all the resources!
NAMESPACE NAME CPU MEMORY default ingress-nginx-ingre 4m 146Mi default ingress-nginx-ingre 0m 3Mi istio-system istio-citadel-84fb7 0m 12Mi istio-system istio-egressgateway 2m 35Mi istio-system istio-galley-655c4f 13m 39Mi istio-system istio-ingressgatewa 3m 37Mi istio-system istio-pilot-6cd69dc 8m 84Mi istio-system istio-policy-77f684 89m 419Mi istio-system istio-policy-77f684 97m 521Mi istio-system istio-policy-77f684 99m 492Mi istio-system istio-policy-77f684 62m 345Mi istio-system istio-policy-77f684 63m 351Mi istio-system istio-sidecar-injec 13m 27Mi istio-system istio-statsd-prom-b 34m 23Mi istio-system istio-telemetry-77f 76m 440Mi istio-system istio-telemetry-77f 105m 559Mi istio-system istio-telemetry-77f 109m 525Mi istio-system istio-telemetry-77f 106m 574Mi istio-system istio-telemetry-77f 79m 437Mi istio-system prometheus-84bd4b97 51m 689Mi kube-system cert-cert-manager-6 2m 22Mi kube-system heapster-6c4947855f 0m 41Mi kube-system kube-dns-v20-5fd69f 18m 27Mi kube-system kube-dns-v20-5fd69f 18m 28Mi kube-system kube-proxy-5rhch 3m 36Mi kube-system kube-proxy-dxk9f 3m 42Mi kube-system kube-svc-redirect-d 11m 156Mi kube-system kube-svc-redirect-z 5m 110Mi kube-system kubernetes-dashboar 0m 15Mi kube-system metrics-server-64f6 0m 26Mi kube-system tiller-deploy-895d5 0m 45Mi kube-system tunnelfront-7794f9f 21m 16Mi logging elasticsearch-867b4 567m 1420Mi logging fluent-bit-56d6z 21m 11Mi logging fluent-bit-8cbnl 17m 12Mi logging logging-fluentd-69f 1m 59Mi logging logging-kibana-7684 1m 152Mi logging sysctl-conf-92l84 0m 0Mi logging sysctl-conf-hb2vn 0m 0Mi monitoring alertmanager-monito 1m 15Mi monitoring monitoring-exporter 3m 37Mi monitoring monitoring-exporter 1m 14Mi monitoring monitoring-exporter 1m 10Mi monitoring monitoring-grafana- 0m 35Mi monitoring monitoring-promethe 2m 30Mi monitoring prometheus-monitori 7m 176Mi socks carts-6994d7d589-6j 5m 340Mi socks carts-db-7dd64bfd7b 5m 96Mi socks catalogue-849865789 4m 47Mi socks catalogue-db-6d6667 3m 236Mi socks front-end-855684fd8 4m 118Mi socks orders-7d9cf5cb46-d 5m 350Mi socks orders-db-6db4678bf 5m 93Mi socks payment-6cdc5b656-8 4m 48Mi socks queue-master-7b99db 5m 301Mi socks rabbitmq-7c5fbf778d 7m 127Mi socks session-db-fdd649d6 3m 52Mi socks shipping-5b9ffdbdfb 5m 321Mi socks user-84ccd5fd57-2vp 4m 47Mi socks user-db-7dcc9649dc- 4m 83Mi