
These projects are each geared toward simulating a multi-node kubernetes cluster (where one container represents each node). There are already two kubernetes SIGs that support running kubernetes in docker. Why do we need another K8s-in-docker solution? It began to feel as though we were hacking Minikube to work in an environment that it isn't designed for (inside a docker container). We spent some time in the Minikube codebase, hoping to modify the pre-flight check. That file wont exist inside your container, even if br_netfilter kernel module is enabled, because the br_netfilter kernel module is not namespace aware. One such check is to ensure the br_netfilter kernel module is enabled by checking the file /proc/sys/net/bridge/bridge-nf-call-iptables. Minikube leverages something called "pre-flight checks" that ensure certain criteria are met before launching your cluster. While there is still bare-metal support in Minikube, we had trouble getting it running inside a container.

At one time, Minikube supported a system called "localkube" which had been leveraged to handle this case, but that feature has since been deprecated.

Many CI systems don't allow spinning up a VM inside your job (even if they did, it's likely to be slow). While Minikube is a great tool for local development, we had trouble getting it running in our CI system (Travis). We can check the output of a running pod or exec into a pod and run commands. With this, our resources are created inside dockernetes. Wait for kubernetes to launch with systemctl is-active -quiet multi-user.target. Launch dockernetes (and mount the repo into the dockernetes container).ĭocker exec into the dockernetes container and: We have a target make k8s_test that does the following:

We include a kubernetes manifest in our github repo. Systemctl is-active -quiet multi-user.target How we use dockernetes at Lyft From inside the container, you can check if kubernetes has finished launching by running the following command. Kubernetes boots in the container via systemd. Once launched, shell into the container with docker exec -it /bin/sh and run commands like kubectl cluster-info
