Docker released a beta version of Docker that includes Kubernetes support. I was excited to try it out on my Mac. Here are my notes and observations from experimenting with Docker for Mac with Kubernetes.
Installation
The Docker folks usually do a great job with a simple user experience, and installation was no exception. I downloaded the edge installer for Docker, which uninstalled my stable version of Docker. In the preferences pane, I enabled Kubernetes, and shortly thereafter, I had a working Kubernetes cluster.
Installing Docker for Mac with Kubernetes
I was also able to use the preexisting
kubectl
on my laptop. The installer assumes your Kubernetes configuration is stored in
$HOME/.kube/config
. If you have set KUBECONFIG to a different file, you’ll want to point it back to
config
.
Docker for Mac and Ingress
I decided to try installing Edge Stack, our Kubernetes-Native API Gateway built on the Envoy Proxy. Ambassador strives to be as idiomatic to Kubernetes as possible (e.g., it’s configured via annotations), so it’s a good real-world test for a Kubernetes implementation.
Docker for Mac is based on Kubernetes 1.8.2, so I installed Ambassador with RBAC:
---
apiVersion: v1
kind: Service
metadata:
labels:
service: ambassador-admin
name: ambassador-admin
spec:
type: NodePort
ports:
- name: ambassador-admin
port: 8877
targetPort: 8877
selector:
service: ambassador
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
name: ambassador
rules:
- apiGroups: [""]
resources:
- services
verbs: ["get", "list", "watch"]
- apiGroups: [""]
resources:
- configmaps
verbs: ["create", "update", "patch", "get", "list", "watch"]
- apiGroups: [""]
resources:
- secrets
verbs: ["get", "list", "watch"]
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: ambassador
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
name: ambassador
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: ambassador
subjects:
- kind: ServiceAccount
name: ambassador
namespace: default
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: ambassador
spec:
replicas: 1
template:
metadata:
labels:
service: ambassador
spec:
serviceAccountName: ambassador
containers:
- name: ambassador
image: datawire/ambassador:0.21.0
imagePullPolicy: Always
resources:
limits:
cpu: 1
memory: 400Mi
requests:
cpu: 200m
memory: 100Mi
env:
- name: AMBASSADOR_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
livenessProbe:
httpGet:
path: /ambassador/v0/check_alive
port: 8877
initialDelaySeconds: 3
periodSeconds: 3
readinessProbe:
httpGet:
path: /ambassador/v0/check_ready
port: 8877
initialDelaySeconds: 3
periodSeconds: 3
- name: statsd-sink
image: datawire/prom-statsd-exporter:0.6.0
I then deployed an Ambassador
LoadBalancer
service:
---
apiVersion: v1
kind: Service
metadata:
labels:
service: ambassador
name: ambassador
spec:
type: LoadBalancer
ports:
- name: ambassador
port: 80
targetPort: 80
selector:
service: ambassador
I wanted to try to connect to Ambassador, but this is what I saw:
$ kubectl get svc NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE ambassador 10.106.236.196 <pending> 80:30612/TCP 45m ambassador-admin 10.102.220.182 <nodes> 8877:31079/TCP 4h kubernetes 10.96.0.1 <none> 443/TCP 4h
Note that the
ambassador
service has been running for 45 minutes, and it was still for an external IP. Hopping on the Docker Slack channel, I found out that the service controller doesn’t update the service object (yet). It turns out that the service is actually exposed locally. So the following worked:
$ curl localhost:80
I added a mapping for Ambassador to route
/httpbin/
to the
httpbin.org
service with the following configuration YAML:
apiVersion: v1
kind: Service
metadata:
name: httpbin
annotations:
getambassador.io/config: |
---
apiVersion: ambassador/v0
kind: Mapping
name: httpbin_mapping
prefix: /httpbin/
service: httpbin.org:80
host_rewrite: httpbin.org
spec:
ports:
- port: 80
And it worked perfectly:
$ curl localhost:80/httpbin/ip { "origin": "65.217.185.138" }
Some other notes on Ingress
In some conversations on the Slack channel, I learned a few other quirks:
- To get a list of open ports, you can compile this binary. I haven’t tried this.
- The service controller does not yet handle collisions between competing services. So the last service will win.
Docker Stacks and CRDs
Docker includes a native integration between Docker Swarm and Kubernetes with a
stack
custom resource definition. I’ve heard from many users how they love the simplicity of Docker Compose , and the
stack
CRD seems like a compelling approach.
Conclusion
Docker for Mac with Kubernetes has a lot of promise. While there are the rough edges you’d expect with any beta software, the Docker team has done an amazing job of building a useful alternative to Minikube. In addition, I’m excited to see how they’ve thought through how to make the experience idiomatic for Kubernetes users. I’m looking forward to updates!