Ingress

This guide covers setting up ingress on a kind cluster.

Setting Up An Ingress Controller 🔗︎

We can leverage KIND’s extraPortMapping config option when creating a cluster to forward ports from the host to an ingress controller running on a node.

We can also setup a custom node label by using node-labels in the kubeadm InitConfiguration, to be used by the ingress controller nodeSelector.

  1. Create a cluster
  2. Deploy an Ingress controller, the following ingress controllers are known to work:

Create Cluster 🔗︎

Create a kind cluster with extraPortMappings and node-labels.

 
cat <<EOF | kind create cluster --config=-
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
  kubeadmConfigPatches:
  - |
    kind: InitConfiguration
    nodeRegistration:
      kubeletExtraArgs:
        node-labels: "ingress-ready=true"
  extraPortMappings:
  - containerPort: 80
    hostPort: 80
    protocol: TCP
  - containerPort: 443
    hostPort: 443
    protocol: TCP
EOF

Contour 🔗︎

Deploy Contour components.

 
kubectl apply -f https://projectcontour.io/quickstart/contour.yaml

Apply kind specific patches to forward the hostPorts to the ingress controller, set taint tolerations and schedule it to the custom labelled node.

{
  "spec": {
    "template": {
      "spec": {
        "nodeSelector": {
          "ingress-ready": "true"
        },
        "tolerations": [
          {
            "key": "node-role.kubernetes.io/control-plane",
            "operator": "Equal",
            "effect": "NoSchedule"
          },
          {
            "key": "node-role.kubernetes.io/master",
            "operator": "Equal",
            "effect": "NoSchedule"
          }
        ]
      }
    }
  }
}

Apply it by running:

 
kubectl patch daemonsets -n projectcontour envoy -p '{"spec":{"template":{"spec":{"nodeSelector":{"ingress-ready":"true"},"tolerations":[{"key":"node-role.kubernetes.io/control-plane","operator":"Equal","effect":"NoSchedule"},{"key":"node-role.kubernetes.io/master","operator":"Equal","effect":"NoSchedule"}]}}}}'

Now the Contour is all setup to be used. Refer to Using Ingress for a basic example usage.

Additional information about Contour can be found at: projectcontour.io

Ingress Kong 🔗︎

Deploy Kong Ingress Controller (KIC).

 
kubectl apply -f https://raw.githubusercontent.com/Kong/kubernetes-ingress-controller/master/deploy/single/all-in-one-dbless.yaml

Apply kind specific patches to forward the hostPorts to the ingress controller, set taint tolerations, and schedule it to the custom labeled node.

{
  "spec": {
    "replicas": 1,
    "template": {
      "spec": {
        "containers": [
          {
            "name": "proxy",
            "ports": [
              {
                "containerPort": 8000,
                "hostPort": 80,
                "name": "proxy-tcp",
                "protocol": "TCP"
              },
              {
                "containerPort": 8443,
                "hostPort": 443,
                "name": "proxy-ssl",
                "protocol": "TCP"
              }
            ]
          }
        ],
        "nodeSelector": {
          "ingress-ready": "true"
        },
        "tolerations": [
          {
            "key": "node-role.kubernetes.io/control-plane",
            "operator": "Equal",
            "effect": "NoSchedule"
          },
          {
            "key": "node-role.kubernetes.io/master",
            "operator": "Equal",
            "effect": "NoSchedule"
          }
        ]
      }
    }
  }
}

Apply it by running:

 
kubectl patch deployment -n kong proxy-kong -p '{"spec":{"replicas":1,"template":{"spec":{"containers":[{"name":"proxy","ports":[{"containerPort":8e3,"hostPort":80,"name":"proxy-tcp","protocol":"TCP"},{"containerPort":8443,"hostPort":443,"name":"proxy-ssl","protocol":"TCP"}]}],"nodeSelector":{"ingress-ready":"true"},"tolerations":[{"key":"node-role.kubernetes.io/control-plane","operator":"Equal","effect":"NoSchedule"},{"key":"node-role.kubernetes.io/master","operator":"Equal","effect":"NoSchedule"}]}}}}'

Apply kind specific patch to change service type to NodePort:

{
  "spec": {
    "type": "NodePort"
  }
}

Apply it by running:

 
kubectl patch service -n kong kong-proxy -p '{"spec":{"type":"NodePort"}}'

KIC can be used to configure ingress now.

You can try the example in Using Ingress at this moment, but KIC will not automatically handle Ingress object defined there. Ingress resources must include ingressClassName: kong under spec of Ingress for being controlled by Kong Ingress Controller (it will be ignored otherwise). So once the example has been loaded, you can add this annotation with:

 
kubectl patch ingress example-ingress -p '{"spec":{"ingressClassName":"kong"}}'

Refer Using Ingress for primary example usage.

Ingress NGINX 🔗︎ 

kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/main/deploy/static/provider/kind/deploy.yaml

The manifests contains kind specific patches to forward the hostPorts to the ingress controller, set taint tolerations and schedule it to the custom labelled node.

Now the Ingress is all setup. Wait until is ready to process requests running:

 
kubectl wait --namespace ingress-nginx \
  --for=condition=ready pod \
  --selector=app.kubernetes.io/component=controller \
  --timeout=90s

Refer Using Ingress for a basic example usage.

Using Ingress 🔗︎

The following example creates simple http-echo services and an Ingress object to route to these services.

kind: Pod
apiVersion: v1
metadata:
  name: foo-app
  labels:
    app: foo
spec:
  containers:
  - command:
    - /agnhost
    - netexec
    - --http-port
    - "8080"
    image: registry.k8s.io/e2e-test-images/agnhost:2.39
    name: foo-app
---
kind: Service
apiVersion: v1
metadata:
  name: foo-service
spec:
  selector:
    app: foo
  ports:
  # Default port used by the image
  - port: 8080
---
kind: Pod
apiVersion: v1
metadata:
  name: bar-app
  labels:
    app: bar
spec:
  containers:
  - command:
    - /agnhost
    - netexec
    - --http-port
    - "8080"
    image: registry.k8s.io/e2e-test-images/agnhost:2.39
    name: bar-app
---
kind: Service
apiVersion: v1
metadata:
  name: bar-service
spec:
  selector:
    app: bar
  ports:
  # Default port used by the image
  - port: 8080
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: example-ingress
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
  rules:
  - http:
      paths:
      - pathType: Prefix
        path: /foo(/|$)(.*)
        backend:
          service:
            name: foo-service
            port:
              number: 8080
      - pathType: Prefix
        path: /bar(/|$)(.*)
        backend:
          service:
            name: bar-service
            port:
              number: 8080
---

Apply the contents

 
kubectl apply -f https://kind.sigs.k8s.io/examples/ingress/usage.yaml

Now verify that the ingress works

 
# should output "foo-app"
curl localhost/foo/hostname
# should output "bar-app"
curl localhost/bar/hostname