K3s Traefik TLS

The usual pattern for exposing Kubernetes services outside the cluster uses an Ingress controller. The Ingress controller exposes itself via LoadBalancer service and handles the traffic routing on the Application layer analyzing HTTP/S paths. This is the most often the case.

Most ingress controllers also support TCP/UDP proxying if you want to expose TLS/SSL services like Kafka, RabbitMQ, or similar services that need TLS termination at the service level itself instead of the TLS terminating on the Ingress controller.

This is convenient since using SNI in the TLS ingress controller can distinguish to which service on Kubernetes to route traffic.

That means that hosting multiple services on the Kubernetes under one domain name can be possible.

Another way to achieve this would be using multiple LoadBalancer services that consume public IP addresses from the pool of public IP addresses. This approach costs more since public IP addresses are scarce resource most often.

Expose service via TLS using Traefik

Traefik comes preinstalled on the K3s. We will approach the problem by exposing the service using TLS on the Traefik. TLS termination will occur on the Traefik and further connection in the cluster itself will continue using plain HTTP.

Automatic certificate generation for the specific domain can be solved using cert-manager and Letsencrypt.

Letsencrypt is a free TLS certificate provider. Cert-manager is a Kubernetes solution for managing certificates on the Kubernetes itself.

Letsencrypt can work in two ways: HTTP and DNS challenge. To verify ownership of the domain the server that wants to generate a certificate for itself should be able to answer the challenge from Letsencrypt - if the challenge is answered correctly certificate is presented to the server which later can be saved for further usage.

In this example, overview of the process will be demonstrated for the HTTP challenges.

Deploying and configuring cert-manager

To deploy cert-manager on the K3s - run the next commands:

helm repo add jetstack https://charts.jetstack.io --force-update
helm upgrade --install \
  cert-manager jetstack/cert-manager \
  --namespace cert-manager \
  --create-namespace \
  --version v1.16.1 \
  --set crds.enabled=true

This will deploy cert-manager in the cert-manager namespace.

The objects that are needed to be created are listed below:

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-staging
spec:
  acme:
    server: https://acme-staging-v02.api.letsencrypt.org/directory
    email: user@example.com
    privateKeySecretRef:
      name: letsencrypt-staging
    solvers:
      - http01:
          ingress:
            ingressClassName: traefik

The ClusterIssuer is the configuration of the Certificate issuer. The certificate request will use the issuer specified - hence issuer must exist before the certificate request is created.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
 name: ingress-example
 namespace: service
 annotations:
   cert-manager.io/cluster-issuer: "letsencrypt-staging"
spec:
 tls:
   - hosts:
       - domain.example.com
     secretName: any-name-you-want
 rules:
   - host: domain.example.com
     http:
       paths:
         - path: /
           pathType: Prefix
           backend:
             service:
               name: caddy
               port:
                 name: http

The cert-manager will automatically scan the new ingresses and generate CertificateRequest for specific ingress. If the challenge is completed successfully Certificate , it will be generated and information will be stored in secret with the name given in the TLS specification.

---
apiVersion: v1
kind: Service
metadata:
  name: caddy
spec:
  selector:
    app: caddy
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80
      name: http
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: caddy
  labels:
    app: caddy
spec:
  replicas: 1
  selector:
    matchLabels:
      app: caddy
  template:
    metadata:
      labels:
        app: caddy
    spec:
      containers:
      - name: caddy
        image: caddy:2.8.4-alpine
        ports:
        - containerPort: 80

Now deploying the caddy web server and HTTPS to the caddy pod is enabled already.

Traefik will handle TLS termination and communication to the pod will be in plain HTTP.