15: DNS in Kubernetes

15: DNS in Kubernetes

Objective

Understand how DNS works inside a Kubernetes cluster, learn the FQDN naming convention for Services, and practice DNS resolution and debugging using CoreDNS.

Theory

CoreDNS Architecture

Every Kubernetes cluster runs a DNS server — CoreDNS — as a set of Pods in the kube-system namespace. CoreDNS watches the Kubernetes API for new Services and Endpoints, and automatically creates DNS records for them.

When a Pod is created, Kubernetes configures its /etc/resolv.conf to point to the CoreDNS Service (typically kube-dns at 10.0.0.10).

FQDN Format

Every Service in Kubernetes gets a fully qualified domain name (FQDN) in the following format:

<service-name>.<namespace>.svc.cluster.local

For example, a Service called nginx-dns-XX in namespace student-XX has the FQDN:

nginx-dns-XX.student-XX.svc.cluster.local

/etc/resolv.conf and Search Domains

When Kubernetes creates a Pod, it injects DNS configuration into /etc/resolv.conf:

nameserver 10.0.0.10
search student-XX.svc.cluster.local svc.cluster.local cluster.local
options ndots:5

This means that from within namespace student-XX:

• nginx-dns-XX resolves (search domain appends student-XX.svc.cluster.local)
• nginx-dns-XX.student-XX resolves (search domain appends svc.cluster.local)
• nginx-dns-XX.student-XX.svc.cluster.local resolves (full FQDN, no search needed)

DNS Resolution Flow

sequenceDiagram
    participant Pod as Application Pod
    participant Resolv as /etc/resolv.conf
    participant CoreDNS as CoreDNS Pod
    participant API as Kubernetes API Server

    Pod->>Resolv: Lookup "nginx-dns-XX"
    Resolv->>CoreDNS: Query: nginx-dns-XX.student-XX.svc.cluster.local
    CoreDNS->>API: Watch Services & Endpoints
    API-->>CoreDNS: Service nginx-dns-XX → ClusterIP 10.0.100.50
    CoreDNS-->>Pod: A record: 10.0.100.50

DNS Record Types

Practical Tasks

Task 1: Create an nginx Deployment and Service

Create a file called nginx-dns.yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-dns-XX                  # Replace XX with your student number
  namespace: student-XX
  labels:
    app: nginx-dns-XX
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx-dns-XX
  template:
    metadata:
      labels:
        app: nginx-dns-XX
    spec:
      containers:
        - name: nginx
          image: nginx:1.27
          ports:
            - containerPort: 80
              name: http
---
apiVersion: v1
kind: Service
metadata:
  name: nginx-dns-XX                  # Replace XX with your student number
  namespace: student-XX
spec:
  selector:
    app: nginx-dns-XX
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80
      name: http

Deploy the resources:

kubectl apply -f nginx-dns.yaml

Verify:

kubectl get svc nginx-dns-XX -n student-XX
kubectl get endpoints nginx-dns-XX -n student-XX

Task 2: DNS Resolution Tests

Launch a temporary busybox Pod for DNS testing:

kubectl run dns-test-XX --rm -it --image=busybox -n student-XX -- sh

Inside the busybox shell, run the following tests:

Check /etc/resolv.conf:

cat /etc/resolv.conf

Note the nameserver, search domains, and ndots:5 setting.

Short name lookup (uses search domains):

nslookup nginx-dns-XX

Fully qualified domain name lookup:

nslookup nginx-dns-XX.student-XX.svc.cluster.local

Cross-namespace lookup (example with kubernetes Service in default namespace):

nslookup kubernetes.default.svc.cluster.local

Reverse DNS lookup (replace with actual ClusterIP):

nslookup 10.0.100.50

Connectivity test:

wget -qO- nginx-dns-XX

Type exit to leave the busybox Pod.

Task 3: SRV Records for Named Ports

SRV records are created for Services with named ports. They allow clients to discover both the port number and hostname of a Service.

Launch another test Pod:

kubectl run srv-test-XX --rm -it --image=busybox -n student-XX -- sh

Query the SRV record for the named port http:

nslookup -type=srv _http._tcp.nginx-dns-XX.student-XX.svc.cluster.local

The response will include:

• The port number (80)
• The target hostname (nginx-dns-XX.student-XX.svc.cluster.local)

Type exit to leave.

Clean Up

kubectl delete -f nginx-dns.yaml

Debugging DNS

Check CoreDNS Pods

Verify that CoreDNS Pods are running:

kubectl get pods -n kube-system -l k8s-app=kube-dns

All CoreDNS Pods should be in Running state with 1/1 READY.

Check CoreDNS Logs

If DNS resolution is failing, inspect CoreDNS logs:

kubectl logs -n kube-system -l k8s-app=kube-dns --tail=50

Check CoreDNS ConfigMap

View the CoreDNS configuration:

kubectl get configmap coredns -n kube-system -o yaml

Test DNS from a Dedicated Debug Pod

If busybox nslookup gives issues, use a more capable DNS debug image:

kubectl run dnsutils-XX --rm -it --image=registry.k8s.io/e2e-test-images/jessie-dnsutils:1.3 -n student-XX -- bash

Inside this Pod, you can use dig and host commands:

dig nginx-dns-XX.student-XX.svc.cluster.local
host nginx-dns-XX.student-XX.svc.cluster.local

Common Problems

Best Practices

1. Use short names within the same namespace — nginx-dns-XX is sufficient when the client Pod is in the same namespace as the Service.
2. Use <service>.<namespace> for cross-namespace calls — This is explicit and readable.
3. Add a trailing dot for external domains — Writing api.example.com. avoids unnecessary search domain lookups and speeds up external DNS resolution.
4. Monitor CoreDNS — Set up monitoring and alerts for CoreDNS Pods, as DNS failures affect the entire cluster.
5. Understand ndots:5 — Be aware that any name with fewer than 5 dots will trigger search domain lookups first, which can cause extra DNS queries.

Summary

In this exercise you learned:

• CoreDNS provides DNS resolution for all Services and Pods in the cluster
• Services get an FQDN in the format <service>.<namespace>.svc.cluster.local
• Pods are configured with /etc/resolv.conf pointing to CoreDNS, with search domains and ndots:5
• Short names resolve within the same namespace thanks to search domains
• SRV records provide port discovery for named ports
• How to debug DNS issues using nslookup, dig, and CoreDNS logs