28: Security Operations

28: Security Operations

Objective

Understand AKS security operations including Defender for Containers, image scanning, cluster upgrade channels, maintenance windows, and node OS security updates. Learn how to check cluster versions and available upgrades.


Theory

Defender for Containers

Microsoft Defender for Containers provides comprehensive security for AKS clusters:

Feature Description
Threat detection Identifies suspicious activities at the cluster level (e.g., crypto mining, privilege escalation)
Vulnerability scanning Scans container images for known CVEs in Azure Container Registry and at runtime
Runtime protection Monitors running containers for anomalous behavior
Security recommendations Provides actionable guidance in Microsoft Defender for Cloud

Defender for Containers is enabled at the subscription level and automatically protects all AKS clusters.

Image Scanning

Azure Container Registry (ACR) integrates with Defender to scan images:

  • Push-time scanning: Images are scanned when pushed to ACR
  • Continuous scanning: Running images are periodically re-scanned for newly discovered vulnerabilities
  • Results: Available in Microsoft Defender for Cloud under Recommendations > Containers

AKS Auto-Upgrade Channels

AKS supports automatic cluster upgrades to keep Kubernetes versions current:

Channel Description
none No automatic upgrades (manual only)
patch Automatically upgrades to the latest patch version (e.g., 1.28.3 to 1.28.5)
stable Upgrades to the latest supported patch of the N-1 minor version
rapid Upgrades to the latest supported patch of the latest minor version
node-image (Legacy) Upgrades only the node OS image (not the Kubernetes version). Use the separate node image autoupgrade NodeImage channel instead.

Configure with (cluster administrator operation):

# Instructor/admin command — do not run
az aks update --resource-group <RG> --name <CLUSTER> --auto-upgrade-channel patch

Maintenance Windows

Planned maintenance windows control when AKS can perform upgrades and maintenance operations:

  • Define specific days and time ranges for maintenance
  • Prevents disruption during business-critical hours
  • Applies to both control plane and node pool upgrades
# Instructor/admin command — do not run
az aks maintenancewindow add \
  --resource-group <RG> \
  --cluster-name <CLUSTER> \
  --name default \
  --day-of-week Saturday \
  --start-time 02:00 \
  --duration 4

Node OS Security Updates

Node images are regularly updated with OS security patches:

  • Node image upgrades: Azure provides updated node images with the latest OS patches
  • Automatic with node-image channel: Nodes are automatically re-imaged when new images are available
  • Manual: az aks nodepool upgrade --node-image-only (cluster administrator operation)

Practical Tasks

Task 1: Check Cluster Version from kubectl (Participant Task)

You can check the Kubernetes version and node details using kubectl without Azure CLI access:

# Check the cluster version
kubectl version

# View node details including OS image and Kubernetes version
kubectl get nodes -o wide

Look at the VERSION column to see the Kubernetes version running on each node, and the OS-IMAGE column for the node operating system.

# Get detailed node information
kubectl describe node <node-name>

In the describe output, look for the System Info section which shows the Kubernetes version, container runtime version, and OS details.


Task 2: Check Cluster Version and Upgrades (Instructor Demo)

Note: The following az aks commands require Azure subscription access and are performed by the instructor. They are included here so you understand how cluster version management works in production.

# Check the current Kubernetes version
az aks show -g <RG> -n <CLUSTER> --query "kubernetesVersion" -o tsv

# Check the current auto-upgrade channel
az aks show -g <RG> -n <CLUSTER> --query "autoUpgradeProfile.upgradeChannel" -o tsv

# List available Kubernetes version upgrades
az aks get-upgrades -g <RG> -n <CLUSTER> -o table

The instructor can also check node image versions:

# Check the node image version for a specific node pool
az aks nodepool show --cluster-name <CLUSTER> -g <RG> -n nodepool1 --query nodeImageVersion -o tsv

# List available node image upgrades
az aks nodepool get-upgrades --cluster-name <CLUSTER> -g <RG> -n nodepool1 -o table

Task 3: View Security Recommendations (Instructor Demo)

Note: Azure Portal access is an instructor-led demo. Participants observe while the instructor navigates the portal.

This task is demonstrated by the instructor in the Azure Portal:

  1. Navigate to Azure Portal > Microsoft Defender for Cloud
  2. Go to Recommendations
  3. Filter by Resource type: Containers or Kubernetes
  4. Review recommendations such as:
    • “Container images should be deployed from trusted registries only”
    • “Kubernetes clusters should not use the default namespace”
    • “Kubernetes cluster pods should use specified labels”
    • “Role-Based Access Control should be used on Kubernetes Services”

Each recommendation includes:

  • Severity level (High, Medium, Low)
  • Affected resources
  • Remediation steps

Common Problems

Problem Cause Solution
az aks get-upgrades shows no upgrades Already on the latest version Expected if running the latest supported version
Upgrade fails with PodDisruptionBudget PDB prevents draining nodes Review PDBs, temporarily relax if needed during maintenance
Node image upgrade causes downtime All nodes upgraded simultaneously Use surge upgrades (--max-surge) to maintain capacity during upgrades
Defender alerts for training workloads Defender detects test/demo activities as suspicious Review and dismiss false positives in Azure Portal

Best Practices

  • Enable auto-upgrade with patch channel — ensures you receive security patches automatically without breaking minor version changes
  • Configure maintenance windows — schedule upgrades for low-traffic periods (e.g., weekends, overnight)
  • Enable Defender for Containers — provides threat detection, vulnerability scanning, and security recommendations
  • Scan images before deployment — integrate image scanning into your CI/CD pipeline using Defender or tools like Trivy
  • Use node surge upgrades — configure --max-surge to maintain capacity during node image upgrades (e.g., --max-surge 1 adds one extra node during upgrade)
  • Monitor upgrade progress — use az aks show and kubectl get nodes to track upgrade status
  • Test upgrades in staging first — always validate Kubernetes version upgrades in a non-production environment
  • Keep node images current — even without Kubernetes version upgrades, regularly update node images for OS security patches

Summary

In this exercise you learned:

  • Microsoft Defender for Containers provides threat detection, vulnerability scanning, and runtime protection for AKS
  • AKS auto-upgrade channels (none, patch, stable, rapid, node-image) control how automatically the cluster is upgraded
  • Maintenance windows define when Azure can perform upgrades to minimize disruption
  • Node image upgrades apply OS-level security patches independently of Kubernetes version upgrades
  • Azure CLI commands (az aks show, az aks get-upgrades, az aks nodepool show) help monitor cluster versions and available updates
  • In production, enable the patch auto-upgrade channel, configure maintenance windows, and enable Defender for Containers

results matching ""

    No results matching ""