Full-Stack Spring Boot on Kubernetes with Volumes

A three-tier stack on Kubernetes is where two things stop being optional: durable storage for the database, and a decision about how the browser reaches the API. Both have a default that works in a demo and fails in a cluster. An emptyDir that survives no restart, and a hard-coded API host that cannot be changed after the image is built.

Written against Kubernetes 1.29, Spring Boot 3.2 and MySQL 8.

Secrets are encoded, not encrypted

kubectl create secret generic mysql-credentials \
  --from-literal=root-password="$ROOT_PASSWORD" \
  --from-literal=username=app \
  --from-literal=password="$APP_PASSWORD"

apiVersion: v1
kind: Secret
metadata:
  name: mysql-credentials
type: Opaque
stringData:            # stringData takes plain text; data takes base64
  username: app
  password: change-me

stringData accepts plain values and Kubernetes encodes them; data requires base64 that you produce. Either way the stored value is base64, not encryption: kubectl get secret -o yaml plus base64 -d reveals it, and by default it sits unencrypted in etcd.

Three consequences worth acting on. Never commit a Secret manifest with real values: use stringData with a placeholder and create the real one out of band, or Sealed Secrets or an external secrets operator. Turn on encryption at rest in the API server if the cluster does not have it. And restrict access with RBAC, because any pod whose service account can read secrets in the namespace can read all of them.

MySQL needs a PersistentVolumeClaim

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: mysql-data
spec:
  accessModes: [ReadWriteOnce]
  storageClassName: standard
  resources:
    requests:
      storage: 20Gi

A PVC is a request; the storageClassName names a provisioner that fulfils it. Omitting the field uses the cluster’s default class, which may not exist, the claim then stays Pending for ever, and the pod stays Pending with it, with no error explaining why.

kubectl get storageclass
kubectl describe pvc mysql-data

ReadWriteOnce means one node may mount it, not one pod. That is the constraint behind the rollout deadlock: a Deployment with the default RollingUpdate strategy starts the new pod before terminating the old one, and if it is scheduled to a different node the volume cannot attach. The new pod waits for the volume, the old pod waits to be replaced, and the rollout hangs.

For a database the answer is a StatefulSet, which replaces pods one at a time and gives each a stable identity and its own volume:

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: mysql
spec:
  serviceName: mysql
  replicas: 1
  selector:
    matchLabels: { app: mysql }
  template:
    metadata:
      labels: { app: mysql }
    spec:
      containers:
        - name: mysql
          image: mysql:8.0
          args: ["--default-authentication-plugin=mysql_native_password"]
          env:
            - name: MYSQL_ROOT_PASSWORD
              valueFrom:
                secretKeyRef: { name: mysql-credentials, key: root-password }
            - name: MYSQL_DATABASE
              value: appdb
            - name: MYSQL_USER
              valueFrom:
                secretKeyRef: { name: mysql-credentials, key: username }
            - name: MYSQL_PASSWORD
              valueFrom:
                secretKeyRef: { name: mysql-credentials, key: password }
          ports:
            - containerPort: 3306
          volumeMounts:
            - name: data
              mountPath: /var/lib/mysql
          readinessProbe:
            exec:
              command: ["mysqladmin", "ping", "-h", "127.0.0.1"]
            initialDelaySeconds: 20
            periodSeconds: 5
  volumeClaimTemplates:
    - metadata: { name: data }
      spec:
        accessModes: [ReadWriteOnce]
        resources: { requests: { storage: 20Gi } }

volumeClaimTemplates creates one PVC per replica and, importantly, does not delete them when the StatefulSet is deleted. That is deliberate and occasionally surprising: removing the workload leaves the data, and recreating it reattaches.

The headless service that gives the pods stable DNS names:

apiVersion: v1
kind: Service
metadata:
  name: mysql
spec:
  clusterIP: None
  selector: { app: mysql }
  ports: [{ port: 3306 }]

clusterIP: None is what makes mysql-0.mysql resolve to the pod rather than load-balancing across replicas, which for a database with one primary is the addressing you want.

Running MySQL in the cluster at all is a decision worth making explicitly. A managed database removes backups, failover and version upgrades from your responsibility, and those are the parts that matter at three in the morning.

The Spring Boot deployment

apiVersion: apps/v1
kind: Deployment
metadata:
  name: api
spec:
  replicas: 2
  selector:
    matchLabels: { app: api }
  template:
    metadata:
      labels: { app: api }
    spec:
      containers:
        - name: api
          image: registry.example.com/api:1.4.0
          env:
            - name: SPRING_DATASOURCE_URL
              value: jdbc:mysql://mysql:3306/appdb
            - name: SPRING_DATASOURCE_USERNAME
              valueFrom:
                secretKeyRef: { name: mysql-credentials, key: username }
            - name: SPRING_DATASOURCE_PASSWORD
              valueFrom:
                secretKeyRef: { name: mysql-credentials, key: password }
          ports: [{ containerPort: 8080 }]
          resources:
            requests: { memory: 512Mi, cpu: 250m }
            limits:   { memory: 1Gi }
          startupProbe:
            httpGet: { path: /actuator/health, port: 8080 }
            failureThreshold: 30
            periodSeconds: 5
          readinessProbe:
            httpGet: { path: /actuator/health/readiness, port: 8080 }
          livenessProbe:
            httpGet: { path: /actuator/health/liveness, port: 8080 }

The environment variable names are Spring’s relaxed binding, SPRING_DATASOURCE_URL maps to spring.datasource.url with no configuration file involved.

The three probes are three different questions. Startup gives a slow JVM time to boot without the liveness probe killing it; without it, a 40-second startup under a 30-second liveness threshold produces a pod that restarts for ever. Readiness removes a pod from the Service when it cannot serve, a lost database connection should fail readiness and not liveness, or every replica restarts together when the database blips. Liveness should only fail when a restart would genuinely help.

A memory limit without a request means the request equals the limit. More importantly, the JVM must be told about it, -XX:MaxRAMPercentage=75, because a container limit is not something the default heap sizing respects, and exceeding it is an OOMKill with exit code 137 rather than an OutOfMemoryError.

Database migrations are the other thing to decide here. Two replicas both running Flyway at startup race for the lock, and the loser can exceed its startup probe. An initContainer or a Job that migrates before the rollout is the cleaner shape.

The React front end

A React build is static files, and the API host has to be decided before the build:

FROM node:20-alpine AS build
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build

FROM nginx:1.25-alpine
COPY --from=build /app/dist /usr/share/nginx/html
COPY nginx.conf /etc/nginx/conf.d/default.conf

That is the constraint people meet late: process.env.REACT_APP_API_URL is substituted at build time, so one image cannot be promoted from staging to production with a different API host.

The way out is not to have an API host at all. Serve the front end and proxy /api from the same nginx, and the browser only ever talks to one origin, which also removes CORS entirely:

server {
    listen 80;
    root /usr/share/nginx/html;

    location /api/ {
        proxy_pass http://api:8080/api/;
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }

    location / {
        try_files $uri $uri/ /index.html;    # client-side routing
    }
}

try_files ... /index.html is what makes a deep link work. Without it, refreshing on /articles/5 returns 404 because no such file exists, the route is a client-side concept.

The trailing slash on both sides of proxy_pass matters: proxy_pass http://api:8080/api/ with the location /api/ passes the path through unchanged, while omitting either produces a doubled or a stripped prefix.

Ingress

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: web
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt
spec:
  ingressClassName: nginx
  tls:
    - hosts: [app.example.com]
      secretName: web-tls
  rules:
    - host: app.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service: { name: web, port: { number: 80 } }

One host, one backend, and the API reached through the front end’s own proxy. Routing /api at the Ingress instead is also valid and puts the routing decision in the cluster rather than in the image — either is defensible, and doing both is how a request ends up proxied twice.

Related: Kubernetes for a single service, a stateful dependency and the same stack in Compose.

Frequently asked questions

Are Kubernetes Secrets encrypted?

No. They are base64-encoded and, by default, stored unencrypted in etcd. Enable encryption at rest, restrict access with RBAC, and keep real values out of version control.

What is the difference between data and stringData in a Secret?

data takes base64 you produce; stringData takes plain text and Kubernetes encodes it. Neither is more secure.

Why is my PersistentVolumeClaim stuck in Pending?

No storage class matched. Either the named class does not exist or the cluster has no default. kubectl describe pvc names the reason.

Why does my database rollout hang?

A Deployment with a ReadWriteOnce volume starts the new pod before removing the old one, and the volume cannot attach to a second node. Use a StatefulSet, or the Recreate strategy.

Does deleting a StatefulSet delete its volumes?

No. PVCs created from volumeClaimTemplates are kept deliberately, so recreating the StatefulSet reattaches the data.

Why does my pod restart during startup?

The liveness probe fired before the JVM finished booting. Add a startupProbe with a generous failureThreshold; liveness does not run until it passes.

Should a lost database connection fail liveness or readiness?

Readiness. Failing liveness restarts every replica at once for a problem a restart cannot fix.

Why is my Java container OOMKilled?

The JVM sized its heap for the node rather than the container. Set -XX:MaxRAMPercentage. Exit code 137 is the kernel killing it, not an OutOfMemoryError.

Why can’t I change the API URL in my React image?

It was substituted at build time. Proxy /api from the nginx serving the front end so there is no API host to configure.

Why does refreshing a React route return 404?

nginx looked for a file that does not exist. Add try_files $uri $uri/ /index.html so the client-side router receives the request.