Last updated 2026-05-28

Deployment — Kubernetes

Manifests for a production-shape Kubernetes deployment of notify: one Deployment, two Services (one internal-only, one exposed via Ingress), a Secret for the credentials, and an HPA tuned for the realtime workload.

When you'd use this

You already run Kubernetes and want notify to ride your existing operational story — Helm-ish chart, GitOps via Argo / Flux, mesh-of- your-choice, etc.

Namespace + Secret

00-namespace-secret.yaml
apiVersion: v1
kind: Namespace
metadata:
  name: notify
---
apiVersion: v1
kind: Secret
metadata:
  name: notify-secrets
  namespace: notify
type: Opaque
stringData:
  NOTIFY_AUTH_JWT_SECRET: "<32-byte hex>"
  NOTIFY_INTERNAL_TOKEN:  "<32-byte hex>"
  NOTIFY_POSTGRES_DSN:    "postgres://notify:password@db.notify:5432/notify?sslmode=require"
  TWILIO_AUTH_TOKEN:      "..."
  FCM_SERVICE_ACCOUNT_JSON: |
    { "type": "service_account", ... }

Project these from your secret manager via External Secrets Operator, Sealed Secrets, or whatever your shop uses. The point is that they live outside the manifest tree.

Deployment

10-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: notify
  namespace: notify
  labels: { app: notify }
spec:
  replicas: 2
  selector: { matchLabels: { app: notify } }
  template:
    metadata:
      labels: { app: notify }
      annotations:
        prometheus.io/scrape: "true"
        prometheus.io/port: "9090"
        prometheus.io/path: "/metrics"
    spec:
      terminationGracePeriodSeconds: 45     # NOTIFY_SHUTDOWN_TIMEOUT + headroom
      containers:
        - name: notify
          image: ghcr.io/elloloop/notify:0.1.0
          imagePullPolicy: IfNotPresent
          ports:
            - name: client
              containerPort: 8080
            - name: internal
              containerPort: 8081
            - name: metrics
              containerPort: 9090
          env:
            - { name: NOTIFY_STORE_DRIVER, value: postgres }
            - { name: NOTIFY_POSTGRES_AUTOMIGRATE, value: "true" }
            - { name: NOTIFY_LOG_LEVEL, value: info }
            - { name: NOTIFY_SHUTDOWN_TIMEOUT, value: "30s" }
            - { name: NOTIFY_AUTH_JWT_ISSUER, value: https://identity.example.com }
            - { name: NOTIFY_ALLOWED_ORIGINS, value: https://app.example.com }
            # Email provider
            - { name: NOTIFY_EMAIL_PROVIDER, value: emailservice }
            - { name: NOTIFY_EMAIL_SERVICE_ADDRESS, value: emailservice.email:50053 }
            - { name: NOTIFY_EMAIL_FROM, value: noreply@example.com }
            # SMS provider
            - { name: NOTIFY_SMS_PROVIDER, value: twilio }
            - { name: NOTIFY_SMS_ACCOUNT_SID, value: ACxxxx }
            - { name: NOTIFY_SMS_FROM, value: "+15555550000" }
          envFrom:
            - secretRef: { name: notify-secrets }
          resources:
            requests: { cpu: 100m, memory: 64Mi }
            limits:   { cpu: 1000m, memory: 256Mi }
          livenessProbe:
            httpGet: { path: /healthz, port: metrics }
            initialDelaySeconds: 5
            periodSeconds: 10
            failureThreshold: 3
          readinessProbe:
            httpGet: { path: /healthz, port: metrics }
            initialDelaySeconds: 2
            periodSeconds: 5
            failureThreshold: 2
          securityContext:
            runAsNonRoot: true
            runAsUser: 65532
            allowPrivilegeEscalation: false
            readOnlyRootFilesystem: true
            capabilities: { drop: ["ALL"] }

Services

20-services.yaml
# Public surface — what the SPA / mobile clients hit, fronted by Ingress.
apiVersion: v1
kind: Service
metadata:
  name: notify-client
  namespace: notify
spec:
  type: ClusterIP
  selector: { app: notify }
  ports:
    - name: client
      port: 8080
      targetPort: client


---
# Internal surface — only backend producers in the cluster reach this.
# Pin a NetworkPolicy that only allows ingress from the producing
# services' namespaces.
apiVersion: v1
kind: Service
metadata:
  name: notify-internal
  namespace: notify
spec:
  type: ClusterIP
  selector: { app: notify }
  ports:
    - name: internal
      port: 8081
      targetPort: internal


---
# Metrics — scraped by your in-cluster Prometheus.
apiVersion: v1
kind: Service
metadata:
  name: notify-metrics
  namespace: notify
  labels: { app: notify }
spec:
  type: ClusterIP
  selector: { app: notify }
  ports:
    - name: metrics
      port: 9090
      targetPort: metrics

Ingress (client surface)

30-ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: notify
  namespace: notify
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-prod
    nginx.ingress.kubernetes.io/proxy-read-timeout: "3600"        # long-lived SSE streams
    nginx.ingress.kubernetes.io/proxy-send-timeout: "3600"
    nginx.ingress.kubernetes.io/backend-protocol: "HTTP"
spec:
  ingressClassName: nginx
  tls:
    - hosts: [notify.example.com]
      secretName: notify-tls
  rules:
    - host: notify.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: notify-client
                port: { name: client }

The long proxy-{read,send}-timeout values matter for StreamEvents — without them nginx will close the long-lived SSE connection after 60s.

NetworkPolicy (internal port lockdown)

40-networkpolicy.yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: notify-internal-allowlist
  namespace: notify
spec:
  podSelector: { matchLabels: { app: notify } }
  policyTypes: [Ingress]
  ingress:
    # Producers in the api-gateway namespace can hit the internal port.
    - from:
        - namespaceSelector: { matchLabels: { name: api-gateway } }
      ports:
        - { port: 8081, protocol: TCP }
    # Anyone in the cluster can hit the client port (Ingress fronts it).
    - from: []
      ports:
        - { port: 8080, protocol: TCP }
    # Prometheus reaches metrics.
    - from:
        - namespaceSelector: { matchLabels: { name: monitoring } }
      ports:
        - { port: 9090, protocol: TCP }

Horizontal Pod Autoscaler

The realtime workload is connection-count bound rather than CPU bound. The HPA below uses CPU as a coarse signal; when the Prometheus metric surface is wired (follow-up wave) you can switch to a custom metric (e.g. notify_live_connections).

50-hpa.yaml
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: notify
  namespace: notify
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: notify
  minReplicas: 2
  maxReplicas: 10
  metrics:
    - type: Resource
      resource:
        name: cpu
        target: { type: Utilization, averageUtilization: 60 }
  behavior:
    scaleDown:
      stabilizationWindowSeconds: 300     # avoid flapping
    scaleUp:
      stabilizationWindowSeconds: 30

PodDisruptionBudget

60-pdb.yaml
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: notify
  namespace: notify
spec:
  minAvailable: 1
  selector: { matchLabels: { app: notify } }

Rolling deploy semantics

  • New pod boots, /healthz returns 503 until Server.Run binds listeners → marks ready.
  • Service starts routing traffic to the new pod.
  • Old pod receives SIGTERM, /healthz immediately flips to 503 → Service stops sending new traffic.
  • Shutdown drains in-flight requests within NOTIFY_SHUTDOWN_TIMEOUT.
  • Open SSE streams: each handler exits on context cancel, client reconnects to whichever pod the LB picks next.

Related