Skip to content

[FEATURE][HELM]: Add optional persistence support for PostgreSQL and Redis #1308

@SharkyND

Description

@SharkyND

🧭 Epic

Title: Add Optional Persistence Support for PostgreSQL and Redis

Goal: Provide production-ready persistence configuration for PostgreSQL and Redis in the Helm chart with proper templating, backup annotations, and conditional logic.

Why now:

  • Production deployments require persistent storage to survive pod restarts
  • Need Velero/backup tool integration for disaster recovery
  • Current implementation needs verification and documentation

Current State:
This feature appears to be implemented!

The Helm chart already includes:

  • postgres-pvc.yaml with proper templating
  • redis-pvc.yaml with proper templating
  • values.yaml persistence configuration for both PostgreSQL and Redis
  • Velero annotation support
  • Configurable storage class, size, and reclaim policy

📖 User Stories

US-1: DevOps Engineer - PostgreSQL Persistence

As a: DevOps engineer

I want: PostgreSQL data to persist across pod restarts

So that: Production data is not lost during deployments or node failures

Acceptance Criteria:

Scenario: PostgreSQL data persists across restarts
  Given postgres.persistence.enabled = true
  When the PostgreSQL pod is restarted
  Then all database data should be preserved
  And no manual data recovery should be needed

Scenario: Configure storage class
  Given postgres.persistence.storageClassName = "gp3"
  When the PVC is created
  Then it should use the gp3 storage class

Scenario: Disable persistence for testing
  Given postgres.persistence.enabled = false
  When the PostgreSQL pod is deployed
  Then it should use emptyDir volume
  And data should be lost on pod restart

Technical Requirements:

  • Conditional PVC creation based on persistence.enabled
  • EmptyDir fallback when persistence disabled
  • Proper Helm templating for resource names
US-2: DevOps Engineer - Redis Persistence

As a: DevOps engineer

I want: Optional Redis persistence for stateful caching scenarios

So that: I can choose between ephemeral caching and persistent Redis based on use case

Acceptance Criteria:

Scenario: Redis ephemeral mode (default)
  Given redis.persistence.enabled = false (default)
  When Redis pod restarts
  Then cache should be empty (expected behavior for caching)

Scenario: Redis persistent mode
  Given redis.persistence.enabled = true
  When Redis pod restarts
  Then cached data should be preserved
  And session data should survive restarts

Technical Requirements:

  • Redis persistence disabled by default (caching use case)
  • Conditional PVC creation
  • Redis AOF/RDB configuration when persistence enabled
US-3: SRE - Backup Integration

As an: SRE

I want: PVCs to have backup tool annotations

So that: Velero can automatically backup the persistent volumes

Acceptance Criteria:

Scenario: Velero backup annotation
  Given postgres.persistence.annotations contains:
    backup.velero.io/backup-volumes: "postgres-data"
  When Velero runs a backup
  Then the PostgreSQL volume should be included
  And the backup should complete successfully

Scenario: Custom annotations
  Given I add custom annotations in values.yaml
  When the PVC is created
  Then all annotations should be present on the PVC

Technical Requirements:

  • Annotation pass-through in PVC templates
  • Documentation of common backup tool annotations

🏗 Architecture

Persistence Configuration Flow

flowchart TD
    subgraph values.yaml
        A[postgres.persistence.enabled]
        B[redis.persistence.enabled]
    end
    
    subgraph Templates
        A -->|true| C[postgres-pvc.yaml]
        A -->|false| D[emptyDir volume]
        B -->|true| E[redis-pvc.yaml]
        B -->|false| F[emptyDir volume]
    end
    
    subgraph Storage
        C --> G[PersistentVolume]
        E --> H[PersistentVolume]
    end
Loading

Current Implementation

# PostgreSQL Persistence (charts/mcp-stack/values.yaml:828)
postgres:
  persistence:
    enabled: true
    storageClassName: ""
    accessModes: [ReadWriteOnce]
    size: 5Gi
    reclaimPolicy: Retain
    annotations: {}

# Redis Persistence (charts/mcp-stack/values.yaml:1018)
redis:
  persistence:
    enabled: false  # Default: caching mode
    storageClassName: ""
    accessModes: [ReadWriteOnce]
    size: 1Gi
    reclaimPolicy: Retain
    annotations: {}

📋 Implementation Tasks

Verification Tasks (Feature Exists)

  • Verify postgres-pvc.yaml template works correctly
  • Verify redis-pvc.yaml template works correctly
  • Verify emptyDir fallback when persistence disabled
  • Test with different storage classes
  • Test Velero backup annotations

Documentation Tasks

  • Document PostgreSQL persistence configuration
  • Document Redis persistence options
  • Document storage class examples for AWS, GCP, Azure
  • Document backup integration with Velero
  • Add troubleshooting section

Potential Enhancements

  • Add Redis AOF/RDB configuration options
  • Add volume snapshot support
  • Add PV reclaim policy configuration in deployment templates
  • Add data migration documentation

⚙️ Configuration Examples

Production Configuration (AWS EKS)

postgres:
  persistence:
    enabled: true
    storageClassName: "gp3"
    size: 50Gi
    reclaimPolicy: Retain
    annotations:
      backup.velero.io/backup-volumes: "postgres-data"

redis:
  persistence:
    enabled: false  # Ephemeral caching

Stateful Redis Configuration

redis:
  persistence:
    enabled: true
    storageClassName: "gp3"
    size: 10Gi
    reclaimPolicy: Retain
    annotations:
      backup.velero.io/backup-volumes: "redis-data"

Development/Testing Configuration

postgres:
  persistence:
    enabled: false  # Use emptyDir

redis:
  persistence:
    enabled: false

✅ Success Criteria

  • PostgreSQL persistence works with enabled: true
  • PostgreSQL uses emptyDir with enabled: false
  • Redis persistence works with enabled: true
  • Redis uses emptyDir with enabled: false
  • Storage class configuration works
  • Velero backup annotations work
  • Documentation complete with examples

🏁 Definition of Done

  • Verify existing implementation works correctly
  • Test PostgreSQL persistence on/off scenarios
  • Test Redis persistence on/off scenarios
  • Test with real storage classes (gp3, standard, etc.)
  • Test Velero backup integration
  • Documentation updated in charts/README.md
  • Add examples for AWS, GCP, Azure storage classes
  • PR reviewed and approved

📝 Implementation Status

Feature Status Location
PostgreSQL PVC template ✅ Implemented charts/mcp-stack/templates/postgres-pvc.yaml
Redis PVC template ✅ Implemented charts/mcp-stack/templates/redis-pvc.yaml
PostgreSQL persistence config ✅ Implemented values.yaml:828
Redis persistence config ✅ Implemented values.yaml:1018
Velero annotation support ✅ Implemented Both PVC templates
EmptyDir fallback ⚠️ Needs verification Deployment templates
Documentation ❌ Needs update charts/README.md

🔗 Related Issues

  • Helm chart enhancements
  • Backup and disaster recovery
  • Production deployment guide

Metadata

Metadata

Labels

COULDP3: Nice-to-have features with minimal impact if left out; included if time permitsenhancementNew feature or requesthelmHelm chart

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions