Overview

This PR introduces a complete credential management system for the Curvine S3 Gateway, enabling secure storage and management of AWS Access Key/Secret Key pairs. The system supports both single-instance and multi-instance deployments with automatic synchronization capabilities.

Key Features

Dual Storage Modes

• Local Storage: File-based storage for single-instance deployments (default: ~/.curvine/credentials.jsonl)
• Distributed Storage: Curvine-based storage for multi-instance deployments with automatic synchronization (default: /system/auth/credentials.jsonl)

Security & Performance

• Memory-protected SecretKey type with automatic zeroization
• Background cache refresh for distributed mode
• JSONL format for human-readable and append-friendly storage

Management Operations

• Add credentials: Manual or auto-generated Access Key/Secret Key pairs
• List credentials: View all stored credentials with optional secret display
• Cache statistics: Monitor cache performance and storage status
• Hot reload: Automatic detection of credential changes in distributed mode

Technical Implementation

Architecture

• Static Dispatch: AccessKeyStoreEnum provides zero-cost abstraction over storage implementations
• Unified Cache: CacheState structure shared between local and distributed stores
• Async Traits: AccesskeyStore and CredentialStore traits for consistent API
• Background Tasks: CacheRefreshTask for automatic credential synchronization

Key Components

• LocalAccessKeyStore: Single-instance local file system storage
• CurvineAccessKeyStore: Multi-instance distributed storage with background refresh
• AccessKeyStoreEnum: Static dispatch wrapper for unified API
• CredentialEntry: Secure credential data structure with validation

Usage

CLI Integration

The credential management is fully integrated into the gateway shell script:

# Generate new credentials
./curvine-s3-gateway.sh credential generate --description "Production API"

# Add existing credentials
./curvine-s3-gateway.sh credential add --access-key AKIAEXAMPLE --secret-key secretkey123 --description "Dev Environment"

# List all credentials
./curvine-s3-gateway.sh credential list

# Show cache statistics
./curvine-s3-gateway.sh credential stats

Configuration

Enable distributed authentication in cluster configuration:

[s3_gateway]
enable_distributed_auth = true
credentials_path = "/system/auth/credentials.jsonl"  # optional
cache_refresh_interval_secs = 300  # optional

For local mode, simply set enable_distributed_auth = false or omit the section.