Search before asking

• I searched in the issues and found nothing similar.

Motivation

Use Case:

Currently, Fluss only supports a single remote data directory via the remote.data.dir configuration option. In production environments, users often want to distribute KV snapshot data files and tiered log storage across multiple remote storage paths (e.g., different OSS buckets, different S3 prefixes, or separate remote filesystems) to achieve better I/O throughput, cost optimization, or fault isolation.

Problem Statement:

With a single remote.data.dir, all remote data is written to one path, which:

• Creates a bottleneck when a single bucket/filesystem cannot handle the throughput
• Prevents cost-tiering strategies (e.g., hot/cold buckets)
• Limits flexibility for multi-tenant or multi-region deployments

Benefits:

• Enables users to spread remote data across multiple storage backends for higher aggregate I/O throughput
• Supports weighted distribution to utilize buckets with different capacities proportionally
• Keeps backward compatibility — the existing remote.data.dir single-path config continues to work

Solution

Proposed Approach:

Introduce three new configuration options alongside the existing remote.data.dir:

Example configuration:

# Round-robin across two OSS buckets
remote.data.dirs: oss://bucket1/fluss-remote-data, oss://bucket2/fluss-remote-data
remote.data.dirs.strategy: ROUND_ROBIN

# Weighted round-robin (2:1 ratio)
remote.data.dirs: oss://bucket1/fluss-remote-data, oss://bucket2/fluss-remote-data
remote.data.dirs.strategy: WEIGHTED_ROUND_ROBIN
remote.data.dirs.weights: 2, 1

Anything else?

This change is backward-compatible. If remote.data.dirs is not set, the system continues to use the single remote.data.dir path as before. The routing/selection logic for writing to one of multiple dirs will be implemented as a follow-up task.

Willingness to contribute

• I'm willing to submit a PR!