[seaweedfs] Traffic locality #1748

nbykov0 · 2025-12-23T21:39:41Z

What this PR does

Addes traffic locality capabilities to Seaweedfs.

Release note

Added traffic locality capabilities to Seaweedfs

Summary by CodeRabbit

Release Notes

New Features
- SeaweedFS upgraded to version 4.05
- Admin component with web-based UI and authentication support
- Worker component for distributed operations
- Enhanced S3 monitoring with new Grafana dashboard panels for buckets, API calls, costs, and performance metrics
- TLS certificate support for admin and worker components
Improvements
- S3 service traffic distribution optimization
- Ingress multi-host support for S3 endpoints
- Pod scheduling optimization for improved availability
- Security enhancements with configurable TLS endpoints

_{✏️ Tip: You can customize this high-level summary in your review settings.}

coderabbitai · 2025-12-23T21:39:46Z

📝 Walkthrough

Walkthrough

The pull request introduces admin and worker component deployments to the SeaweedFS Helm chart, bumps the version from 4.02 to 4.05, refactors the Makefile to use patch-based customization instead of Docker builds, enhances S3 observability metrics in Grafana, updates pod scheduling policies across components, and updates ingress configurations for multi-host and traffic distribution support.

Changes

Cohort / File(s)	Summary
Version and Chart Metadata `packages/system/seaweedfs/charts/seaweedfs/Chart.yaml`	Updated appVersion to 4.05 and version to 4.0.405
Admin Component (New) `packages/system/seaweedfs/charts/seaweedfs/templates/admin/admin-statefulset.yaml`, `admin-service.yaml`, `admin-ingress.yaml`, `admin-secret.yaml`, `admin-servicemonitor.yaml`	Introduced comprehensive admin component with StatefulSet, Service (headless with metrics port), Ingress (version-aware), Secret (pre-install/upgrade hooks), and ServiceMonitor resources; includes validation for master server configuration, secret handling, and storage options
Worker Component (New) `packages/system/seaweedfs/charts/seaweedfs/templates/worker/worker-deployment.yaml`, `worker-service.yaml`, `worker-servicemonitor.yaml`	Introduced worker component as Deployment with Service (headless, metrics-enabled) and ServiceMonitor; includes guard requiring adminServer configuration, support for capabilities, maxConcurrent, workingDir, and flexible volume/security setups
Certificate Management (New) `packages/system/seaweedfs/charts/seaweedfs/templates/cert/admin-cert.yaml`, `worker-cert.yaml`	Added cert-manager Certificate resources for admin and worker components with version-aware apiVersion selection and GRPC endpoint TLS configuration
Security Configuration `packages/system/seaweedfs/charts/seaweedfs/templates/shared/security-configmap.yaml`	Extended security TOML with TLS paths for grpc.admin and grpc.worker endpoints
Helper Templates and Refactoring `packages/system/seaweedfs/charts/seaweedfs/templates/shared/_helpers.tpl`, `filer/filer-statefulset.yaml`, `master/master-statefulset.yaml`, `volume/volume-statefulset.yaml`	Added helpers (admin.image, worker.image, admin.pvc_exists, seaweedfs.masterServers, seaweedfs.masterServerArg); refactored master server argument generation to use template helpers in filer, master, and volume StatefulSets
S3 Enhancements `packages/system/seaweedfs/charts/seaweedfs/templates/s3/s3-ingress.yaml`, `s3-service.yaml`, `patches/s3-traffic-distribution.patch`	Updated S3 ingress to support multi-host configuration with backward-compatible host handling; added trafficDistribution: PreferClose to S3 Service; introduced patch for traffic distribution
Build Process Changes `packages/system/seaweedfs/Makefile`, `images/seaweedfs/Dockerfile`	Removed image target from Makefile and Docker build logic; simplified tarball extraction; added patch application for s3-traffic-distribution; removed dynamic base image ARG VERSION from Dockerfile
Values Configuration `packages/system/seaweedfs/charts/seaweedfs/values.yaml`, `values.yaml`	Added comprehensive admin/worker configuration blocks (enabled, ports, replicas, storage, resources, affinity); expanded allInOne deployment config; added volume.id field; updated S3 config in root values to enable S3 with replicas, affinity, and service-upstream annotation
Observability `packages/system/seaweedfs/charts/seaweedfs/dashboards/seaweedfs-grafana-dashboard.json`, `README.md`	Added S3-related Grafana panels (API calls per bucket, bucket size, object count, traffic, QPS, cost, request duration quantiles, Filer metrics); documented Admin and Worker component configuration and deployment strategies
Ingress Controller Configuration `packages/system/ingress-nginx/values.yaml`	Added pod anti-affinity policy to ingress-nginx controller with preferred hostname and zone-based scheduling constraints (weights 10 and 100)

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Poem

🐰 New admin and workers now take their flight,
With patches applied and templates just right,
S3 spreads traffic PreferClose and wide,
While affinity rules keep pods side by side,
From 4.02 to 4.05, we've come such a way! 🌟

🚥 Pre-merge checks | ✅ 3

✅ Passed checks (3 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The PR title '[seaweedfs] Traffic locality' clearly and specifically describes the main change—adding traffic locality capabilities to SeaweedFS, which is well-reflected in the commit messages and file modifications across affinity rules, S3 service traffic distribution, and topology-aware scheduling.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

gemini-code-assist · 2025-12-23T21:39:55Z

Summary of Changes

Hello @nbykov0, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request enhances the traffic locality capabilities within the system by implementing strategic Kubernetes scheduling and service routing configurations. It ensures that ingress-nginx controllers are spread across the cluster for better resilience and that SeaweedFS S3 traffic is directed to the nearest available endpoints, thereby optimizing network performance and reducing latency. Additionally, it refines the SeaweedFS ingress setup with an updated host and a specific NGINX annotation.

Highlights

Ingress-Nginx Pod Anti-Affinity: Configured ingress-nginx controller pods with podAntiAffinity rules to ensure they are distributed across different hosts and Kubernetes zones. This enhances high availability and improves traffic locality by preventing multiple controller instances from running on the same physical or logical node.
SeaweedFS S3 Traffic Distribution: Modified the SeaweedFS S3 service to include trafficDistribution: PreferClose. This setting instructs Kubernetes to route traffic to S3 endpoints that are topologically closer to the client, reducing latency and improving performance.
SeaweedFS Ingress Configuration Update: Updated the ingress host for SeaweedFS from seaweedfs2.demo.cozystack.io to seaweedfs.demo.cozystack.io and added the nginx.ingress.kubernetes.io/service-upstream: true annotation. This annotation ensures that NGINX Ingress Controller uses the service name directly for upstream resolution, which can be beneficial for certain traffic routing scenarios.
Makefile Patch Integration: Integrated a new patch file, s3-traffic-distribution.patch, into the SeaweedFS Makefile. This patch is responsible for applying the trafficDistribution: PreferClose setting to the S3 service during the build process.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist

Code Review

This pull request aims to add traffic locality capabilities to SeaweedFS. It does so by configuring pod anti-affinity for ingress-nginx to spread pods across zones, and by setting trafficDistribution: PreferClose on the SeaweedFS S3 service. While these are good steps, there are a couple of areas for improvement. I've pointed out a way to make the ingress-nginx affinity rules more maintainable. More importantly, I've identified a configuration in the SeaweedFS ingress annotations that conflicts with the goal of traffic locality for ingress traffic and should be addressed.

packages/system/seaweedfs/values.yaml

packages/system/ingress-nginx/values.yaml

Signed-off-by: nbykov0 <166552198+nbykov0@users.noreply.github.com>

coderabbitai

Actionable comments posted: 4

🤖 Fix all issues with AI agents

In @packages/system/seaweedfs/charts/seaweedfs/Chart.yaml:
- Around line 4-6: The Chart.yaml appVersion is invalid ("4.05"); update the
appVersion field in Chart.yaml to a valid SeaweedFS release (e.g., "4.00") and
ensure the chart's version/metadata remain consistent; do not edit the vendored
chart in-place—implement this change via the repository's chart patch mechanism
(create an appropriate patch under patches/ or use the established chart
vendoring workflow) so the vendored upstream remains untouched.

In @packages/system/seaweedfs/charts/seaweedfs/templates/s3/s3-service.yaml:
- Line 17: Revert the direct change that hardcodes trafficDistribution:
PreferClose in the s3 service template; instead remove the hardcoded
trafficDistribution line and implement it via chart values and proper vendoring
(overlay/kustomize or chart dependency) so users can opt-in or set it
per-cluster. Introduce a chart value (e.g., values.s3.trafficDistributionEnabled
and values.s3.trafficDistributionValue) and conditionally render the
trafficDistribution field in the s3 service template only when
trafficDistributionEnabled is true, and document that PreferClose should only be
used on clusters that support the feature (Kubernetes GA >= 1.33). Ensure the
vendoring change is applied through an overlay or dependency update rather than
a direct edit to the vendored template.

In
@packages/system/seaweedfs/charts/seaweedfs/templates/worker/worker-deployment.yaml:
- Around line 134-138: The admin flag construction for the worker is appending
the gRPC port as a dot-suffixed token, creating invalid addresses (e.g.,
host:port.grpcPort); update the template that builds the -admin flag so it uses
either .Values.worker.adminServer when set or the generated host plus only
.Values.admin.port (remove any use of .Values.admin.grpcPort in the -admin
value). Locate the block that references -admin (the template using
"seaweedfs.name", .Values.worker.adminServer, .Values.admin.port and
.Values.admin.grpcPort) and delete the conditional that appends
.Values.admin.grpcPort so the -admin flag becomes host:port only; handle gRPC
port separately if needed elsewhere.

In @packages/system/seaweedfs/patches/s3-traffic-distribution.patch:
- Line 9: The chart currently sets trafficDistribution: PreferClose which is
deprecated; update the value to a supported option (e.g., trafficDistribution:
PreferSameZone or trafficDistribution: PreferSameNode) and adjust the chart's
kubeVersion constraint to require at least Kubernetes v1.33 (the v1 API
stabilization version) so the minimum version reflects the stable API; locate
the trafficDistribution setting and the kubeVersion field in the chart and make
these two coordinated edits.

🧹 Nitpick comments (7)

packages/system/seaweedfs/charts/seaweedfs/templates/cert/worker-cert.yaml (1)

1-42: LGTM! Certificate template correctly configured.

The worker certificate template is properly structured with:

Correct conditional rendering (security enabled, external certs disabled)

Appropriate cert-manager.io/v1 API usage

Well-formed DNS names covering all cluster DNS resolution levels

Consistent pattern matching other component certificates

Optional: Remove the extra blank line at the end of the file (line 43) for consistency with Helm template conventions.
packages/system/seaweedfs/charts/seaweedfs/templates/cert/admin-cert.yaml (1)
43-43: Minor: Remove trailing blank line.

The static analysis flagged a trailing blank line at the end of the file.
🧹 Remove trailing blank line
   renewBefore: {{ .Values.certificates.renewBefore }}
 {{- end }}
-
packages/system/seaweedfs/charts/seaweedfs/templates/admin/admin-servicemonitor.yaml (1)

1-33: LGTM! ServiceMonitor configuration is correct.

The ServiceMonitor properly configures metrics collection for the admin component with appropriate conditionals, labels, and endpoint configuration.

Minor: Remove trailing blank line

Per yamllint, there's a trailing blank line at the end of the file (line 39 in the full template). This is a minor formatting inconsistency.

packages/system/seaweedfs/charts/seaweedfs/templates/admin/admin-service.yaml (1)

1-38: LGTM! Service configuration is correct.

The Service properly exposes the admin component's http, grpc, and optional metrics ports with appropriate labels and selectors.

Minor: Remove trailing blank line

Per yamllint, there's a trailing blank line at the end of the file (line 39). This is a minor formatting inconsistency.
packages/system/seaweedfs/charts/seaweedfs/templates/shared/_helpers.tpl (1)
284-290: Consider removing unused variable.

The $fullname variable is defined but never used. The helper uses ${SEAWEEDFS_FULLNAME} (shell variable) instead, which is resolved at runtime from the pod's environment.
🧹 Optional cleanup
 {{- define "seaweedfs.masterServers" -}}
-{{- $fullname := include "seaweedfs.name" . -}}
 {{- range $index := until (.Values.master.replicas | int) -}}
 {{- if $index }},{{ end -}}
 ${SEAWEEDFS_FULLNAME}-master-{{ $index }}.${SEAWEEDFS_FULLNAME}-master.{{ $.Release.Namespace }}:{{ $.Values.master.port }}
 {{- end -}}
 {{- end -}}
packages/system/seaweedfs/charts/seaweedfs/templates/worker/worker-deployment.yaml (2)
34-46: Inconsistent whitespace control in template directives.

Lines 34 and 41 use {{ with ...}} while similar constructs elsewhere use {{- with ...}}. This inconsistency may introduce unwanted blank lines in the rendered output.
Suggested fix for consistency
-      {{ with .Values.podLabels }}
+      {{- with .Values.podLabels }}
       {{- toYaml . | nindent 8 }}
       {{- end }}
       {{- with .Values.worker.podLabels }}
       {{- toYaml . | nindent 8 }}
       {{- end }}
       annotations:
-      {{ with .Values.podAnnotations }}
+      {{- with .Values.podAnnotations }}
       {{- toYaml . | nindent 8 }}
       {{- end }}
283-287: Potential indentation issues with indent usage.

Lines 283 and 286 use indent 8 combined with trim, which may produce incorrect indentation:

indent adds leading spaces, but trim removes them

The rest of the file uses nindent (which adds a newline before indenting)

At this nesting level, volumes require 8-space indentation, but nodeSelector content should align differently

Consider using nindent consistently and verify the rendered output.
Suggested fix for consistency
-        {{ tpl .Values.worker.extraVolumes . | indent 8 | trim }}
+        {{- tpl .Values.worker.extraVolumes . | nindent 8 | trim }}
       {{- if .Values.worker.nodeSelector }}
       nodeSelector:
-        {{ tpl .Values.worker.nodeSelector . | indent 8 | trim }}
+        {{- tpl .Values.worker.nodeSelector . | nindent 8 | trim }}
       {{- end }}

📜 Review details

Configuration used: defaults

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 56cba3f and bcd4b89.

📒 Files selected for processing (26)

packages/system/ingress-nginx/values.yaml
packages/system/seaweedfs/Makefile
packages/system/seaweedfs/charts/seaweedfs/Chart.yaml
packages/system/seaweedfs/charts/seaweedfs/README.md
packages/system/seaweedfs/charts/seaweedfs/dashboards/seaweedfs-grafana-dashboard.json
packages/system/seaweedfs/charts/seaweedfs/templates/admin/admin-ingress.yaml
packages/system/seaweedfs/charts/seaweedfs/templates/admin/admin-secret.yaml
packages/system/seaweedfs/charts/seaweedfs/templates/admin/admin-service.yaml
packages/system/seaweedfs/charts/seaweedfs/templates/admin/admin-servicemonitor.yaml
packages/system/seaweedfs/charts/seaweedfs/templates/admin/admin-statefulset.yaml
packages/system/seaweedfs/charts/seaweedfs/templates/cert/admin-cert.yaml
packages/system/seaweedfs/charts/seaweedfs/templates/cert/worker-cert.yaml
packages/system/seaweedfs/charts/seaweedfs/templates/filer/filer-statefulset.yaml
packages/system/seaweedfs/charts/seaweedfs/templates/master/master-statefulset.yaml
packages/system/seaweedfs/charts/seaweedfs/templates/s3/s3-ingress.yaml
packages/system/seaweedfs/charts/seaweedfs/templates/s3/s3-service.yaml
packages/system/seaweedfs/charts/seaweedfs/templates/shared/_helpers.tpl
packages/system/seaweedfs/charts/seaweedfs/templates/shared/security-configmap.yaml
packages/system/seaweedfs/charts/seaweedfs/templates/volume/volume-statefulset.yaml
packages/system/seaweedfs/charts/seaweedfs/templates/worker/worker-deployment.yaml
packages/system/seaweedfs/charts/seaweedfs/templates/worker/worker-service.yaml
packages/system/seaweedfs/charts/seaweedfs/templates/worker/worker-servicemonitor.yaml
packages/system/seaweedfs/charts/seaweedfs/values.yaml
packages/system/seaweedfs/images/seaweedfs/Dockerfile
packages/system/seaweedfs/patches/s3-traffic-distribution.patch
packages/system/seaweedfs/values.yaml

💤 Files with no reviewable changes (1)

packages/system/seaweedfs/images/seaweedfs/Dockerfile

🧰 Additional context used

📓 Path-based instructions (3)

packages/system/**

📄 CodeRabbit inference engine (AGENTS.md)

Use Helm Chart umbrella pattern with vendored upstream charts in charts/ directory

Files:

packages/system/seaweedfs/charts/seaweedfs/templates/worker/worker-servicemonitor.yaml
packages/system/seaweedfs/charts/seaweedfs/templates/shared/security-configmap.yaml
packages/system/seaweedfs/charts/seaweedfs/templates/volume/volume-statefulset.yaml
packages/system/seaweedfs/charts/seaweedfs/templates/cert/admin-cert.yaml
packages/system/seaweedfs/Makefile
packages/system/seaweedfs/charts/seaweedfs/templates/master/master-statefulset.yaml
packages/system/seaweedfs/charts/seaweedfs/templates/shared/_helpers.tpl
packages/system/seaweedfs/charts/seaweedfs/README.md
packages/system/seaweedfs/charts/seaweedfs/templates/admin/admin-servicemonitor.yaml
packages/system/ingress-nginx/values.yaml
packages/system/seaweedfs/patches/s3-traffic-distribution.patch
packages/system/seaweedfs/charts/seaweedfs/templates/admin/admin-statefulset.yaml
packages/system/seaweedfs/charts/seaweedfs/templates/worker/worker-deployment.yaml
packages/system/seaweedfs/charts/seaweedfs/templates/admin/admin-ingress.yaml
packages/system/seaweedfs/charts/seaweedfs/templates/admin/admin-service.yaml
packages/system/seaweedfs/charts/seaweedfs/templates/admin/admin-secret.yaml
packages/system/seaweedfs/charts/seaweedfs/templates/s3/s3-service.yaml
packages/system/seaweedfs/values.yaml
packages/system/seaweedfs/charts/seaweedfs/templates/cert/worker-cert.yaml
packages/system/seaweedfs/charts/seaweedfs/templates/filer/filer-statefulset.yaml
packages/system/seaweedfs/charts/seaweedfs/templates/worker/worker-service.yaml
packages/system/seaweedfs/charts/seaweedfs/values.yaml
packages/system/seaweedfs/charts/seaweedfs/Chart.yaml
packages/system/seaweedfs/charts/seaweedfs/templates/s3/s3-ingress.yaml
packages/system/seaweedfs/charts/seaweedfs/dashboards/seaweedfs-grafana-dashboard.json

**/*.yaml

📄 CodeRabbit inference engine (AGENTS.md)

Do NOT directly edit upstream Helm charts in vendored charts directory; use proper chart vendoring mechanisms

Files:

packages/system/seaweedfs/charts/seaweedfs/templates/worker/worker-servicemonitor.yaml
packages/system/seaweedfs/charts/seaweedfs/templates/shared/security-configmap.yaml
packages/system/seaweedfs/charts/seaweedfs/templates/volume/volume-statefulset.yaml
packages/system/seaweedfs/charts/seaweedfs/templates/cert/admin-cert.yaml
packages/system/seaweedfs/charts/seaweedfs/templates/master/master-statefulset.yaml
packages/system/seaweedfs/charts/seaweedfs/templates/admin/admin-servicemonitor.yaml
packages/system/ingress-nginx/values.yaml
packages/system/seaweedfs/charts/seaweedfs/templates/admin/admin-statefulset.yaml
packages/system/seaweedfs/charts/seaweedfs/templates/worker/worker-deployment.yaml
packages/system/seaweedfs/charts/seaweedfs/templates/admin/admin-ingress.yaml
packages/system/seaweedfs/charts/seaweedfs/templates/admin/admin-service.yaml
packages/system/seaweedfs/charts/seaweedfs/templates/admin/admin-secret.yaml
packages/system/seaweedfs/charts/seaweedfs/templates/s3/s3-service.yaml
packages/system/seaweedfs/values.yaml
packages/system/seaweedfs/charts/seaweedfs/templates/cert/worker-cert.yaml
packages/system/seaweedfs/charts/seaweedfs/templates/filer/filer-statefulset.yaml
packages/system/seaweedfs/charts/seaweedfs/templates/worker/worker-service.yaml
packages/system/seaweedfs/charts/seaweedfs/values.yaml
packages/system/seaweedfs/charts/seaweedfs/Chart.yaml
packages/system/seaweedfs/charts/seaweedfs/templates/s3/s3-ingress.yaml

**/*.md

📄 CodeRabbit inference engine (AGENTS.md)

Git commits should follow [component] Description format and include --signoff flag

Files:

packages/system/seaweedfs/charts/seaweedfs/README.md

🧠 Learnings (2)

📚 Learning: 2025-12-25T09:45:26.524Z

Learnt from: CR
Repo: cozystack/cozystack PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-12-25T09:45:26.524Z
Learning: Applies to **/*.yaml : Do NOT directly edit upstream Helm charts in vendored charts directory; use proper chart vendoring mechanisms

Applied to files:

packages/system/seaweedfs/charts/seaweedfs/templates/admin/admin-ingress.yaml

📚 Learning: 2025-12-25T09:45:26.524Z

Learnt from: CR
Repo: cozystack/cozystack PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-12-25T09:45:26.524Z
Learning: Applies to packages/apps/** : Use Helm Chart umbrella pattern with vendored upstream charts in `charts/` directory

Applied to files:

packages/system/seaweedfs/charts/seaweedfs/Chart.yaml

🪛 YAMLlint (1.37.1)

packages/system/seaweedfs/charts/seaweedfs/templates/worker/worker-servicemonitor.yaml