Skip to content

!feat(server): EO Pages 切换存储为 Blob#978

Merged
imaegoo merged 1 commit into
twikoojs:mainfrom
Mintimate:main
Jun 2, 2026
Merged

!feat(server): EO Pages 切换存储为 Blob#978
imaegoo merged 1 commit into
twikoojs:mainfrom
Mintimate:main

Conversation

@Mintimate

@Mintimate Mintimate commented Jun 1, 2026 •
edited by sourcery-ai Bot
Loading

Copy link
Copy Markdown
Contributor
  • 解决 KV 最终一致性导致的数据不一致问题
  • 消除边缘节点回源中心节点的互访开销，减少一跳 RTT。大幅度提高性能。
  • 统一存储层，移除 kv.js

⚠️ 破坏性变更：需手动迁移 KV 数据至 Blob，详见 README。

Summary by Sourcery

将 EO Pages 服务器实现从基于 KV 的存储迁移到基于 Blob 的数据库，并调整构建和打包配置以使用新的云函数入口点。

新特性：

  • 在 EO Pages 云函数中引入基于 Blob 的数据库层，用于评论、配置和计数器。

增强内容：

  • 将评论过滤逻辑从已移除的 KV 层迁移到新的 Blob 存储层，同时保持查询语义不变。
  • 更新构建脚本、生成的 ip2region 资源路径以及包的入口点，使其指向新的 cloud-functions 目录并集成 Blob 存储。

文档：

  • 记录 EO Pages 从 KV 存储迁移到 Blob 存储的破坏性变更，并为现有 KV 数据用户添加手动迁移步骤。
Original summary in English

Summary by Sourcery

Migrate the EO Pages server implementation from KV-based storage to a Blob-backed database and adjust build and packaging to use the new cloud function entrypoint.

New Features:

  • Introduce a Blob-based database layer for comments, configuration, and counters within the EO Pages cloud function.

Enhancements:

  • Move comment filtering logic from the removed KV layer into the new Blob storage layer while preserving query semantics.
  • Update build scripts, generated ip2region asset paths, and package entrypoint to target the new cloud-functions directory and Blob storage integration.

Documentation:

  • Document the breaking change from KV to Blob storage for EO Pages and add manual migration steps for existing KV data users.

@sourcery-ai

sourcery-ai Bot commented Jun 1, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

审阅者指南

将 EdgeOne Pages 服务器实现从基于 KV 的存储层迁移到基于 Blob 的存储层，同时内联之前在 KV 中进行的评论过滤逻辑，并相应调整构建流程、入口文件、依赖和文档。

文件级变更

变更 细节 文件
使用基于 Blob 的数据访问层替换 KV 代理层，并将之前在 KV 中处理的评论过滤逻辑内联到服务端。
  • 移除调用 /api/kv edge function 的远程 KV 代理抽象以及相关的错误处理/日志逻辑。
  • 引入可复用的评论过滤工具函数（filterComments/matchCondition），实现之前在 KV 中执行的查询语义。
  • 新增基于 Blob 的数据库封装（createBlobDatabase），通过启用强一致性的 getStore 来管理评论、配置和计数器，使用 JSON Blob 存储和针对评论的内存缓存。
  • 实现对评论的增删改查（CRUD）及批量操作（包括 ID 生成），并将配置和计数器操作迁移到 Blob 键上，使用显式的 JSON 读写（get/set）。
 src/server/eo-pages/cloud-functions/index.js
更新配置加载和处理器装配逻辑，使其使用新的 Blob 数据库层而非 KV。
  • 修改 readConfig，使其不再依赖请求对象，并改为使用 createBlobDatabase()。
  • 更新主 POST 处理器，在处理事件时构造 Blob 数据库实例，而不是 KV 代理。
 src/server/eo-pages/cloud-functions/index.js
调整构建流水线、项目结构预期以及运行时入口，使其使用 cloud-functions（Blob），并移除 KV 特定的构件。
  • 将 ip2region 构建输出路径和日志从 node-functions/* 更新为 cloud-functions/*。
  • 修改 requiredFiles 列表和项目结构输出，引用 cloud-functions 文件，并移除对 KV API edge function 的要求。
  • 将 package.json 的 main 入口设置为 cloud-functions/index.js，并添加 @edgeone/pages-blob 依赖。
  • 删除 edge-functions/api/kv.js 文件，移除 KV API 实现。
 src/server/eo-pages/build.cjs
src/server/eo-pages/package.json
src/server/eo-pages/edge-functions/api/kv.js
记录这一存储后端的破坏性变更，并为从 KV 升级到 Blob 的用户提供手动数据迁移步骤。
  • 更新 README 中的功能描述，说明现在使用的是 Blob 存储而不是 KV。
  • 新增“破坏性变更”章节，说明 KV 数据不会自动迁移，并提供使用管理面板进行导出/导入的分步说明。
  • 简化部署说明，去掉 KV namespace 绑定，并说明 Blob 会自动初始化。
 src/server/eo-pages/README.md
提示与命令

与 Sourcery 交互

  • 触发新审阅： 在 Pull Request 中评论 @sourcery-ai review。
  • 继续讨论： 直接回复 Sourcery 的审阅评论。
  • 从审阅评论生成 GitHub Issue： 在审阅评论下回复，请求 Sourcery 从该评论创建 Issue。你也可以在审阅评论下回复 @sourcery-ai issue 来从该评论创建 Issue。
  • 生成 Pull Request 标题： 在 Pull Request 标题的任意位置写上 @sourcery-ai，即可随时生成标题。你也可以在 Pull Request 中评论 @sourcery-ai title 来（重新）生成标题。
  • 生成 Pull Request 摘要： 在 Pull Request 描述正文中任意位置写上 @sourcery-ai summary，即可在对应位置生成 PR 摘要。你也可以在 Pull Request 中评论 @sourcery-ai summary 来（重新）生成摘要。
  • 生成审阅者指南： 在 Pull Request 中评论 @sourcery-ai guide，即可随时（重新）生成审阅者指南。
  • 解决所有 Sourcery 评论： 在 Pull Request 中评论 @sourcery-ai resolve，即可将所有 Sourcery 评论标记为已解决。如果你已经处理完所有评论且不想再看到它们，这会非常有用。
  • 忽略所有 Sourcery 审阅： 在 Pull Request 中评论 @sourcery-ai dismiss，即可忽略所有现有的 Sourcery 审阅。特别适用于你想从头开始新一轮审阅的场景 —— 记得再评论 @sourcery-ai review 触发新审阅！

自定义你的体验

访问你的 dashboard 以：

  • 启用或禁用审阅功能，例如 Sourcery 生成的 Pull Request 摘要、审阅者指南等。
  • 修改审阅语言。
  • 添加、移除或编辑自定义审阅说明。
  • 调整其他审阅设置。

获取帮助

Original review guide in English

Reviewer's Guide

Migrates EdgeOne Pages server implementation from a KV-based storage layer to a Blob-based storage layer, inlining comment filtering logic and adjusting build, entrypoint, dependencies, and documentation accordingly.

File-Level Changes

Change Details Files
Replace the KV proxy layer with a Blob-backed data access layer and inline comment filtering logic previously handled by KV.
  • Remove the remote KV proxy abstraction that called the /api/kv edge function and associated error handling/logging.
  • Introduce reusable comment filtering utilities (filterComments/matchCondition) that implement the query semantics previously executed in KV.
  • Add a Blob-based database wrapper (createBlobDatabase) using getStore with strong consistency to manage comments, config, and counters via JSON blobs and an in-memory cache for comments.
  • Implement CRUD and bulk operations on comments, including ID generation, and migrate config and counter operations to Blob keys with explicit JSON get/set.
 src/server/eo-pages/cloud-functions/index.js
Update configuration loading and handler wiring to use the new Blob database layer instead of KV.
  • Change readConfig to be independent of the request object and to use createBlobDatabase().
  • Update the main POST handler to construct a Blob database instance instead of a KV proxy when handling events.
 src/server/eo-pages/cloud-functions/index.js
Adjust build pipeline, project structure expectations, and runtime entrypoint to use cloud-functions (Blob) and drop KV-specific artifacts.
  • Update ip2region build output paths and logging from node-functions/* to cloud-functions/*.
  • Change the requiredFiles list and project structure output to reference cloud-functions files and remove the KV API edge function requirement.
  • Set package.json main entry to cloud-functions/index.js and add @edgeone/pages-blob as a dependency.
  • Delete the edge-functions/api/kv.js file, removing the KV API implementation.
 src/server/eo-pages/build.cjs
src/server/eo-pages/package.json
src/server/eo-pages/edge-functions/api/kv.js
Document the breaking storage backend change and manual data migration steps for users upgrading from KV to Blob.
  • Update README feature description to state Blob storage is used instead of KV.
  • Add a breaking change section explaining that KV data is not auto-migrated and provide step-by-step export/import instructions using the admin panel.
  • Simplify deployment instructions by removing KV namespace binding and mentioning Blob auto-initialization.
 src/server/eo-pages/README.md
Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

sourcery-ai[bot]
sourcery-ai Bot reviewed Jun 1, 2026
View reviewed changes

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - 我在这里给出了一些整体层面的反馈：

  • Blob 数据库层使用了一个内存中的 commentsCache，并基于读-改-写操作但没有任何并发控制，这在并发请求或多实例场景下可能导致更新丢失；建议改为每次操作都重新读取，或者使用版本号/锁机制来代替共享的可变数组。
  • 所有评论查询（getComments、countComments）目前都会把整个 comments:all 集合加载到内存中并在进程内过滤，随着数据增长，这可能会导致性能和内存问题；建议增加一些基础的分片或按键存储（例如按 URL 或按时间分桶），并在服务端实现限制/分页。
供 AI 代理使用的提示词 
Please address the comments from this code review:

## Overall Comments
- The Blob database layer uses an in-memory `commentsCache` plus read-modify-write operations without any concurrency control, which can lead to lost updates under concurrent requests or multiple instances; consider using per-operation reads or a version/lock mechanism instead of a shared mutable array.
- All comment queries (`getComments`, `countComments`) currently load the entire `comments:all` collection into memory and filter in-process, which may cause performance and memory issues as data grows; consider adding some basic sharding or keyed storage (e.g., per-url or time buckets) and server-side limiting/pagination.
Sourcery 对开源项目免费——如果你觉得我们的评审有帮助，欢迎分享 ✨
帮我变得更有用！请在每条评论上点 👍 或 👎，我会根据你的反馈改进后续评审。
Original comment in English

Hey - I've left some high level feedback:

  • The Blob database layer uses an in-memory commentsCache plus read-modify-write operations without any concurrency control, which can lead to lost updates under concurrent requests or multiple instances; consider using per-operation reads or a version/lock mechanism instead of a shared mutable array.
  • All comment queries (getComments, countComments) currently load the entire comments:all collection into memory and filter in-process, which may cause performance and memory issues as data grows; consider adding some basic sharding or keyed storage (e.g., per-url or time buckets) and server-side limiting/pagination.
Prompt for AI Agents 
Please address the comments from this code review:

## Overall Comments
- The Blob database layer uses an in-memory `commentsCache` plus read-modify-write operations without any concurrency control, which can lead to lost updates under concurrent requests or multiple instances; consider using per-operation reads or a version/lock mechanism instead of a shared mutable array.
- All comment queries (`getComments`, `countComments`) currently load the entire `comments:all` collection into memory and filter in-process, which may cause performance and memory issues as data grows; consider adding some basic sharding or keyed storage (e.g., per-url or time buckets) and server-side limiting/pagination.
Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

@Mintimate

Mintimate commented Jun 1, 2026

Copy link
Copy Markdown
Contributor Author

性能可控，满足预期:

image

考虑到 Twikoo 部分社群对项目可用性、连续性要求较高。项目维护者可以自行决定是否合并该 PR。

主要是提升非常多（一致性得到解决，RTT解决后部分情况延迟减少 70%），不然之前 PR 时候，我也承诺不再参加维护。

@imaegoo imaegoo merged commit 2fd0e3d into twikoojs:main Jun 2, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@sourcery-ai sourcery-ai[bot] sourcery-ai[bot] left review comments

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@Mintimate @imaegoo