Skip to content

Conversation

@hanabi1224
Copy link
Contributor

@hanabi1224 hanabi1224 commented Sep 18, 2025

Summary of changes

Some stats:
size of spine snapshot @ 5329650: 19G
size of spine snapshot + 1 state tree @ 5329650: 72G
size of spine snapshot + 2000 state tree @ 5329650: 75G

(a chain spine means a chain of pure tipset headers)

Changes introduced in this pull request:

Reference issue to close (if applicable)

Closes

Other information and links

Change checklist

  • I have performed a self-review of my own code,
  • I have made corresponding changes to the documentation. All new code adheres to the team's documentation standards,
  • I have added tests that prove my fix is effective or that my feature works (if possible),
  • I have made sure the CHANGELOG is up-to-date. All user-facing changes should be reflected in this document.

Summary by CodeRabbit

  • New Features
    • Snapshot export now requires a depth value with a sensible default and derives snapshots from the actual tipset epoch, improving predictability.
    • Snapshot filenames now include vendor and tipset-based info for clearer identification.
  • Bug Fixes
    • Increased timeout for tipset retrieval to reduce premature failures on slow networks.
    • Removed a pre-export restriction so smaller recent-root depths can be exported.
  • Documentation
    • Updated snapshot help text to clarify depth semantics and meaning of 0.

@hanabi1224 hanabi1224 changed the title feat: allow export spine-only snapshots with `forest-cli snapshot exp… feat: allow export spine-only snapshots with forest-cli snapshot export -d 0 Sep 18, 2025
@coderabbitai
Copy link
Contributor

coderabbitai bot commented Sep 18, 2025

Walkthrough

Made chain_muxer module and its DEFAULT_RECENT_STATE_ROOTS constant public; changed snapshot export CLI to take a non-optional depth with a DEFAULT_RECENT_STATE_ROOTS default, updated tipset retrieval to a request-with-timeout flow and snapshot filename/params usage; removed recent_roots vs finality validation in chain export RPC.

Changes

Cohort / File(s) Summary of changes
Chain sync visibility
src/chain_sync/chain_muxer.rs, src/chain_sync/mod.rs
DEFAULT_RECENT_STATE_ROOTS changed from private const to pub const; mod chain_muxer;pub mod chain_muxer;.
Snapshot CLI export (depth, tipset, filename, params)
src/cli/subcommands/snapshot_cmd.rs
Export.depth changed from Option<ChainEpochDelta> to ChainEpochDelta with default_value_t = DEFAULT_RECENT_STATE_ROOTS; removed runtime fallback to SyncConfig; tipset obtained via ChainGetTipSetByHeight::request(...).with_timeout(15m) and executed via client call; snapshot::filename now takes TrustedVendor argument; ForestChainExportParams fields updated (epoch from tipset.epoch(), recent_roots = depth, tipset_keys = tipset.key().clone().into()).
RPC chain export validation removal
src/rpc/methods/chain.rs
Removed pre-export check that returned an error when recent_roots < chain_finality; export flow (head load, timestamp, V1/V2 handling) proceeds without that validation.

Sequence Diagram(s)

sequenceDiagram
  autonumber
  participant User
  participant CLI as Snapshot CLI
  participant RPC as RPC Client
  participant Node as Node RPC Server

  User->>CLI: forest snapshot export --depth (defaults to DEFAULT_RECENT_STATE_ROOTS)
  CLI->>RPC: ChainGetTipSetByHeight.request(epoch?).with_timeout(15m)
  RPC->>Node: call(request)
  Node-->>RPC: tipset (or timeout)
  RPC-->>CLI: tipset
  CLI->>Node: Forest.ChainExport(params{ epoch=tipset.epoch(), recent_roots=depth, tipset_keys=tipset.key() })
  Note right of Node: (removed pre-check: recent_roots >= chain_finality)
  Node-->>CLI: export stream/result
  CLI-->>User: write snapshot file (filename uses TrustedVendor, chain_name, date, epoch)
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

Suggested labels

RPC

Suggested reviewers

  • akaladarshi
  • LesnyRumcajs
  • sudo-shashank

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. You can run @coderabbitai generate docstrings to improve docstring coverage.
✅ Passed checks (2 passed)
Check name Status Explanation
Title Check ✅ Passed The title clearly and concisely summarizes the primary change — enabling spine-only snapshot exports via the CLI flag -d 0 — and directly matches the PR objectives and code changes that make depth=0 supported and adjust snapshot export behavior.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
✨ Finishing touches
  • 📝 Generate Docstrings
🧪 Generate unit tests
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch hm/cli-snapshot-export-spine

Comment @coderabbitai help to get the list of available commands and usage tips.

@hanabi1224 hanabi1224 marked this pull request as ready for review September 18, 2025 14:27
@hanabi1224 hanabi1224 requested a review from a team as a code owner September 18, 2025 14:27
@hanabi1224 hanabi1224 requested review from LesnyRumcajs and akaladarshi and removed request for a team September 18, 2025 14:27
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (4)
src/chain_sync/mod.rs (1)

6-6: Avoid widening the public API surface; re-export the needed items instead.

Making the whole chain_muxer module public expands the crate’s API unnecessarily. Prefer keeping the module private (or pub(crate)) and re-export only what you need (SyncConfig, DEFAULT_RECENT_STATE_ROOTS). This avoids committing to additional symbols long-term.

Apply:

- pub mod chain_muxer;
+ mod chain_muxer;

And re-export explicitly:

 pub use self::{
     bad_block_cache::BadBlockCache,
     chain_follower::{ChainFollower, load_full_tipset},
-    chain_muxer::SyncConfig,
+    chain_muxer::{SyncConfig, DEFAULT_RECENT_STATE_ROOTS},
     consensus::collect_errs,
     sync_status::{ForkSyncInfo, ForkSyncStage, NodeSyncStatus, SyncStatusReport},
     validation::{TipsetValidationError, TipsetValidator},
 };
src/chain_sync/chain_muxer.rs (1)

6-6: Public constant LGTM; consider documenting non-negativity invariant.

Exposing DEFAULT_RECENT_STATE_ROOTS is fine. Add a brief comment stating consumers should treat this as a non-negative count, and consider a future typed wrapper if we keep passing it across RPC boundaries.

src/cli/subcommands/snapshot_cmd.rs (2)

5-5: Import via re-export to avoid depending on a public submodule path.

If you keep chain_muxer non-public (as suggested), switch the import to the re-export on chain_sync.

- use crate::chain_sync::chain_muxer::DEFAULT_RECENT_STATE_ROOTS;
+ use crate::chain_sync::DEFAULT_RECENT_STATE_ROOTS;

86-93: Tipset lookup timeout may still be too short for deep heights; consider making it configurable or longer.

15 minutes can be tight on cold nodes far from head. Either increase it (e.g., 30–60 minutes) or make it a hidden/advanced CLI flag.

-                            .with_timeout(Duration::from_secs(60 * 15)),
+                            .with_timeout(Duration::from_secs(60 * 30)),
📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between eb32d45 and bbd1f73.

📒 Files selected for processing (4)
  • src/chain_sync/chain_muxer.rs (1 hunks)
  • src/chain_sync/mod.rs (1 hunks)
  • src/cli/subcommands/snapshot_cmd.rs (4 hunks)
  • src/rpc/methods/chain.rs (0 hunks)
💤 Files with no reviewable changes (1)
  • src/rpc/methods/chain.rs
🧰 Additional context used
🧠 Learnings (2)
📓 Common learnings
Learnt from: hanabi1224
PR: ChainSafe/forest#5930
File: build.rs:64-77
Timestamp: 2025-08-13T09:43:20.301Z
Learning: hanabi1224 prefers hard compile-time errors in build scripts rather than runtime safeguards or collision detection, believing it's better to fail fast and fix root causes of issues like malformed snapshot names.
📚 Learning: 2025-08-25T13:35:24.230Z
Learnt from: hanabi1224
PR: ChainSafe/forest#5969
File: src/tool/subcommands/snapshot_cmd.rs:412-412
Timestamp: 2025-08-25T13:35:24.230Z
Learning: In src/tool/subcommands/snapshot_cmd.rs, the +1 in `last_epoch = ts.epoch() - epochs as i64 + 1` fixes an off-by-1 bug where specifying --check-stateroots=N would validate N+1 epochs instead of N epochs, causing out-of-bounds errors when the snapshot contains only N recent state roots.

Applied to files:

  • src/cli/subcommands/snapshot_cmd.rs
🧬 Code graph analysis (1)
src/cli/subcommands/snapshot_cmd.rs (3)
src/rpc/client.rs (2)
  • client (114-114)
  • request (272-285)
src/rpc/reflect/mod.rs (1)
  • request (250-260)
src/chain_sync/chain_muxer.rs (1)
  • default (18-22)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (7)
  • GitHub Check: Build Ubuntu
  • GitHub Check: cargo-publish-dry-run
  • GitHub Check: Build MacOS
  • GitHub Check: tests-release
  • GitHub Check: tests
  • GitHub Check: Build forest binaries on Linux AMD64
  • GitHub Check: All lint checks
🔇 Additional comments (1)
src/cli/subcommands/snapshot_cmd.rs (1)

111-119: Depth wiring LGTM; server semantics confirmed for 0 and anchoring. recent_roots = 0 is accepted (lookup_depth = 0 yields no stateroot range and only block headers “spine-only”), and ApiTipsetKey(Some(head)) correctly anchors the export at the specified epoch.

return Err(anyhow::anyhow!("Another chain export job is still in progress").into());
}

let chain_finality = ctx.chain_config().policy.chain_finality;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's still worth putting a warning. Zero-depth snapshots are a niche use case.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a warning on the client(forest-cli) side.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

♻️ Duplicate comments (1)
src/cli/subcommands/snapshot_cmd.rs (1)

73-76: Validation correctly implemented.

The depth validation properly handles the spine-only case by allowing 0 while rejecting negative values, which prevents invalid parameters from flowing to the export logic.

🧹 Nitpick comments (1)
src/cli/subcommands/snapshot_cmd.rs (1)

93-103: Verify the 15-minute timeout is sufficient for deep chain traversals.

The timeout change from infinite wait to 15 minutes is reasonable for most cases, but may be insufficient when requesting epochs very far behind the chain head on large networks like mainnet.

Consider making the timeout configurable or increasing it for production use cases where users might legitimately need to export from very old epochs.
Based on the search results, I can see that Filecoin considers 120 blocks (1 hour) a conservative number of confirmations and messages typically take around 30 seconds (1 blocktime). However, I need more specific information about block/epoch timing to better assess the 15-minute timeout.Perfect! Now I have confirmed that Filecoin has a block time of 30 seconds, and 120 blocks (1 hour) is considered a conservative number of confirmations for high-value transfers.

Given this information:

  • Each epoch = 30 seconds
  • 15 minutes = 900 seconds = 30 epochs
  • For very old epochs (like months or years back), the client would need to traverse potentially hundreds of thousands of epochs
📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between bbd1f73 and efa4204.

📒 Files selected for processing (1)
  • src/cli/subcommands/snapshot_cmd.rs (6 hunks)
🧰 Additional context used
🧠 Learnings (2)
📓 Common learnings
Learnt from: hanabi1224
PR: ChainSafe/forest#5930
File: build.rs:64-77
Timestamp: 2025-08-13T09:43:20.301Z
Learning: hanabi1224 prefers hard compile-time errors in build scripts rather than runtime safeguards or collision detection, believing it's better to fail fast and fix root causes of issues like malformed snapshot names.
📚 Learning: 2025-08-25T13:35:24.230Z
Learnt from: hanabi1224
PR: ChainSafe/forest#5969
File: src/tool/subcommands/snapshot_cmd.rs:412-412
Timestamp: 2025-08-25T13:35:24.230Z
Learning: In src/tool/subcommands/snapshot_cmd.rs, the +1 in `last_epoch = ts.epoch() - epochs as i64 + 1` fixes an off-by-1 bug where specifying --check-stateroots=N would validate N+1 epochs instead of N epochs, causing out-of-bounds errors when the snapshot contains only N recent state roots.

Applied to files:

  • src/cli/subcommands/snapshot_cmd.rs
🧬 Code graph analysis (1)
src/cli/subcommands/snapshot_cmd.rs (3)
src/db/car/forest.rs (1)
  • new_forest_car_temp_path_in (471-478)
src/rpc/client.rs (3)
  • client (114-114)
  • request (272-285)
  • call (92-151)
src/chain_sync/chain_muxer.rs (1)
  • default (18-22)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (7)
  • GitHub Check: cargo-publish-dry-run
  • GitHub Check: Build MacOS
  • GitHub Check: Build Ubuntu
  • GitHub Check: tests
  • GitHub Check: tests-release
  • GitHub Check: All lint checks
  • GitHub Check: Build forest binaries on Linux AMD64
🔇 Additional comments (5)
src/cli/subcommands/snapshot_cmd.rs (5)

5-5: LGTM!

The import correctly brings in the DEFAULT_RECENT_STATE_ROOTS constant that is now used as the default value for the depth parameter.


38-40: LGTM!

The depth parameter change is well-implemented:

  • Changed from Option<ChainEpochDelta> to required ChainEpochDelta with sensible default
  • Clear documentation explains that 0 means spine-only snapshots
  • Uses the same constant (DEFAULT_RECENT_STATE_ROOTS) that was previously used as fallback

78-82: Good addition of finality warning.

The warning appropriately informs users when depth is below CHAIN_FINALITY, helping them understand potential implications for lite snapshot validity while still allowing the operation to proceed.


106-117: LGTM!

The filename generation correctly integrates the new tipset-based approach:

  • Uses TrustedVendor::Forest as the first parameter
  • Derives timestamp from the tipset's minimum ticket block
  • Uses tipset.epoch() for the epoch parameter

122-130: LGTM!

The ForestChainExportParams construction correctly reflects the API changes:

  • Uses tipset.epoch() for the epoch
  • Uses depth directly (no unwrap needed)
  • Uses tipset.key().clone().into() for tipset_keys

@hanabi1224 hanabi1224 added this pull request to the merge queue Sep 19, 2025
Merged via the queue into main with commit a8f7d48 Sep 19, 2025
42 of 43 checks passed
@hanabi1224 hanabi1224 deleted the hm/cli-snapshot-export-spine branch September 19, 2025 12:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants