-
Notifications
You must be signed in to change notification settings - Fork 182
feat: allow export spine-only snapshots with forest-cli snapshot export -d 0
#6097
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
forest-cli snapshot export -d 0
WalkthroughMade chain_muxer module and its DEFAULT_RECENT_STATE_ROOTS constant public; changed snapshot export CLI to take a non-optional depth with a DEFAULT_RECENT_STATE_ROOTS default, updated tipset retrieval to a request-with-timeout flow and snapshot filename/params usage; removed recent_roots vs finality validation in chain export RPC. Changes
Sequence Diagram(s)sequenceDiagram
autonumber
participant User
participant CLI as Snapshot CLI
participant RPC as RPC Client
participant Node as Node RPC Server
User->>CLI: forest snapshot export --depth (defaults to DEFAULT_RECENT_STATE_ROOTS)
CLI->>RPC: ChainGetTipSetByHeight.request(epoch?).with_timeout(15m)
RPC->>Node: call(request)
Node-->>RPC: tipset (or timeout)
RPC-->>CLI: tipset
CLI->>Node: Forest.ChainExport(params{ epoch=tipset.epoch(), recent_roots=depth, tipset_keys=tipset.key() })
Note right of Node: (removed pre-check: recent_roots >= chain_finality)
Node-->>CLI: export stream/result
CLI-->>User: write snapshot file (filename uses TrustedVendor, chain_name, date, epoch)
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related PRs
Suggested labels
Suggested reviewers
Pre-merge checks and finishing touches❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✨ Finishing touches
🧪 Generate unit tests
Comment |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 1
🧹 Nitpick comments (4)
src/chain_sync/mod.rs (1)
6-6: Avoid widening the public API surface; re-export the needed items instead.Making the whole
chain_muxermodule public expands the crate’s API unnecessarily. Prefer keeping the module private (orpub(crate)) and re-export only what you need (SyncConfig,DEFAULT_RECENT_STATE_ROOTS). This avoids committing to additional symbols long-term.Apply:
- pub mod chain_muxer; + mod chain_muxer;And re-export explicitly:
pub use self::{ bad_block_cache::BadBlockCache, chain_follower::{ChainFollower, load_full_tipset}, - chain_muxer::SyncConfig, + chain_muxer::{SyncConfig, DEFAULT_RECENT_STATE_ROOTS}, consensus::collect_errs, sync_status::{ForkSyncInfo, ForkSyncStage, NodeSyncStatus, SyncStatusReport}, validation::{TipsetValidationError, TipsetValidator}, };src/chain_sync/chain_muxer.rs (1)
6-6: Public constant LGTM; consider documenting non-negativity invariant.Exposing
DEFAULT_RECENT_STATE_ROOTSis fine. Add a brief comment stating consumers should treat this as a non-negative count, and consider a future typed wrapper if we keep passing it across RPC boundaries.src/cli/subcommands/snapshot_cmd.rs (2)
5-5: Import via re-export to avoid depending on a public submodule path.If you keep
chain_muxernon-public (as suggested), switch the import to the re-export onchain_sync.- use crate::chain_sync::chain_muxer::DEFAULT_RECENT_STATE_ROOTS; + use crate::chain_sync::DEFAULT_RECENT_STATE_ROOTS;
86-93: Tipset lookup timeout may still be too short for deep heights; consider making it configurable or longer.15 minutes can be tight on cold nodes far from head. Either increase it (e.g., 30–60 minutes) or make it a hidden/advanced CLI flag.
- .with_timeout(Duration::from_secs(60 * 15)), + .with_timeout(Duration::from_secs(60 * 30)),
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (4)
src/chain_sync/chain_muxer.rs(1 hunks)src/chain_sync/mod.rs(1 hunks)src/cli/subcommands/snapshot_cmd.rs(4 hunks)src/rpc/methods/chain.rs(0 hunks)
💤 Files with no reviewable changes (1)
- src/rpc/methods/chain.rs
🧰 Additional context used
🧠 Learnings (2)
📓 Common learnings
Learnt from: hanabi1224
PR: ChainSafe/forest#5930
File: build.rs:64-77
Timestamp: 2025-08-13T09:43:20.301Z
Learning: hanabi1224 prefers hard compile-time errors in build scripts rather than runtime safeguards or collision detection, believing it's better to fail fast and fix root causes of issues like malformed snapshot names.
📚 Learning: 2025-08-25T13:35:24.230Z
Learnt from: hanabi1224
PR: ChainSafe/forest#5969
File: src/tool/subcommands/snapshot_cmd.rs:412-412
Timestamp: 2025-08-25T13:35:24.230Z
Learning: In src/tool/subcommands/snapshot_cmd.rs, the +1 in `last_epoch = ts.epoch() - epochs as i64 + 1` fixes an off-by-1 bug where specifying --check-stateroots=N would validate N+1 epochs instead of N epochs, causing out-of-bounds errors when the snapshot contains only N recent state roots.
Applied to files:
src/cli/subcommands/snapshot_cmd.rs
🧬 Code graph analysis (1)
src/cli/subcommands/snapshot_cmd.rs (3)
src/rpc/client.rs (2)
client(114-114)request(272-285)src/rpc/reflect/mod.rs (1)
request(250-260)src/chain_sync/chain_muxer.rs (1)
default(18-22)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (7)
- GitHub Check: Build Ubuntu
- GitHub Check: cargo-publish-dry-run
- GitHub Check: Build MacOS
- GitHub Check: tests-release
- GitHub Check: tests
- GitHub Check: Build forest binaries on Linux AMD64
- GitHub Check: All lint checks
🔇 Additional comments (1)
src/cli/subcommands/snapshot_cmd.rs (1)
111-119: Depth wiring LGTM; server semantics confirmed for 0 and anchoring. recent_roots = 0 is accepted (lookup_depth = 0 yields no stateroot range and only block headers “spine-only”), and ApiTipsetKey(Some(head)) correctly anchors the export at the specified epoch.
| return Err(anyhow::anyhow!("Another chain export job is still in progress").into()); | ||
| } | ||
|
|
||
| let chain_finality = ctx.chain_config().policy.chain_finality; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's still worth putting a warning. Zero-depth snapshots are a niche use case.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added a warning on the client(forest-cli) side.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 0
♻️ Duplicate comments (1)
src/cli/subcommands/snapshot_cmd.rs (1)
73-76: Validation correctly implemented.The depth validation properly handles the spine-only case by allowing 0 while rejecting negative values, which prevents invalid parameters from flowing to the export logic.
🧹 Nitpick comments (1)
src/cli/subcommands/snapshot_cmd.rs (1)
93-103: Verify the 15-minute timeout is sufficient for deep chain traversals.The timeout change from infinite wait to 15 minutes is reasonable for most cases, but may be insufficient when requesting epochs very far behind the chain head on large networks like mainnet.
Consider making the timeout configurable or increasing it for production use cases where users might legitimately need to export from very old epochs.
Based on the search results, I can see that Filecoin considers 120 blocks (1 hour) a conservative number of confirmations and messages typically take around 30 seconds (1 blocktime). However, I need more specific information about block/epoch timing to better assess the 15-minute timeout.Perfect! Now I have confirmed that Filecoin has a block time of 30 seconds, and 120 blocks (1 hour) is considered a conservative number of confirmations for high-value transfers.Given this information:
- Each epoch = 30 seconds
- 15 minutes = 900 seconds = 30 epochs
- For very old epochs (like months or years back), the client would need to traverse potentially hundreds of thousands of epochs
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
src/cli/subcommands/snapshot_cmd.rs(6 hunks)
🧰 Additional context used
🧠 Learnings (2)
📓 Common learnings
Learnt from: hanabi1224
PR: ChainSafe/forest#5930
File: build.rs:64-77
Timestamp: 2025-08-13T09:43:20.301Z
Learning: hanabi1224 prefers hard compile-time errors in build scripts rather than runtime safeguards or collision detection, believing it's better to fail fast and fix root causes of issues like malformed snapshot names.
📚 Learning: 2025-08-25T13:35:24.230Z
Learnt from: hanabi1224
PR: ChainSafe/forest#5969
File: src/tool/subcommands/snapshot_cmd.rs:412-412
Timestamp: 2025-08-25T13:35:24.230Z
Learning: In src/tool/subcommands/snapshot_cmd.rs, the +1 in `last_epoch = ts.epoch() - epochs as i64 + 1` fixes an off-by-1 bug where specifying --check-stateroots=N would validate N+1 epochs instead of N epochs, causing out-of-bounds errors when the snapshot contains only N recent state roots.
Applied to files:
src/cli/subcommands/snapshot_cmd.rs
🧬 Code graph analysis (1)
src/cli/subcommands/snapshot_cmd.rs (3)
src/db/car/forest.rs (1)
new_forest_car_temp_path_in(471-478)src/rpc/client.rs (3)
client(114-114)request(272-285)call(92-151)src/chain_sync/chain_muxer.rs (1)
default(18-22)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (7)
- GitHub Check: cargo-publish-dry-run
- GitHub Check: Build MacOS
- GitHub Check: Build Ubuntu
- GitHub Check: tests
- GitHub Check: tests-release
- GitHub Check: All lint checks
- GitHub Check: Build forest binaries on Linux AMD64
🔇 Additional comments (5)
src/cli/subcommands/snapshot_cmd.rs (5)
5-5: LGTM!The import correctly brings in the
DEFAULT_RECENT_STATE_ROOTSconstant that is now used as the default value for the depth parameter.
38-40: LGTM!The depth parameter change is well-implemented:
- Changed from
Option<ChainEpochDelta>to requiredChainEpochDeltawith sensible default- Clear documentation explains that 0 means spine-only snapshots
- Uses the same constant (
DEFAULT_RECENT_STATE_ROOTS) that was previously used as fallback
78-82: Good addition of finality warning.The warning appropriately informs users when depth is below
CHAIN_FINALITY, helping them understand potential implications for lite snapshot validity while still allowing the operation to proceed.
106-117: LGTM!The filename generation correctly integrates the new tipset-based approach:
- Uses
TrustedVendor::Forestas the first parameter- Derives timestamp from the tipset's minimum ticket block
- Uses
tipset.epoch()for the epoch parameter
122-130: LGTM!The ForestChainExportParams construction correctly reflects the API changes:
- Uses
tipset.epoch()for the epoch- Uses
depthdirectly (no unwrap needed)- Uses
tipset.key().clone().into()for tipset_keys
Summary of changes
Some stats:
size of spine snapshot @ 5329650: 19G
size of spine snapshot + 1 state tree @ 5329650: 72G
size of spine snapshot + 2000 state tree @ 5329650: 75G
(a chain spine means a chain of pure tipset headers)
Changes introduced in this pull request:
Reference issue to close (if applicable)
Closes
Other information and links
Change checklist
Summary by CodeRabbit