feat: Data reliability removal#2134
Merged
Merged
Conversation
Test Results 7 files ± 0 128 suites - 3 33m 31s ⏱️ -55s Results for commit cc2ec0e. ± Comparison against base commit 087295c. This pull request removes 80 and adds 46 tests. Note that renamed tests count towards both.♻️ This comment has been updated with latest results. |
Disable Data Reliability feature without breaking changes: This is Phase 1 (Conservative approach): - All DR functions kept (not deleted, just not called) - All packages kept (finalizationconsensus, reliabilitymanager, x/conflict) - All proto fields kept (FinalizedBlocksHashes, SigBlocks) - Fully reversible - can uncomment code blocks if needed - No proto/gRPC breaking changes
Phase 2 DR removal - eliminate DataReliabilityParams() interface method. Remove from ChainParser interface and all implementations. Update SignRelayResponse() to remove dataReliabilityEnabled parameter. Replace all calls with hardcoded behavior (always disabled). Update tests and TypeScript SDK. Proto fields remain for backward compatibility.
Phase 2 DR removal - delete finalizationconsensus, finalizationverification, reliabilitymanager packages. Deleted directories: - protocol/lavaprotocol/finalizationconsensus/ - protocol/lavaprotocol/finalizationverification/ - protocol/rpcprovider/reliabilitymanager/ Removed all references: - Deleted vote_updater.go (only used for reliability manager) - Removed VoteEvents(), getLatestVoteEvents() methods from updaters - Removed SendVoteReveal(), SendVoteCommitment() from state trackers - Removed RegisterReliabilityManagerForVoteUpdates() interface method - Removed reliabilityManager parameter from ServeRPCRequests() - Removed all imports and mock methods x/conflict module remains intact (dormant) - these were client libraries. Build status: ✅ Compiles successfully
Fixed nil pointer panic at rpcprovider_server.go:186 that occurred after removing data reliability components (finalizationconsensus, finalizationverification, reliabilitymanager). Changes: - Replaced all reliabilityManager references with chainTracker in provider code - reliabilityManager was just a wrapper around chainTracker methods (GetLatestBlockData, GetLatestBlockNum) - Updated RPCProviderServer to use chainTracker directly - Fixed ServeRPCRequests to accept chainTracker parameter - Updated all tests (unit and integration) to use MockChainTracker - Removed reliabilitymanager imports Part of Data Reliability removal - Phase 2
Completed Phase 2 cleanup by removing final unused DR functions and fields: 1. RelayProcessor (relaycore/relay_processor.go): - Removed skipDataReliability field - Removed SetSkipDataReliability() and GetSkipDataReliability() methods 2. Request Builder (lavaprotocol/request_builder.go): - Removed VerifyReliabilityResults() function - Removed compareRelaysFindConflict() function - Removed findFirstDifferentChar() helper function - Removed unused imports: bytes, common, conflicttypes, conflictconstruct 3. State Tracker (statetracker/tx_sender.go): - Removed TxSenderConflictDetection() function - Removed conflicttypes import Part of Data Reliability removal - Phase 2
Removed all 'Data Reliability disabled - Phase 2' comments from the codebase. Kept only one clean comment in RPCProviderServer struct to document that chainTracker replaced reliabilityManager (which was just a wrapper). Also removed: - Last two SetSkipDataReliability() calls that were missed - Cleaned up empty lines left by comment removal
All DR functionality has been completely removed from Phase 2.
- Remove all 'Data Reliability disabled - Phase 2' comments (29 occurrences) - Fix mockReliabilityManager bug: rename to mockChainTracker - Rename DataReliabilityTimeoutIncrease to CacheWriteTimeout - Remove Data Reliability session infrastructure: - Delete DataReliabilitySessionId, DataReliabilityRelayNumber, DataReliabilityCuSum constants - Delete getDataReliabilitySingleSession() (dead code - never called) - Delete PrepareDataReliabilitySessionForUsage() and onDataReliabilitySessionFailure() - Remove isDataReliability field from ProviderSessionsWithConsumerProject - Remove Session ID 0 reservation and skip logic - Delete DataReliabilitySession struct - Remove 6 DR-related error types - Fix MockChainTracker to properly implement IChainTracker interface - Update TypeScript SDK to remove DR comments Impact: Session ID 0 now available for use, simplified session management
…ue to missing finalization logic
…updating calls in consumer_ws_subscription_manager_test.go
…ion_manager_test.go
Remove unused named return parameters (success, err) that were causing ineffassign linter error. The function now uses unnamed return types.
1. Fix TestRelayInnerProviderUniqueIdFlow failure: - Always reset ProviderTrailer before each relay to ensure fresh trailer data - Prevents stale trailer values from persisting across multiple relay calls - This ensures provider unique ID verification works correctly 2. Remove obsolete TestSameProviderConflictBasicResponseCheck: - Test was checking for conflict detection which was removed in DR removal - Related test TestSameProviderConflictReport was already removed - This test no longer serves a purpose without Data Reliability feature
…dRelays Same issue as in RPCConsumerServer - the sendCraftedRelays function had unused named return parameters (success, err) causing ineffassign linter error. Changed to unnamed return types and ignored the unused err from craftRelay since it's already handled by the ok boolean.
…mer pattern The test was failing in CI because the mock only checked for exactly 1 option, but the actual gRPC call may pass multiple options. Updated to: 1. Loop through all opts to find TrailerCallOption (not just len(opts)==1) 2. Reset trailer to nil before each call in the test helper This matches the pattern used in rpcconsumer test which passes successfully.
Temporarily disable the provider-side cache test as the caching logic is currently tied to finalized block data, which is not handled after the removal of Data Reliability. The test will be re-enabled once the caching mechanism is refactored to decouple it from Data Reliability.
Deleted the backup file chain_router.go.backup as it is no longer needed. This helps in maintaining a cleaner codebase.
- Added missing import for protocol/provideroptimizer package - Required for CumulativeProbabilityFunctionForPoissonDist() used in line 1324 - Resolves compilation error after rebasing data_reliability_removal branch onto main
e38cb4a to
ffa852c
Compare
- Created GetParametersForCache() function to calculate cache parameters without DR - Extracts latestBlock from chainTracker.GetAtomicLatestBlockNum() - Gets requestedBlockHash from chainTracker.GetLatestBlockData() for specific blocks - Calculates finalized status using spectypes.IsFinalizedBlock() - Cache now works independently of Data Reliability feature Technical details: - latestBlock is used as SeenBlock in cache for inter-consumer consistency - ChainTracker polls node periodically (adaptive based on block time) to keep latestBlock fresh - reply.LatestBlock is NOT populated by node (only reply.Data), so must calculate separately - Supports archive functionality and provider seed block correctly Fixes broken cache that was always using finalized=false and requestedBlockHash=nil
- Set reply.LatestBlock = latestBlock before signing response - This replaces the behavior previously done by BuildRelayFinalizedBlockHashes() - Consumers use reply.LatestBlock for consistency tracking and provider selection - Without this, reply.LatestBlock would be 0 (node only populates reply.Data) Critical for: - Consumer consistency tracking (SetSeenBlock) - Provider selection based on block height - Archive extension decisions - QoS metrics that depend on block information
…moval When Data Reliability was removed, code was changed to always use VerificationsOnlyChainFetcher which doesn't poll for new blocks. This broke the lava_latest_block metric on provider side. Restored full ChainFetcher (previously used when DR was enabled) which: - Actively polls node for new blocks - Triggers recordMetricsOnNewBlock callback - Updates lava_latest_block metric properly Also added comprehensive [METRIC-TRACE] logging to track: - ChainTracker callback triggers - Block detection and processing - Metric updates Fixes provider metrics that were not being populated.
- testutil/common/common.go: return acc explicitly instead of naked return - testutil/keeper/keepers_init.go: return err explicitly instead of naked return - protocol/chainlib/provider_node_subscription_manager.go: fix comment indentation - protocol/integration/protocol_test.go: remove extra blank line
- Add --cache-latest-block flag (default: false) to control caching of LATEST_BLOCK requests - Convert LATEST_BLOCK (-2) to actual block number before caching to comply with cache service validation - Finalized blocks are always cached regardless of this flag - Cache service rejects negative block numbers, so conversion is required for LATEST_BLOCK - When disabled, only historical/finalized blocks are cached
…BlockEnabled parameter
- Changed log messages in rpcprovider_server.go and rpcprovider.go from LavaFormatInfo to LavaFormatDebug for cache and metric trace events. - This change enhances log verbosity for debugging purposes without altering the functionality of the cache operations.
- Deleted the old provider_state_machine.go and rpcprovider_server.go.backup files as they are no longer needed. - This cleanup helps streamline the codebase and reduces clutter from obsolete files.
…derSideCache - Re-add missing cache hit assertions that were removed in cc3519d - Test now properly verifies that the provider-side cache works - Validates that 2 cache hits occur for repeated requests to block 1000 - Confirms finalization cache functionality after DR removal
… provider observations - Add cleanup logic in Record() to remove observations older than 1 hour - Prevents unbounded memory growth from inactive/offline providers - Cleanup happens automatically during normal operation (no background goroutine) - Map size is now bounded by number of active providers (those reporting within last hour) - Addresses CR feedback about memory leak in provider observation tracking
- Updated `TestChainTrackerCallbacks` to track actual blocks processed and ensure total blocks processed matches total advancement. - Introduced new tests for `LatestBlockEstimator` to cover various scenarios including handling of observations, clock skew, and invalid inputs. - Added comprehensive tests for `GetParametersForCache` in `rpcprovider` to validate behavior for different block requests and error handling. - Improved mock implementations to facilitate testing of chain tracker and RPC provider functionalities.
…to data_reliability_removal
…provider_server_test.go - Add AppendHeader implementation for mockChainMessageForProviderHeader - Remove unused imports: context, pairingtypes, spectypes - Fixes linter errors introduced in previous test additions
- Fix trailing whitespace in cache_parameters_test.go - Fix indentation in rpcprovider_server.go - Fix trailing newline in rpcprovider_server_test.go - Addresses CI gofmt/gofumpt linter errors
TestGetParametersForCache_LatestBlock was incorrectly expecting LATEST_BLOCK to be finalized. By definition, the latest/current block cannot be finalized as it needs to age by blockDistanceToFinalization blocks first. The finalization check: requestedBlock <= latestBlock - finalizationCriteria For LATEST_BLOCK: 1000 <= 1000 - 15 → 1000 <= 985 → FALSE (correct) Changed expectation from require.True to require.False with correct explanation.
avitenzer
approved these changes
Jan 8, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Closes: #XXXX
Author Checklist
All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow up issues.
I have...
!in the type prefix if API or client breaking changemainbranchReviewers Checklist
All items are required. Please add a note if the item is not applicable and please add
your handle next to the items reviewed if you only reviewed selected items.
I have...