• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

• Replaced tier-based provider selection mechanism with a new Weighted Random Selection (WRS)
  algorithm for improved QoS-aware provider selection
• Implemented WeightedSelector component that calculates composite scores based on availability,
  latency, sync, and stake metrics with configurable weights
• Added AdaptiveMaxCalculator using T-Digest with exponential decay for adaptive P10-P90
  normalization of latency and sync metrics
• Introduced ProviderStakeCache to maintain provider stake amounts for selection algorithms
• Integrated selection statistics tracking throughout the consumer session flow with metrics
  reporting via SetProviderSelected() and StartSelectionStatsUpdater()
• Updated RPC consumer and smart router to use weighted selector configuration with new flags for
  availability, latency, sync, and stake weights
• Added test mode support to RPC provider with artificial delay, availability probability, and
  latest block configuration
• Refactored all provider optimizer tests to validate weighted selection probabilities instead of
  tier membership
• Updated consumer session manager and end-to-end tests to use new method signatures supporting
  forced provider selection via headers
• Removed legacy tier-based selection logic, tier configuration variables, and exploration mechanism
• Added comprehensive test coverage for weighted selector, adaptive calculator, and selection
  statistics

Migrate provider optimizer tests to weighted random selection algorithm

• Refactored tests to use weighted random selection algorithm instead of tier-based selection
• Replaced ChooseProvider calls to accept context.Background() as first parameter and removed
 tier return values
• Updated test assertions to validate weighted selection probabilities rather than tier membership
• Added SetDeterministicSeed() calls for deterministic test behavior and removed global
 rand.InitRandomSeed() calls
• Removed tier-based test logic (CalculateSelectionTiers, GetTier, ShiftTierChance) and
 replaced with direct selection count validation
• Added new tests for block availability calculation (TestCalculateBlockAvailability,
 TestHistoricalQuerySelectionDistribution, TestLatestBlockRequestIgnoresSync,
 TestProviderOptimizerBlockAvailabilityIntegration)
• Simplified test helper runChooseManyTimesAndReturnResults to return only selection counts
 instead of tier results
• Updated latency and sync test scenarios to use more extreme differences for deterministic weighted
 selection outcomes

protocol/provideroptimizer/provider_optimizer_test.go

Update consumer session manager test method signatures

• Updated GetSessions method calls to include two additional empty string parameters
• Modified CreateConsumerSessionManager helper to use SetDeterministicSeed() instead of global
 rand.InitRandomSeed()
• Updated getValidProviderAddresses method calls with additional empty string parameters
• Consistent parameter signature changes across all test functions that call session management
 methods

protocol/lavasession/consumer_session_manager_test.go

Update end-to-end session test method signatures

• Updated all GetSessions method calls to include two additional empty string parameters at the
 end
• Changes applied consistently across TestHappyFlowE2EEmergency,
 TestHappyFlowEmergencyInConsumer, and prepareSessionsWithFirstRelay functions

protocol/lavasession/end_to_end_lavasession_test.go

Description

The new crypto/rand implementation calls panic() on invalid bounds and on crypto/rand
failures, which can crash the process instead of handling errors gracefully. The panic message
concatenates err.Error(), which may expose internal error details if it reaches user-visible
output.

Code

utils/rand/rand.go[R42-49]

+	if n <= 0 {
+		panic("invalid argument to Intn")
+	}
+	maxVal := big.NewInt(int64(n))
+	result, err := cryptorand.Int(cryptorand.Reader, maxVal)
+	if err != nil {
+		panic("crypto/rand failed: " + err.Error())
+	}

Evidence

PR Compliance ID 3 requires handling failure points and edge cases without uncontrolled failures,
and PR Compliance ID 4 requires not exposing internal error details; the added panic() paths
violate both by crashing and embedding internal crypto/rand error strings in the panic text.

Rule 3: Generic: Robust Error Handling and Edge Case Management
Rule 4: Generic: Secure Error Handling
utils/rand/rand.go[42-49]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`utils/rand/rand.go` uses `panic()` for invalid bounds and `crypto/rand` failures, which can crash the process and may leak internal error details via panic messages.

## Issue Context
Compliance requires robust error handling (no uncontrolled crashes where possible) and secure error handling (don’t expose internal details to user-facing outputs).

## Fix Focus Areas
- utils/rand/rand.go[42-49]
- utils/rand/rand.go[60-62]
- utils/rand/rand.go[72-74]
- utils/rand/rand.go[84-86]
- utils/rand/rand.go[96-98]
- utils/rand/rand.go[106-112]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Description

IsStaticProvider ignores pairingPurge, so a static provider still serving an active WS subscription
after epoch handover can be treated as non-static and have reply signature verification enabled,
likely terminating the subscription stream.

Code

protocol/lavasession/consumer_session_manager.go[R99-125]

+// IsStaticProvider returns true when the given provider address belongs to a
+// static provider in the current pairing (including backup providers).
+//
+// This is used by higher-level flows (e.g. WS subscriptions) to decide whether
+// to skip reply signature verification, matching the behavior of regular RPC
+// calls for static providers.
+func (csm *ConsumerSessionManager) IsStaticProvider(providerAddr string) bool {
+	if csm == nil || providerAddr == "" {
+		return false
+	}
+
+	csm.lock.RLock()
+	defer csm.lock.RUnlock()
+
+	if cswp, ok := csm.pairing[providerAddr]; ok && cswp != nil {
+		cswp.Lock.RLock()
+		defer cswp.Lock.RUnlock()
+		return cswp.StaticProvider
+	}
+
+	if cswp, ok := csm.backupProviders[providerAddr]; ok && cswp != nil {
+		cswp.Lock.RLock()
+		defer cswp.Lock.RUnlock()
+		return cswp.StaticProvider
+	}
+
+	return false

Evidence

WS subscription verification explicitly uses ConsumerSessionManager.IsStaticProvider to decide
whether to call VerifyRelayReply. Providers can remain active during epoch handover while residing
in pairingPurge (connections are intentionally not purged for active subscriptions), but
IsStaticProvider only checks pairing and backupProviders, so static providers in pairingPurge will
be misdetected.

protocol/lavasession/consumer_session_manager.go[99-126]
protocol/lavasession/consumer_session_manager.go[203-214]
protocol/lavasession/consumer_session_manager.go[324-356]
protocol/chainlib/consumer_ws_subscription_manager.go[560-581]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`ConsumerSessionManager.IsStaticProvider` is used by WS subscription verification to decide whether to skip reply signature verification for static providers. During epoch handover, providers can be moved into `pairingPurge` while still actively serving subscriptions, but `IsStaticProvider` does not check `pairingPurge`, causing static providers to be misclassified and signature verification to be incorrectly enforced.

### Issue Context
- Purged providers may remain in use (subscriptions) and are explicitly excluded from immediate connection purging.
- WS subscription message verification uses `IsStaticProvider` to decide whether to call `VerifyRelayReply`.

### Fix Focus Areas
- protocol/lavasession/consumer_session_manager.go[99-126]
- protocol/lavasession/consumer_session_manager.go[203-214]
- protocol/lavasession/consumer_session_manager.go[324-356]
- protocol/chainlib/consumer_ws_subscription_manager.go[560-581]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Description

PeriodicProbeProviders creates a time.Ticker but no longer stops it; when ctx is canceled, the
function returns without releasing ticker resources.

Code

protocol/lavasession/consumer_session_manager.go[R359-363]

func (csm *ConsumerSessionManager) PeriodicProbeProviders(ctx context.Context, interval time.Duration) {
	ticker := time.NewTicker(interval)
-	defer ticker.Stop() // Ensure ticker is stopped to prevent goroutine/channel leak

	for {
		select {

Evidence

The current implementation returns on ctx.Done() without calling ticker.Stop(), which can leak
ticker resources when probe loops are started/stopped repeatedly (tests, reconfiguration, lifecycle
restarts).

protocol/lavasession/consumer_session_manager.go[359-372]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`PeriodicProbeProviders` creates a `time.Ticker` but does not stop it on exit. When `ctx.Done()` triggers, the function returns without calling `ticker.Stop()`, leaking underlying timer resources.

### Issue Context
This function can be started/stopped across component lifecycles and tests; not stopping tickers is a common source of resource leaks.

### Fix Focus Areas
- protocol/lavasession/consumer_session_manager.go[359-372]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Description

The proxy code adds println(...) and log.Println(...), producing unstructured logs and printing
full response payloads, which can hinder auditing and may expose sensitive data in logs. Compliance
requires structured logging and avoiding logging sensitive payloads.

Code

testutil/e2e/proxy/proxy.go[R236-241]

+					println(process.port+" ::: Fake Response ::: ", orderedJSON)
					fakeCount += 1
				}
				time.Sleep(500 * time.Millisecond)
-				conn.WriteMessage(msgType, []byte(orderedJSON))
+				if err := conn.WriteMessage(msgType, []byte(orderedJSON)); err != nil {
+					log.Println("Write error (cached response):", err)

Evidence

PR Compliance ID 5 requires structured logging and forbids sensitive data in logs; the added
println/log.Println statements are unstructured and include full JSON response content and raw
errors.

Rule 5: Generic: Secure Logging Practices
testutil/e2e/proxy/proxy.go[236-241]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`testutil/e2e/proxy/proxy.go` introduces unstructured logging (`println`, `log.Println`) and prints full JSON responses.

## Issue Context
Compliance requires structured logs for auditability and prohibits logging potentially sensitive payload data.

## Fix Focus Areas
- testutil/e2e/proxy/proxy.go[235-243]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Description

Weighted selection constructs per-candidate probability maps and a large attributes slice on every
selection before calling LavaFormatDebug; these allocations occur even when debug logging is
disabled.

Code

protocol/provideroptimizer/weighted_selector.go[R678-717]

+			// Calculate selection probability for each provider
+			selectionProbabilities := make(map[string]float64)
+			for _, p := range providerScores {
+				selectionProbabilities[p.Address] = (p.SelectionWeight / totalScore) * 100.0
+			}
+
+			// Log comprehensive selection summary
+			logAttrs := []utils.Attribute{
+				utils.LogAttr("GUID", ctx),
+				utils.LogAttr("selected_provider", ps.Address),
+				utils.LogAttr("selected_score", ps.SelectionWeight),
+				utils.LogAttr("selected_probability_pct", selectionProbabilities[ps.Address]),
+				utils.LogAttr("total_score", totalScore),
+				utils.LogAttr("random_value", randomValue),
+				utils.LogAttr("num_candidates", len(providerScores)),
+			}
+
+			// Add all candidates' scores and probabilities with parameter breakdown
+			for i, p := range providerScores {
+				prefix := fmt.Sprintf("candidate_%d", i+1)
+				logAttrs = append(logAttrs,
+					utils.LogAttr(prefix+"_provider", p.Address),
+					utils.LogAttr(prefix+"_score", p.SelectionWeight),
+					utils.LogAttr(prefix+"_probability_pct", selectionProbabilities[p.Address]),
+				)
+
+				// Add parameter breakdown if available
+				if i < len(scoreDetails) {
+					detail := scoreDetails[i]
+					logAttrs = append(logAttrs,
+						utils.LogAttr(prefix+"_availability", detail.Availability),
+						utils.LogAttr(prefix+"_latency", detail.Latency),
+						utils.LogAttr(prefix+"_sync", detail.Sync),
+						utils.LogAttr(prefix+"_stake", detail.Stake),
+						utils.LogAttr(prefix+"_composite", detail.Composite),
+					)
+				}
+			}
+
+			utils.LavaFormatDebug("Provider selection completed", logAttrs...)

Evidence

The selection path unconditionally allocates and populates selectionProbabilities and logAttrs
for all candidates. LavaFormatLog does log-level gating internally, but that happens after these
allocations are already performed, so disabled debug logs still pay the cost.

protocol/provideroptimizer/weighted_selector.go[670-717]
utils/lavalog.go[301-340]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`SelectProviderWithStats` builds a probability map and a large attribute slice for all candidates before calling `LavaFormatDebug`. Even when debug logs are disabled, this work still happens, adding avoidable allocations in a potentially frequent code path.

### Issue Context
`LavaFormatLog` has an internal log-level early return, but callers still pay for any pre-call computation. The fix should prevent building per-candidate log payloads when debug is not enabled.

### Fix Focus Areas
- protocol/provideroptimizer/weighted_selector.go[670-717]
- utils/lavalog.go[301-340]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools