• Introduces centralized error classification system with two-tier architecture (Tier 2:
  chain-specific, Tier 1: generic/transport)
• Implements 100+ named error codes across four layers (Protocol, Node, Blockchain, User) with
  structured metadata (code, name, category, retryable flag)
• Adds comprehensive error matchers (CodeEquals, MessageContains, MessageRegex,
  HTTPStatusContains, GRPCCodeEquals) for flexible pattern matching
• Defines ChainFamily enum mapping 50+ chain IDs to families (EVM, Solana, Bitcoin, Cosmos,
  Starknet, NEAR, Aptos, etc.)
• Implements LavaError struct with Is() and ABCICode() methods for error.Is() compatibility
  and gRPC wire protocol support
• Adds DetectConnectionError with three-layer detection (structured, string fallback, syscall
  errno) for connection-level failures
• Integrates structured logging via LogCodedError/LogCodedWarning with error_code, error_name,
  error_category, retryable, and chain_id fields
• Implements Prometheus lava_errors_total counter with full label cardinality for observability
• Provides legacy sdkerrors → LavaError mapping for backward compatibility
• Refactors all relay path error handlers (JSON-RPC, REST, gRPC, Tendermint) to use unified
  classification with chain context
• Enhances relay error selection with classification-aware precedence (majority consensus,
  external-beats-internal preference)
• Fixes JSON-RPC empty error message handling bug
• Adds 1000+ lines of comprehensive test coverage including registry invariants, classification
  validation, fixture-based regression tests, and metrics integration

Add comprehensive error registry test coverage

• Comprehensive test suite (1030 lines) for error registry integrity, classification, and matchers
• Tests registry invariants: no duplicate codes/names, code ranges, and all error codes registered
• Validates error classification across chain families (EVM, Solana, Bitcoin, Cosmos, Starknet,
 NEAR, Aptos)
• Tests connection error detection, user input validation, and unsupported method classification
• Verifies matcher precedence and prevents shadowed matchers in Tier-1 and Tier-2 mappings

protocol/common/error_registry_test.go

Implement error classification with Tier-1 and Tier-2 matchers

• Implements Tier-2 chain-specific error mappings for Solana, Bitcoin, Cosmos, Starknet, and NEAR
• Implements Tier-1 generic error mappings partitioned by transport (JSON-RPC, REST, gRPC)
• Adds ClassifyError central classification function with connection error precedence
• Implements DetectConnectionError with three-layer detection (structured, string fallback,
 syscall errno)
• Adds helper functions: IsUnsupportedMethodError, IsUserInputError, ClassifyMessage,
 IsClientCancellation

protocol/common/error_classifier.go

Integrate structured error logging and classification

• Replace utils.LavaFormatError calls with common.LogCodedError and common.LogCodedWarning for
 structured error logging
• Add error classification via common.ClassifyLegacyError and common.ExtractLavaError for
 protocol and node errors
• Refactor unsupported method error handling to use chainlib.IsUnsupportedMethodErrorType and
 return early
• Update error logging in session management, relay validation, and subscription handling

protocol/rpcprovider/rpcprovider_server.go

Description

A new documentation file is named with hyphens (error-registry-design.md) instead of snake_case,
violating the repository naming convention requirement. This reduces consistency and can break
tooling or conventions that assume snake_case paths.

Code

error-registry-design.md[1]

+# Error Categorization & Standardized Logging Plan

Evidence

PR Compliance ID 4 requires file names to use snake_case. The PR adds error-registry-design.md,
which uses hyphens rather than underscores.

AGENTS.md
error-registry-design.md[1-1]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
A newly added documentation file uses hyphens in its filename instead of the required `snake_case` convention.

## Issue Context
Compliance requires file names to be `snake_case`.

## Fix Focus Areas
- error-registry-design.md[1-1]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Description

Several newly added tests do not follow the required TestComponent_Scenario naming convention
(missing the _ separator). This reduces test discoverability/consistency and violates the project
test naming standard.

Code

protocol/common/error_registry_test.go[R18-245]

+func TestRegistryNoDuplicateCodes(t *testing.T) {
+	// registerError panics on duplicate codes at init time,
+	// but let's also verify the registry is consistent.
+	seen := make(map[uint32]string)
+	for code, le := range errorRegistry {
+		if existing, ok := seen[code]; ok {
+			t.Errorf("duplicate code %d: %s and %s", code, existing, le.Name)
+		}
+		seen[code] = le.Name
+		assert.Equal(t, code, le.Code, "registry key %d doesn't match error code %d (%s)", code, le.Code, le.Name)
+	}
+}
+
+func TestRegistryNoDuplicateNames(t *testing.T) {
+	seen := make(map[string]uint32)
+	for _, le := range errorRegistry {
+		if existingCode, ok := seen[le.Name]; ok {
+			t.Errorf("duplicate name %s: codes %d and %d", le.Name, existingCode, le.Code)
+		}
+		seen[le.Name] = le.Code
+	}
+}
+
+func TestRegistryCodeRanges(t *testing.T) {
+	for _, le := range errorRegistry {
+		if le.Code == 0 {
+			continue // UNKNOWN_ERROR
+		}
+		switch {
+		case le.Code >= 1000 && le.Code < 2000:
+			assert.Equal(t, CategoryInternal, le.Category,
+				"code %d (%s) is in protocol range but not CategoryInternal", le.Code, le.Name)
+		case le.Code >= 2000 && le.Code < 5000:
+			assert.Equal(t, CategoryExternal, le.Category,
+				"code %d (%s) is in external range but not CategoryExternal", le.Code, le.Name)
+		default:
+			t.Errorf("code %d (%s) is outside valid ranges (1000-4999)", le.Code, le.Name)
+		}
+	}
+}
+
+func TestAllErrorCodesRegistered(t *testing.T) {
+	// Spot-check that key error codes are in the registry
+	// Verify UNKNOWN_ERROR is registered at code 0
+	assert.Equal(t, LavaErrorUnknown, getLavaError(0))
+	assert.Equal(t, "UNKNOWN_ERROR", getLavaError(0).Name)
+
+	codes := []uint32{
+		1001, // CONNECTION_TIMEOUT
+		1002, // CONNECTION_REFUSED
+		2001, // METHOD_NOT_FOUND
+		2005, // RATE_LIMITED
+		3001, // NONCE_TOO_LOW
+		3101, // EXECUTION_REVERTED
+		4001, // PARSE_ERROR
+	}
+	for _, code := range codes {
+		le := getLavaError(code)
+		assert.NotEqual(t, LavaErrorUnknown, le, "code %d should be registered", code)
+		assert.Equal(t, code, le.Code)
+	}
+}
+
+func TestRegisterError_RejectsReservedCodeZero(t *testing.T) {
+	assert.PanicsWithValue(t,
+		"error code 0 is reserved for LavaErrorUnknown; MY_NEW_ERROR must use a non-zero code",
+		func() {
+			registerError(&LavaError{Code: 0, Name: "MY_NEW_ERROR"})
+		},
+	)
+}
+
+func TestRegisterError_DuplicateCodeMentionsExistingName(t *testing.T) {
+	// LavaErrorConnectionTimeout has code 1001 — re-registering under the same
+	// code but a different name must panic with a message that names the existing
+	// owner so the offender can find the collision fast.
+	defer func() {
+		r := recover()
+		if r == nil {
+			t.Fatalf("expected panic on duplicate code")
+		}
+		msg, ok := r.(string)
+		if !ok {
+			t.Fatalf("expected string panic, got %T: %v", r, r)
+		}
+		assert.Contains(t, msg, "duplicate error code: 1001")
+		assert.Contains(t, msg, "PROTOCOL_CONNECTION_TIMEOUT") // existing owner
+		assert.Contains(t, msg, "COLLIDING_NAME")              // new entrant
+	}()
+	registerError(&LavaError{Code: 1001, Name: "COLLIDING_NAME"})
+}
+
+func TestRegisterError_DuplicateNameMentionsExistingCode(t *testing.T) {
+	defer func() {
+		r := recover()
+		if r == nil {
+			t.Fatalf("expected panic on duplicate name")
+		}
+		msg, ok := r.(string)
+		if !ok {
+			t.Fatalf("expected string panic, got %T: %v", r, r)
+		}
+		assert.Contains(t, msg, "duplicate error name: PROTOCOL_CONNECTION_TIMEOUT")
+		assert.Contains(t, msg, "existing code 1001")
+	}()
+	registerError(&LavaError{Code: 99999, Name: "PROTOCOL_CONNECTION_TIMEOUT"})
+}
+
+// ---------------------------------------------------------------------------
+// Lookup helper tests
+// ---------------------------------------------------------------------------
+
+func TestErrorRegistry_GetLavaErrorByCode(t *testing.T) {
+	le := getLavaError(1001)
+	assert.Equal(t, "PROTOCOL_CONNECTION_TIMEOUT", le.Name)
+
+	le = getLavaError(99999)
+	assert.Equal(t, LavaErrorUnknown, le)
+}
+
+func TestErrorRegistry_GetLavaErrorByName(t *testing.T) {
+	le := getLavaErrorByName("PROTOCOL_CONNECTION_TIMEOUT")
+	assert.Equal(t, uint32(1001), le.Code)
+
+	le = getLavaErrorByName("NONEXISTENT")
+	assert.Equal(t, LavaErrorUnknown, le)
+}
+
+func TestErrorRegistry_IsRetryableStates(t *testing.T) {
+	assert.True(t, isRetryable(1001))  // CONNECTION_TIMEOUT
+	assert.False(t, isRetryable(1004)) // TLS_MISMATCH
+	assert.False(t, isRetryable(3001)) // NONCE_TOO_LOW
+	assert.True(t, isRetryable(0))     // UNKNOWN — retryable by default
+}
+
+func TestErrorRegistry_IsInternalExternalFlags(t *testing.T) {
+	assert.True(t, IsInternal(1001))  // PROTOCOL_CONNECTION_TIMEOUT
+	assert.False(t, IsExternal(1001)) // not external
+
+	assert.True(t, IsExternal(2001))  // NODE_METHOD_NOT_FOUND
+	assert.False(t, IsInternal(2001)) // not internal
+
+	assert.True(t, IsExternal(3001)) // CHAIN_NONCE_TOO_LOW
+	assert.True(t, IsExternal(4001)) // USER_PARSE_ERROR
+	assert.True(t, IsExternal(0))    // UNKNOWN — external
+}
+
+// ---------------------------------------------------------------------------
+// LavaError as error interface + errors.Is tests
+// ---------------------------------------------------------------------------
+
+func TestLavaError_String(t *testing.T) {
+	assert.Equal(t, "[3001] CHAIN_NONCE_TOO_LOW", LavaErrorChainNonceTooLow.String())
+}
+
+func TestLavaError_ABCICode(t *testing.T) {
+	assert.Equal(t, uint32(3001), LavaErrorChainNonceTooLow.ABCICode())
+	assert.Equal(t, uint32(1001), LavaErrorConnectionTimeout.ABCICode())
+	assert.Equal(t, uint32(0), LavaErrorUnknown.ABCICode())
+}
+
+func TestLavaError_IsNonMatch(t *testing.T) {
+	// Is returns false for non-LavaError targets
+	assert.False(t, LavaErrorChainNonceTooLow.Is(errors.New("not a LavaError")))
+}
+
+func TestLavaWrappedError_EmptyContext(t *testing.T) {
+	wrapped := NewLavaError(LavaErrorChainNonceTooLow, "")
+	assert.Contains(t, wrapped.Error(), "CHAIN_NONCE_TOO_LOW")
+}
+
+func TestLavaWrappedError_IsNonMatch(t *testing.T) {
+	wrapped := NewLavaError(LavaErrorChainNonceTooLow, "context")
+	// Is returns false for non-LavaError targets
+	assert.False(t, errors.Is(wrapped, errors.New("not a LavaError")))
+}
+
+func TestLavaWrappedError_Unwrap(t *testing.T) {
+	wrapped := NewLavaError(LavaErrorChainNonceTooLow, "context")
+	unwrapped := errors.Unwrap(wrapped)
+	require.NotNil(t, unwrapped)
+	assert.Equal(t, LavaErrorChainNonceTooLow, unwrapped)
+}
+
+func TestLavaError_ErrorInterface(t *testing.T) {
+	var err error = LavaErrorChainNonceTooLow
+	assert.Contains(t, err.Error(), "CHAIN_NONCE_TOO_LOW")
+}
+
+func TestLavaError_ErrorsIs(t *testing.T) {
+	// Direct match
+	assert.True(t, errors.Is(LavaErrorChainNonceTooLow, LavaErrorChainNonceTooLow))
+	assert.False(t, errors.Is(LavaErrorChainNonceTooLow, LavaErrorConnectionTimeout))
+
+	// Wrapped with NewLavaError
+	wrapped := NewLavaError(LavaErrorChainNonceTooLow, "tx failed")
+	assert.True(t, errors.Is(wrapped, LavaErrorChainNonceTooLow))
+	assert.False(t, errors.Is(wrapped, LavaErrorConnectionTimeout))
+	assert.Contains(t, wrapped.Error(), "tx failed")
+
+	// Wrapped with fmt.Errorf %w
+	doubleWrapped := fmt.Errorf("relay error: %w", wrapped)
+	assert.True(t, errors.Is(doubleWrapped, LavaErrorChainNonceTooLow))
+}
+
+// ---------------------------------------------------------------------------
+// ErrorCategory / ErrorSubCategory tests
+// ---------------------------------------------------------------------------
+
+func TestErrorCategoryString(t *testing.T) {
+	assert.Equal(t, "internal", CategoryInternal.String())
+	assert.Equal(t, "external", CategoryExternal.String())
+	assert.Equal(t, "unknown", ErrorCategory(99).String())
+}
+
+func TestErrorSubCategoryString(t *testing.T) {
+	assert.Equal(t, "none", SubCategoryNone.String())
+	assert.Equal(t, "unsupported_method", SubCategoryUnsupportedMethod.String())
+	assert.Equal(t, "user_error", SubCategoryUserError.String())
+}
+
+func TestUnsupportedMethodSubCategory(t *testing.T) {
+	// 2002 (NODE_METHOD_NOT_SUPPORTED) is intentionally excluded: it's retryable on another provider
+	unsupportedCodes := []uint32{2001, 2008, 2009, 2010}
+	for _, code := range unsupportedCodes {
+		le := getLavaError(code)
+		require.NotEqual(t, LavaErrorUnknown, le, "code %d not registered", code)
+		assert.True(t, le.SubCategory.IsUnsupportedMethod(),

Evidence

PR Compliance ID 5 requires test names to follow TestComponent_Scenario. The added tests include
names like TestRegistryNoDuplicateCodes, TestRegistryNoDuplicateNames, and
TestErrorCategoryString, which do not contain the _ separator between component and scenario.

AGENTS.md
protocol/common/error_registry_test.go[18-245]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
New tests in `protocol/common/error_registry_test.go` do not follow the `TestComponent_Scenario` naming convention (they omit the `_` separator).

## Issue Context
The repository standard requires consistent test naming for readability and discoverability.

## Fix Focus Areas
- protocol/common/error_registry_test.go[18-245]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Description

ExtractNodeErrorDetails correctly extracts JSON-RPC .error.code/.error.message from an
rpcclient.HTTPError body, but then unconditionally overwrites errorCode with the HTTP status code.
This breaks downstream classification (ClassifyNodeErrorWithDetails → ClassifyError) for
gateways/proxies that return JSON-RPC error bodies with non-2xx HTTP statuses.

Code

protocol/chainlib/node_error_handler.go[R51-70]

+	// Extract error code from HTTP-wrapped JSON-RPC errors
+	if jsonMsg := TryRecoverNodeErrorFromClientError(nodeError); jsonMsg != nil && jsonMsg.Error != nil {
+		errorCode = jsonMsg.Error.Code
+		if jsonMsg.Error.Message != "" {
+			errorMessage = jsonMsg.Error.Message
+		}
+	}

-// NewUnsupportedMethodError creates a new UnsupportedMethodError with optional method name
-func NewUnsupportedMethodError(originalError error, methodName string) *UnsupportedMethodError {
-	return &UnsupportedMethodError{
-		originalError: originalError,
-		methodName:    methodName,
+	// Extract gRPC status code
+	if st, ok := status.FromError(nodeError); ok {
+		errorCode = int(st.Code())
+		if st.Message() != "" {
+			errorMessage = st.Message()
+		}
	}
-}

-// SolanaNonRetryableError represents a Solana error that should not be retried.
-// Currently covers error code -32009 ("missing in long-term storage") which indicates
-// the slot data is permanently unavailable.
-// Note: -32007 (ledger jump) IS retryable as another provider may have the data.
-type SolanaNonRetryableError struct {
-	originalError error
-}
+	// Extract HTTP status code
+	if httpError, ok := nodeError.(rpcclient.HTTPError); ok {
+		errorCode = httpError.StatusCode
+	}

Evidence

The function documents a priority order (JSON-RPC body first, HTTP status last) but the final HTTP
status extraction overwrites the previously extracted JSON-RPC code. ClassifyNodeErrorWithDetails
then uses this overwritten code for error classification.

protocol/chainlib/node_error_handler.go[47-73]
protocol/chainlib/node_error_handler.go[349-360]
protocol/chainlib/node_error_handler.go[92-99]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`ExtractNodeErrorDetails` extracts a JSON-RPC error code from an `rpcclient.HTTPError` body, but then overwrites it with `httpError.StatusCode`. This violates the function’s documented precedence and causes misclassification.

### Issue Context
This affects `ClassifyNodeErrorWithDetails`, which relies on `ExtractNodeErrorDetails` outputs to drive `common.ClassifyError`.

### Fix Focus Areas
- protocol/chainlib/node_error_handler.go[47-73]

### Suggested fix
Refactor the extraction logic to respect precedence, e.g.:
- If a JSON-RPC body was successfully parsed (and `.error` exists), **do not** overwrite `errorCode` with HTTP status.
- Prefer explicit control flow (`if/else if/else`) or guard the HTTP status assignment with `if errorCode == 0` (or another sentinel indicating “no structured code found”).

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Description

RelayProcessor.buildFailureResult calls LogCodedError and then immediately returns
utils.LavaFormatError with the same description/error, producing duplicate error logs for one
failure. Because LogCodedError also emits the Prometheus metric, this path can double-increment
lava_errors_total for a single relay failure.

Code

protocol/relaycore/relay_processor.go[R861-869]

+	// Log with classified error code for metrics/observability
+	if bestLavaError != nil {
+		chainID, _ := rp.chainIdAndApiInterfaceGetter.GetChainIdAndApiInterface()
+		common.LogCodedError("failed relay, insufficient results", processingError, bestLavaError,
+			chainID, 0, "", utils.LogAttr("GUID", rp.guid))
+	}
+
	return returnedResult, utils.LavaFormatError("failed relay, insufficient results", processingError, utils.LogAttr("GUID", rp.guid))
}

Evidence

buildFailureResult explicitly logs via LogCodedError and then logs again via LavaFormatError;
LogCodedError itself logs by calling utils.LavaFormatError internally (and emits the metric).

protocol/relaycore/relay_processor.go[833-869]
protocol/common/error_logging.go[65-88]
utils/lavalog.go[446-452]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`buildFailureResult` logs the same failure twice: once via `common.LogCodedError` and again via `utils.LavaFormatError`. This duplicates logs and can double-count `lava_errors_total`.

### Issue Context
`LogCodedError` already logs (via `utils.LavaFormatError`) and emits the error metric.

### Fix Focus Areas
- protocol/relaycore/relay_processor.go[833-869]
- protocol/common/error_logging.go[65-88]

### Suggested fix
Choose a single logging path:
- Prefer returning the `LogCodedError(...)` result directly (include GUID as an attribute there) and remove the subsequent `utils.LavaFormatError(...)` call.
 - i.e., replace the current `LogCodedError(...)` + `return ..., LavaFormatError(...)` with `return ..., LogCodedError(...)`.
Or, if you must keep `LavaFormatError` for legacy behavior:
- Remove the `LogCodedError` call and use `common.EmitErrorMetric(...)` to increment metrics without a second log.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Description

InitErrorMetrics ignores Prometheus registration errors except AlreadyRegisteredError, so if
registration fails for another reason the system silently loses lava_errors_total. The callback
still increments a counter that may not be registered in the default registry, making missing
metrics hard to detect.

Code

protocol/metrics/error_metrics.go[R24-31]

+		// Best-effort registration — if already registered (e.g., in tests), reuse.
+		if err := prometheus.Register(counter); err != nil {
+			if existing, ok := err.(prometheus.AlreadyRegisteredError); ok {
+				if reused, ok := existing.ExistingCollector.(*prometheus.CounterVec); ok {
+					counter = reused
+				}
+			}
+		}

Evidence

The code only handles AlreadyRegisteredError and drops all other registration failures on the floor,
without logging or failing fast.

protocol/metrics/error_metrics.go[15-45]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`InitErrorMetrics` silently ignores Prometheus registration failures (other than AlreadyRegistered), which can lead to missing `lava_errors_total` with no signal.

### Issue Context
This runs during server init and is foundational observability; silent failure makes production debugging harder.

### Fix Focus Areas
- protocol/metrics/error_metrics.go[24-31]

### Suggested fix
On `prometheus.Register(counter)` error:
- If `AlreadyRegisteredError`, reuse the existing collector (as today).
- Otherwise, log and/or fail fast (panic/return error) so startup makes the issue visible.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Description

DetectConnectionError’s net.OpError errno path uses errors.As into *syscall.Errno, which only
matches when the underlying error is a pointer to Errno. This can miss common syscall error shapes
and makes Layer-3 detection unreliable unless the string-fallback happens to match.

Code

protocol/common/error_classifier.go[R518-531]

+	// Layer 3: net.OpError with a raw syscall errno.
+	var opErr *net.OpError
+	if errors.As(err, &opErr) {
+		var syscallErr *syscall.Errno
+		if errors.As(opErr.Err, &syscallErr) {
+			switch *syscallErr {
+			case syscall.ECONNREFUSED:
+				return LavaErrorConnectionRefused
+			case syscall.ECONNRESET:
+				return LavaErrorConnectionReset
+			case syscall.ENETUNREACH, syscall.EHOSTUNREACH:
+				return LavaErrorNetworkUnreachable
+			}
+		}

Evidence

The code only attempts to extract *syscall.Errno, and tests construct pointer-shaped errno values
(e.g., Err: &errno) to exercise this path; other errno shapes won’t be covered by Layer-3 and will
depend on brittle string matching.

protocol/common/error_classifier.go[518-533]
protocol/rpcsmartrouter/error_mapper_test.go[21-38]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
Layer-3 connection detection in `DetectConnectionError` only extracts `*syscall.Errno`. This is unnecessarily strict and can miss syscall errors that aren’t pointer-shaped.

### Issue Context
This path is intended to be a structured fallback after string matching; it should be robust across typical Go errno wrapping shapes.

### Fix Focus Areas
- protocol/common/error_classifier.go[518-533]

### Suggested fix
Update the errno extraction to handle both pointer and value forms, e.g.:
- Try `var errno syscall.Errno; errors.As(opErr.Err, &errno)` and switch on `errno`.
- Optionally also keep a pointer-form check for compatibility.
Add/adjust a unit test that passes a `net.OpError{Err: syscall.ECONNREFUSED}` (non-pointer) and asserts it’s detected.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.