• Convert latency metrics from Gauge to HistogramVec for better distribution analysis
• Add apiInterface label to block fetch metrics and latest block tracking
• Merge relay/errored counters into unified metrics with function labels
• Remove consumer QoS metrics from provider side (lava_consumer_QoS)
• Simplify metric initialization and constructor signatures
• Add comprehensive provider metrics documentation with PromQL examples

Convert latency metrics to histograms and simplify structure

• Convert providerLatencyMetric and providerEndToEndLatencyMetric from GaugeVec to HistogramVec
• Convert requestLatencyPerFunctionMetric from MappedLabelsGaugeVec to HistogramVec
• Change latency recording from Set() to Observe() method calls
• Remove totalRequestsPerFunctionMetric and totalErrorsPerFunctionMetric fields
• Remove consumerQoSMetric field and all QoS-related code from AddRelay()
• Merge error tracking: AddFunctionError() now increments both totalRelaysServicedMetric and
 totalErroredMetric
• Simplify NewProviderMetrics() constructor signature by removing 4 parameters
• Remove unnecessary mutex locks from latency methods

protocol/metrics/provider_metrics.go

Add apiInterface labels and convert to histogram metrics

• Add apiInterface parameter to block fetch metric methods: SetLatestBlockFetchError(),
 SetSpecificBlockFetchError(), SetLatestBlockFetchSuccess(), SetSpecificBlockFetchSuccess()
• Convert providerLatencyMetric and providerEndToEndLatencyMetric from GaugeVec to HistogramVec
 with predefined buckets
• Create requestLatencyPerFunctionMetric as HistogramVec instead of MappedLabelsGaugeVec
• Remove totalRequestsPerFunctionMetric, totalErrorsPerFunctionMetric, and consumerQoSMetric
 fields
• Rename lava_latest_block to lava_provider_latest_block and add apiInterface label
• Add apiInterface label to all block fetch metrics
• Update SetLatestBlock() to accept apiInterface parameter
• Remove AddApiMethodCallsMetrics constant
• Rename virtual_epoch to lava_provider_virtual_epoch
• Update metric registration and manager initialization

protocol/metrics/provider_metrics_manager.go

Pass apiInterface to block fetch metrics

• Add cs.endpoint.ApiInterface parameter to all four block fetch metric calls
• Update SetLatestBlockFetchError(), SetLatestBlockFetchSuccess(),
 SetSpecificBlockFetchError(), and SetSpecificBlockFetchSuccess() invocations

protocol/chaintracker/chain_tracker.go

Description

Provider histogram metrics are created with Buckets: LatencyBuckets, but LatencyBuckets is not
defined in the metrics package, causing compilation to fail. This blocks building any binary that
imports protocol/metrics.

Code

protocol/metrics/provider_metrics_manager.go[R90-94]

+	requestLatencyPerFunctionMetric := prometheus.NewHistogramVec(prometheus.HistogramOpts{
+		Name:    "lava_provider_request_latency_milliseconds",
+		Help:    "Distribution of relay latency per function in milliseconds.",
+		Buckets: LatencyBuckets,
+	}, []string{"spec", "apiInterface", "function"})

Evidence

NewProviderMetricsManager uses LatencyBuckets for HistogramVec bucket configuration, but the
metrics package has no LatencyBuckets definition (only defaultLatencyBuckets exists elsewhere).
This results in an undefined identifier at compile time.

protocol/metrics/provider_metrics_manager.go[90-94]
protocol/metrics/provider_metrics_manager.go[149-159]
protocol/metrics/smartrouter_metrics_manager.go[19-22]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`protocol/metrics/provider_metrics_manager.go` references `LatencyBuckets` when creating histograms, but `LatencyBuckets` is not defined in the `metrics` package.
## Issue Context
There is already an in-package bucket list (`defaultLatencyBuckets`) used by smart router metrics. Provider metrics should use a defined bucket set as well (ideally shared).
## Fix Focus Areas
- protocol/metrics/provider_metrics_manager.go[90-94]
- protocol/metrics/provider_metrics_manager.go[149-159]
- protocol/metrics/smartrouter_metrics_manager.go[19-22]
## Suggested fix
- Add a package-level exported var/const in `protocol/metrics` (e.g., in a new file `protocol/metrics/latency_buckets.go`):
- `var LatencyBuckets = []float64{1,2,5,10,25,50,100,250,500,1000,2500,5000,10000,30000}`
- Optionally refactor smart router to reuse `LatencyBuckets` instead of maintaining `defaultLatencyBuckets` (to keep docs/code aligned).

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Description

metrics.AddApiMethodCallsMetrics was removed from the metrics package constants, but rpcconsumer
and rpcsmartrouter still reference it for flag binding and viper lookups. This causes compilation
failures for those binaries.

Code

protocol/metrics/provider_metrics_manager.go[18]

-	AddApiMethodCallsMetrics      = "add-api-method-metrics"

Evidence

The metrics package const block no longer defines AddApiMethodCallsMetrics, yet both rpcconsumer
and rpcsmartrouter still call viper.GetBool(metrics.AddApiMethodCallsMetrics) and register a flag
using that constant, producing an undefined identifier error at build time.

protocol/metrics/provider_metrics_manager.go[16-27]
protocol/rpcconsumer/rpcconsumer.go[624-726]
protocol/rpcsmartrouter/rpcsmartrouter.go[1322-1433]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The exported constant `AddApiMethodCallsMetrics` was removed from the `metrics` package, but it is still referenced by `rpcconsumer` and `rpcsmartrouter`.
## Issue Context
Both binaries use `metrics.AddApiMethodCallsMetrics` as the flag name and for viper reads; removing it breaks compilation.
## Fix Focus Areas
- protocol/metrics/provider_metrics_manager.go[16-27]
- protocol/rpcconsumer/rpcconsumer.go[624-726]
- protocol/rpcsmartrouter/rpcsmartrouter.go[1322-1433]
## Suggested fix
- Re-introduce `AddApiMethodCallsMetrics` as an exported constant in the `metrics` package (e.g., in `provider_metrics_manager.go` const block or a new shared `protocol/metrics/flags.go`).
- Keep the flag name string stable (the previous value in this repo was `"add-api-method-metrics"`).

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Description

AvailabilityLabel, SyncLabel, and LatencyLabel were removed from the metrics package, but
ConsumerMetricsManager.SetQOSMetrics still uses them when setting QoS metrics labels. This causes
compilation errors in the metrics package itself.

Code

protocol/metrics/provider_metrics.go[L12-14]

-	AvailabilityLabel = "availability"
-	SyncLabel         = "sync/freshness"
-	LatencyLabel      = "latency"

Evidence

provider_metrics.go no longer defines the AvailabilityLabel/SyncLabel/LatencyLabel
constants, but consumer_metrics_manager.go references them directly in the same metrics package.
Go compilation fails with undefined identifiers.

protocol/metrics/provider_metrics.go[11-18]
protocol/metrics/consumer_metrics_manager.go[709-736]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`AvailabilityLabel`, `SyncLabel`, and `LatencyLabel` constants were removed from the `metrics` package, but are still used by consumer-side metrics code in `ConsumerMetricsManager.SetQOSMetrics`.
## Issue Context
These constants are used as label values for the consumer QoS metrics (`qos_metric` label). Removing them breaks compilation.
## Fix Focus Areas
- protocol/metrics/provider_metrics.go[11-18]
- protocol/metrics/consumer_metrics_manager.go[709-736]
## Suggested fix
- Re-add the constants in the `metrics` package (preferably in a consumer-oriented constants file to avoid future provider cleanups removing them):
- `AvailabilityLabel = "availability"`
- `SyncLabel = "sync/freshness"` (note: this differs from `SelectionSyncLabel`)
- `LatencyLabel = "latency"`
- Keep `Selection*Label` constants as-is for selection stats metrics.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.