The test helper check_autoscaling_metrics_include_prometheus and the associated tests are not strong enough. They only check for the presence of the metric key in the results, not the correctness of its value. Because the implementation falls back to 0.0 on any error (including metric not found), the tests will pass even if the metric fetching logic is broken.

To make the tests more robust, they should be updated to:

1. Use a simple, predictable metric for testing (e.g., a gauge that you can set manually).
2. Assert that the fetched metric value is correct, not just that the key exists.

This will help catch bugs like the incorrect handling of histogram metrics.

Bug: Hardcoded Prometheus Host; needs Configurable Override

The method _get_prometheus_metrics hardcodes "localhost:9090" as the Prometheus server address when calling self._prometheus_query_func("localhost:9090", metric_name). This is inconsistent with how the replica code handles this in replica.py (line 462), which uses the configurable RAY_SERVE_REPLICA_AUTOSCALING_METRIC_PROMETHEUS_HOST environment variable. The hardcoded value should be replaced with a configurable host parameter or environment variable to ensure consistency and allow for flexible Prometheus server configurations.

Bug: Broken Dependency Injection Prevents Mocking

The _fetch_prometheus_metrics method calls the module-level prometheus_handler function directly instead of using self._prometheus_handler. This breaks the dependency injection pattern established in the constructor and the set_prometheus_handler method, preventing test code from mocking the Prometheus handler as intended. The call should use self._prometheus_handler(...) instead of prometheus_handler(...).

Bug: Mocked Handler Ignored, Breaking Tests

The _fetch_prometheus_metrics method calls prometheus_handler instead of self._prometheus_handler. This bypasses the instance variable that can be set via set_prometheus_handler for testing, causing the method to always use the module-level import rather than the potentially mocked handler. This breaks the testing infrastructure where set_prometheus_handler is used to inject mock implementations.

Bug: Replica init fails: Unhandled parameter.

The ReplicaBase.__init__ method doesn't accept a prometheus_handler parameter, but line 647 attempts to pass it to create_replica_metrics_manager. When create_replica_impl(**kwargs) is called with prometheus_handler in kwargs, it will fail because ReplicaBase.__init__ doesn't accept this parameter. The prometheus_handler parameter needs to be added to ReplicaBase.__init__'s signature and stored so it can be passed to create_replica_metrics_manager.

Bug: Prometheus Handler Bypass Breaks Testability

The _fetch_prometheus_metrics method calls prometheus_handler directly instead of self._prometheus_handler. This bypasses the instance variable that can be set via set_prometheus_handler for testing purposes, making it impossible to inject mock handlers. The call should use self._prometheus_handler to respect the configured handler.

Bug: Prometheus Tests Hit Real Server

The _fetch_prometheus_metrics method calls prometheus_handler instead of self._prometheus_handler, which prevents the injected test handler from being used. This breaks the testing infrastructure where set_prometheus_handler is used to inject mock handlers, causing tests to call the actual Prometheus server instead of the mock.

Bug: Prometheus Handler: Mocked Logic Bypassed

The _fetch_prometheus_metrics method uses prometheus_handler (which is fetch_from_prom_server) but passes self._prometheus_handler as a parameter during initialization. However, when calling it on line 474, the code uses the global prometheus_handler directly instead of self._prometheus_handler. This means the injected mock handler for testing won't be used, breaking the testing infrastructure. The call should use self._prometheus_handler instead of the global prometheus_handler.

Bug: Prometheus Address Inconsistency Breaks Metrics

The Prometheus server address is hardcoded to "localhost:9090" in _get_prometheus_metrics, but replicas use the configurable RAY_SERVE_REPLICA_AUTOSCALING_METRIC_PROMETHEUS_HOST environment variable. This inconsistency means the controller and replicas may query different Prometheus servers, causing metrics collection to fail or return incorrect data. The controller should use the same configurable address as replicas.

Bug: Prometheus Handler: Module Variable Attribute Error

The test attempts to set ray.serve._private.replica.prometheus_handler but this module-level variable doesn't exist in replica.py. The module only imports default_prometheus_handler from ray._common.prometheus_utils and uses it as a default parameter. This will cause an AttributeError or create a new module attribute that won't affect the actual code behavior since the default parameter value is evaluated at function definition time, not at call time.

Bug: Redundant HTTP Prefix Malforms Prometheus URL

The environment variable RAY_SERVE_REPLICA_AUTOSCALING_METRIC_PROMETHEUS_HOST is set to "http://localhost:9090" which includes the http:// prefix. However, fetch_from_prom_server in prometheus_utils.py expects just the hostname:port and prepends http:// itself, resulting in a malformed URL http://http://localhost:9090/api/v1/query that will cause requests to fail.

Bug: URL Prefix Duplication Breaks Prometheus Calls

The environment variable RAY_SERVE_REPLICA_AUTOSCALING_METRIC_PROMETHEUS_HOST is set to "http://localhost:9090" which includes the http:// prefix. However, fetch_from_prom_server in prometheus_utils.py expects just the hostname:port and prepends http:// itself, resulting in a malformed URL http://http://localhost:9090/api/v1/query that will cause requests to fail.

Bug: Double HTTP prefix in Prometheus URL construction

The environment variable RAY_SERVE_REPLICA_AUTOSCALING_METRIC_PROMETHEUS_HOST is set to "http://localhost:9090" in the test configuration, but the fetch_from_prom_server function in prometheus_utils.py already prepends "http://" to the address parameter when constructing the URL. This results in a malformed URL like "http://http://localhost:9090/api/v1/query" with a double HTTP prefix, which will cause Prometheus queries to fail. The environment variable should be set to just "localhost:9090" without the protocol prefix.