Add autoscaling metrics aggregation function by abrarsheikh · Pull Request #56871 · ray-project/ray

abrarsheikh · 2025-09-24T02:07:15Z

Add support for min, max and time weighted average as aggregation function over timeseries data

Signed-off-by: abrar <abrar@anyscale.com>

gemini-code-assist

Code Review

This pull request introduces a valuable feature by allowing the autoscaling metrics aggregation function to be configured. Users can now choose between 'mean', 'max', and 'min' aggregation, providing more flexibility for different use cases. The implementation is solid, with the introduction of a new AggregationFunction enum, an aggregate_timeseries utility, and updates to the AutoscalingConfig. The changes are also well-covered by new and updated tests. I have a few minor suggestions to improve the docstring, memory efficiency, and exception handling.

python/ray/serve/_private/metrics_utils.py

python/ray/serve/config.py

Signed-off-by: abrar <abrar@anyscale.com>

arcyleung · 2025-09-24T17:04:33Z

python/ray/serve/_private/metrics_utils.py

+def aggregate_timeseries(
+    timeseries: List[TimeStampedValue],
+    aggregation_function: AggregationFunction,
+    last_window_s: float = 1.0,
+) -> Optional[float]:
+    """Aggregate the values in a timeseries using a specified function."""
+    if aggregation_function == AggregationFunction.MEAN:
+        return time_weighted_average(timeseries, last_window_s=last_window_s)
+    elif aggregation_function == AggregationFunction.MAX:
+        return max(ts.value for ts in timeseries) if timeseries else None
+    elif aggregation_function == AggregationFunction.MIN:
+        return min(ts.value for ts in timeseries) if timeseries else None
+    else:
+        raise ValueError(f"Invalid aggregation function: {aggregation_function}")


Suggested change

def aggregate_timeseries(

timeseries: List[TimeStampedValue],

aggregation_function: AggregationFunction,

last_window_s: float = 1.0,

) -> Optional[float]:

"""Aggregate the values in a timeseries using a specified function."""

if aggregation_function == AggregationFunction.MEAN:

return time_weighted_average(timeseries, last_window_s=last_window_s)

elif aggregation_function == AggregationFunction.MAX:

return max(ts.value for ts in timeseries) if timeseries else None

elif aggregation_function == AggregationFunction.MIN:

return min(ts.value for ts in timeseries) if timeseries else None

else:

raise ValueError(f"Invalid aggregation function: {aggregation_function}")

def aggregate_timeseries(

timeseries: List[TimeStampedValue],

aggregation_function: AggregationFunction,

last_window_s: float = 1.0,

) -> Optional[float]:

"""Aggregate the values in a timeseries using a specified function."""

if not timeseries:

return None

if aggregation_function == AggregationFunction.MEAN:

return time_weighted_average(timeseries, last_window_s=last_window_s)

elif aggregation_function == AggregationFunction.MAX:

# maybe cleaner to refactor to windowed min/max function, the same way _get_datapoints is doing it

window_start = datetime.now() - last_window_s

return max(ts.value for ts in timeseries if ts.timestamp > window_start )

elif aggregation_function == AggregationFunction.MIN:

window_start = datetime.now() - last_window_s

return min(ts.value for ts in timeseries if ts.timestamp > window_start )

else:

raise ValueError(f"Invalid aggregation function: {aggregation_function}")

i dont understand this suggestion can you elaborate what this does

window_start = datetime.now() - last_window_s return max(ts.value for ts in timeseries if ts.timestamp > window_start )

landscapepainter

LGTM! By the way, do we need to add any tests that check how the autoscaling behavior changes based on used aggregation_function value? I assume the number of scaled replicas can change based on aggregation_function either being min, max, or mean.

src/ray/protobuf/serve.proto

landscapepainter · 2025-09-30T04:41:44Z

python/ray/serve/_private/metrics_utils.py

+def aggregate_timeseries(
+    timeseries: List[TimeStampedValue],
+    aggregation_function: AggregationFunction,
+    last_window_s: float = 1.0,


Why do we set the default value of last_window_s to be 1.0? Was wondering the implication of it.

in reality we are not depending on default value. The value basically decided how much to weight the last data point. 1 is safe default.

Signed-off-by: abrar <abrar@anyscale.com>

akyang-anyscale

Should we support percentiles too?

abrarsheikh · 2025-10-02T19:52:07Z

Should we support percentiles too?

ok to leave it for the future?

akyang-anyscale · 2025-10-02T20:00:45Z

Should we support percentiles too?

ok to leave it for the future?

yeah that's fine

Signed-off-by: abrar <abrar@anyscale.com>

Add support for min, max and time weighted average as aggregation function over timeseries data --------- Signed-off-by: abrar <abrar@anyscale.com> Signed-off-by: Pavitra Bhalla <pavitra@rayai.com>

Add support for min, max and time weighted average as aggregation function over timeseries data --------- Signed-off-by: abrar <abrar@anyscale.com> Signed-off-by: Josh Kodi <joshkodi@gmail.com>

Add support for min, max and time weighted average as aggregation function over timeseries data --------- Signed-off-by: abrar <abrar@anyscale.com>

Add support for min, max and time weighted average as aggregation function over timeseries data --------- Signed-off-by: abrar <abrar@anyscale.com> Signed-off-by: xgui <xgui@anyscale.com>

Add support for min, max and time weighted average as aggregation function over timeseries data --------- Signed-off-by: abrar <abrar@anyscale.com> Signed-off-by: elliot-barn <elliot.barnwell@anyscale.com>

PR ray-project#56871 introduced the AggregationFunction enum for autoscaling metrics aggregation. However, the serve build command's YAML serialization path was not updated to handle Enum objects, causing RepresenterError when serializing AggregationFunction enum values in autoscaling config. This fix adds a helper function that recursively converts Enum objects to their string values before YAML dump, ensuring proper serialization of all enum types in the configuration. Fixes ray-project#58485 Signed-off-by: Nikhil Ghosh <nikhil@anyscale.com>

Add support for min, max and time weighted average as aggregation function over timeseries data --------- Signed-off-by: abrar <abrar@anyscale.com>

Add support for min, max and time weighted average as aggregation function over timeseries data --------- Signed-off-by: abrar <abrar@anyscale.com> Signed-off-by: Aydin Abiar <aydin@anyscale.com>

…on enum (#58509) PR #56871 introduced the AggregationFunction enum for autoscaling metrics aggregation. However, the serve build command's YAML serialization path was not updated to handle Enum objects, causing RepresenterError when serializing AggregationFunction enum values in autoscaling config. ex. `yaml.representer.RepresenterError: ('cannot represent an object', <AggregationFunction.MEAN: 'mean'>)` This fix adds 1. a helper function that recursively converts Enum objects to their string values before YAML dump, ensuring proper serialization of all enum types in the configuration. 2. a test that (a) Creates a temporary output YAML file (b) Reads the config from that file (c) Verifies that AggregationFunction.MEAN is correctly serialized as "mean" (string) Fixes #58485 --------- Signed-off-by: Nikhil Ghosh <nikhil@anyscale.com> Signed-off-by: Nikhil G <nrghosh@users.noreply.github.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

…on enum (ray-project#58509) PR ray-project#56871 introduced the AggregationFunction enum for autoscaling metrics aggregation. However, the serve build command's YAML serialization path was not updated to handle Enum objects, causing RepresenterError when serializing AggregationFunction enum values in autoscaling config. ex. `yaml.representer.RepresenterError: ('cannot represent an object', <AggregationFunction.MEAN: 'mean'>)` This fix adds 1. a helper function that recursively converts Enum objects to their string values before YAML dump, ensuring proper serialization of all enum types in the configuration. 2. a test that (a) Creates a temporary output YAML file (b) Reads the config from that file (c) Verifies that AggregationFunction.MEAN is correctly serialized as "mean" (string) Fixes ray-project#58485 --------- Signed-off-by: Nikhil Ghosh <nikhil@anyscale.com> Signed-off-by: Nikhil G <nrghosh@users.noreply.github.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

…on enum (ray-project#58509) PR ray-project#56871 introduced the AggregationFunction enum for autoscaling metrics aggregation. However, the serve build command's YAML serialization path was not updated to handle Enum objects, causing RepresenterError when serializing AggregationFunction enum values in autoscaling config. ex. `yaml.representer.RepresenterError: ('cannot represent an object', <AggregationFunction.MEAN: 'mean'>)` This fix adds 1. a helper function that recursively converts Enum objects to their string values before YAML dump, ensuring proper serialization of all enum types in the configuration. 2. a test that (a) Creates a temporary output YAML file (b) Reads the config from that file (c) Verifies that AggregationFunction.MEAN is correctly serialized as "mean" (string) Fixes ray-project#58485 --------- Signed-off-by: Nikhil Ghosh <nikhil@anyscale.com> Signed-off-by: Nikhil G <nrghosh@users.noreply.github.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Signed-off-by: YK <1811651+ykdojo@users.noreply.github.com>

…on enum (ray-project#58509) PR ray-project#56871 introduced the AggregationFunction enum for autoscaling metrics aggregation. However, the serve build command's YAML serialization path was not updated to handle Enum objects, causing RepresenterError when serializing AggregationFunction enum values in autoscaling config. ex. `yaml.representer.RepresenterError: ('cannot represent an object', <AggregationFunction.MEAN: 'mean'>)` This fix adds 1. a helper function that recursively converts Enum objects to their string values before YAML dump, ensuring proper serialization of all enum types in the configuration. 2. a test that (a) Creates a temporary output YAML file (b) Reads the config from that file (c) Verifies that AggregationFunction.MEAN is correctly serialized as "mean" (string) Fixes ray-project#58485 --------- Signed-off-by: Nikhil Ghosh <nikhil@anyscale.com> Signed-off-by: Nikhil G <nrghosh@users.noreply.github.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

Add support for min, max and time weighted average as aggregation function over timeseries data --------- Signed-off-by: abrar <abrar@anyscale.com> Signed-off-by: Future-Outlier <eric901201@gmail.com>

…on enum (ray-project#58509) PR ray-project#56871 introduced the AggregationFunction enum for autoscaling metrics aggregation. However, the serve build command's YAML serialization path was not updated to handle Enum objects, causing RepresenterError when serializing AggregationFunction enum values in autoscaling config. ex. `yaml.representer.RepresenterError: ('cannot represent an object', <AggregationFunction.MEAN: 'mean'>)` This fix adds 1. a helper function that recursively converts Enum objects to their string values before YAML dump, ensuring proper serialization of all enum types in the configuration. 2. a test that (a) Creates a temporary output YAML file (b) Reads the config from that file (c) Verifies that AggregationFunction.MEAN is correctly serialized as "mean" (string) Fixes ray-project#58485 --------- Signed-off-by: Nikhil Ghosh <nikhil@anyscale.com> Signed-off-by: Nikhil G <nrghosh@users.noreply.github.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Signed-off-by: peterxcli <peterxcli@gmail.com>

Add autoscaling metrics aggregation function

cbafa55

Signed-off-by: abrar <abrar@anyscale.com>

abrarsheikh requested a review from a team as a code owner September 24, 2025 02:07

abrarsheikh added the go add ONLY when ready to merge, run all tests label Sep 24, 2025

gemini-code-assist bot reviewed Sep 24, 2025

View reviewed changes

python/ray/serve/_private/metrics_utils.py Outdated Show resolved Hide resolved

python/ray/serve/_private/metrics_utils.py Outdated Show resolved Hide resolved

python/ray/serve/config.py Outdated Show resolved Hide resolved

change var name

64d9e9e

Signed-off-by: abrar <abrar@anyscale.com>

This comment was marked as outdated.

Sign in to view

abrarsheikh added 2 commits September 24, 2025 03:35

fix docs

700e792

Signed-off-by: abrar <abrar@anyscale.com>

fix tests

88cf995

Signed-off-by: abrar <abrar@anyscale.com>

abrarsheikh requested a review from a team as a code owner September 24, 2025 04:19

ray-gardener bot added serve Ray Serve Related Issue core Issues that should be addressed in Ray Core labels Sep 24, 2025

arcyleung reviewed Sep 24, 2025

View reviewed changes

landscapepainter approved these changes Sep 30, 2025

View reviewed changes

update doc

f568d4b

Signed-off-by: abrar <abrar@anyscale.com>

akyang-anyscale approved these changes Oct 2, 2025

View reviewed changes

abrarsheikh added 2 commits October 7, 2025 00:30

Merge branch 'master' into SERVE-1102-abrar-agg_p4

9700e9f

fix

ee5c6cf

Signed-off-by: abrar <abrar@anyscale.com>

zcin approved these changes Oct 9, 2025

View reviewed changes

zcin merged commit f07db71 into master Oct 9, 2025
6 checks passed

zcin deleted the SERVE-1102-abrar-agg_p4 branch October 9, 2025 23:21

harshit-anyscale pushed a commit that referenced this pull request Oct 15, 2025

Add autoscaling metrics aggregation function (#56871)

e9b6cf3

Add support for min, max and time weighted average as aggregation function over timeseries data --------- Signed-off-by: abrar <abrar@anyscale.com>

nrghosh mentioned this pull request Nov 10, 2025

[serve][llm] Fix serve build YAML serialization for AggregationFunction enum #58509

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add autoscaling metrics aggregation function#56871

Add autoscaling metrics aggregation function#56871
zcin merged 7 commits intomasterfrom
SERVE-1102-abrar-agg_p4

abrarsheikh commented Sep 24, 2025 •

edited

Loading

Uh oh!

gemini-code-assist bot left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This comment was marked as outdated.

Uh oh!

arcyleung Sep 24, 2025 •

edited

Loading

Uh oh!

abrarsheikh Oct 1, 2025

Uh oh!

landscapepainter left a comment

Uh oh!

Uh oh!

landscapepainter Sep 30, 2025

Uh oh!

abrarsheikh Oct 1, 2025

Uh oh!

akyang-anyscale left a comment

Uh oh!

abrarsheikh commented Oct 2, 2025

Uh oh!

akyang-anyscale commented Oct 2, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

abrarsheikh commented Sep 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This comment was marked as outdated.

Uh oh!

arcyleung Sep 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

abrarsheikh Oct 1, 2025

Choose a reason for hiding this comment

Uh oh!

landscapepainter left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

landscapepainter Sep 30, 2025

Choose a reason for hiding this comment

Uh oh!

abrarsheikh Oct 1, 2025

Choose a reason for hiding this comment

Uh oh!

akyang-anyscale left a comment

Choose a reason for hiding this comment

Uh oh!

abrarsheikh commented Oct 2, 2025

Uh oh!

akyang-anyscale commented Oct 2, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

abrarsheikh commented Sep 24, 2025 •

edited

Loading

arcyleung Sep 24, 2025 •

edited

Loading