Skip to content

api: adds cost specifier to RateLimitRule#4957

Merged
arkodg merged 14 commits intoenvoyproxy:mainfrom
mathetake:applyonstreamdone
Jan 9, 2025
Merged

api: adds cost specifier to RateLimitRule#4957
arkodg merged 14 commits intoenvoyproxy:mainfrom
mathetake:applyonstreamdone

Conversation

@mathetake
Copy link
Copy Markdown
Member

@mathetake mathetake commented Dec 20, 2024

What type of PR is this?

This adds a new API into rate limit API.

What this PR does / why we need it:

This is for #4756. Basically, this adds the API to specify the hits_addend for the rate limit rule.
Especially, configuring the hits_addend in the response_path allows us to "reduce" the counter
based on the response content that affects the subsequent requests. This will enable the "token"
based request limits that are required fro AI gateway.

This is based on the Envoy's brand new features called apply_on_stream_done
and per-descriptor level hits_addend configuration introduced in

Which issue(s) this PR fixes:
#4756

Release Notes: No (until the implementation is done)

@mathetake mathetake requested a review from a team as a code owner December 20, 2024 06:42
@codecov
Copy link
Copy Markdown

codecov bot commented Dec 20, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 66.69%. Comparing base (5427854) to head (0d506e2).
Report is 3 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #4957      +/-   ##
==========================================
- Coverage   66.74%   66.69%   -0.06%     
==========================================
  Files         209      209              
  Lines       32264    32264              
==========================================
- Hits        21535    21518      -17     
- Misses       9432     9445      +13     
- Partials     1297     1301       +4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@mathetake
Copy link
Copy Markdown
Member Author

feel like maybe calling it "weight" instead of usage maybe be better? idk

@mathetake
Copy link
Copy Markdown
Member Author

will add the cel validation this afternoon (in japan)

@mathetake
Copy link
Copy Markdown
Member Author

maybe simply Usage->HitAddends would be consistent

@mathetake mathetake changed the title api: usage based rate limit API support api: adds hits_addend specifier to global rate limit API Dec 23, 2024
@mathetake mathetake changed the title api: adds hits_addend specifier to global rate limit API api: adds hits_addend specifier to RateLimitRule Dec 23, 2024
@mathetake
Copy link
Copy Markdown
Member Author

@zirain @zhaohuabing ptal - I removed the implementation-hide as envoyproxy/ratelimit#802 got merged

@mathetake
Copy link
Copy Markdown
Member Author

i will do the implementation in a subsequent PR

@zhaohuabing
Copy link
Copy Markdown
Member

zhaohuabing commented Dec 27, 2024

@zirain @zhaohuabing ptal - I removed the implementation-hide as envoyproxy/ratelimit#802 got merged

@mathetake the implementation comment is used to hide the unimplemented EG API in the EG docs, we normally hide the API at the API PR and unhide it at the implementation PR.

zhaohuabing
zhaohuabing previously approved these changes Dec 27, 2024
Copy link
Copy Markdown
Member

@zhaohuabing zhaohuabing left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM thanks!
Prefer to hide the API in the docs as it's not implemented yet.

@mathetake
Copy link
Copy Markdown
Member Author

ah ok - will do

@mathetake
Copy link
Copy Markdown
Member Author

@zhaohuabing hid the APIs!

Copy link
Copy Markdown
Member

@zirain zirain left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, hold for a while as @arkodg may want to take a look.

zhaohuabing
zhaohuabing previously approved these changes Dec 27, 2024
Copy link
Copy Markdown
Member

@zhaohuabing zhaohuabing left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM defer to @arkodg

@mathetake
Copy link
Copy Markdown
Member Author

👍

Signed-off-by: Takeshi Yoneda <t.y.mathetake@gmail.com>
Signed-off-by: Takeshi Yoneda <t.y.mathetake@gmail.com>
Signed-off-by: Takeshi Yoneda <t.y.mathetake@gmail.com>
Signed-off-by: Takeshi Yoneda <t.y.mathetake@gmail.com>
Signed-off-by: Takeshi Yoneda <t.y.mathetake@gmail.com>
Signed-off-by: Takeshi Yoneda <t.y.mathetake@gmail.com>
Signed-off-by: Takeshi Yoneda <t.y.mathetake@gmail.com>
Signed-off-by: Takeshi Yoneda <t.y.mathetake@gmail.com>
Signed-off-by: Takeshi Yoneda <t.y.mathetake@gmail.com>
Signed-off-by: Takeshi Yoneda <t.y.mathetake@gmail.com>
Signed-off-by: Takeshi Yoneda <t.y.mathetake@gmail.com>
Signed-off-by: Takeshi Yoneda <t.y.mathetake@gmail.com>
Signed-off-by: Takeshi Yoneda <t.y.mathetake@gmail.com>
@zirain zirain force-pushed the applyonstreamdone branch from a458c84 to 57290ad Compare January 9, 2025 03:52
@mathetake
Copy link
Copy Markdown
Member Author

mathetake commented Jan 9, 2025

is this flake or what

@zirain
Copy link
Copy Markdown
Member

zirain commented Jan 9, 2025

/retest

@mathetake
Copy link
Copy Markdown
Member Author

sorry the e2e tests logs are hard to grok ... will take a look

Signed-off-by: Takeshi Yoneda <t.y.mathetake@gmail.com>
@mathetake
Copy link
Copy Markdown
Member Author

@zirain passed now

@@ -91,6 +92,91 @@ type RateLimitRule struct {
// 429 HTTP status code is sent back to the client when
// the selected requests have reached the limit.
Limit RateLimitValue `json:"limit"`
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems like it is still required to specify the number of requests limit, how we can handle the case when we only care of the cost not the number of the request limit?

Copy link
Copy Markdown
Member Author

@mathetake mathetake Jan 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok so first of all, this limit is nothing to do with that purpose. And to do the limit purely on the token number (in this context response cost only), you can simply set the cost.response = {from: Number, Number: 0} as per the comment - zero can be used to "only check the budget and if not left anything, then reject".

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also with this new cost field, limit.requests doesnt quite fit anymore, something like limit.total feels better and can be an alias to limit.requests in the future to improve defining intent

@arkodg arkodg merged commit 70e8b66 into envoyproxy:main Jan 9, 2025
@mathetake mathetake deleted the applyonstreamdone branch January 11, 2025 18:38
yuzisun pushed a commit to envoyproxy/ai-gateway that referenced this pull request Jan 18, 2025
This adds the RequestCost field to AIGatewayRoute,
which will allows users to do the rate limiting etc
based on the calculated "token usage". 

This is based on the new feature introduced in 
* envoyproxy/gateway#4957
* envoyproxy/gateway#5035

and because of the feature, the only thing we have to do
from AI Gateway side is to set a dynamic metadata as per 
the comment in the API.

---------

Signed-off-by: Takeshi Yoneda <t.y.mathetake@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants