Skip to content

fix: ensure arrival_time is set before calculating queue time#21918

Merged
Harshit28j merged 1 commit intoBerriAI:mainfrom
Harshit28j:litellm_arrival_time_fix
Feb 23, 2026
Merged

fix: ensure arrival_time is set before calculating queue time#21918
Harshit28j merged 1 commit intoBerriAI:mainfrom
Harshit28j:litellm_arrival_time_fix

Conversation

@Harshit28j
Copy link
Copy Markdown
Contributor

@Harshit28j Harshit28j commented Feb 23, 2026 •
edited
Loading

Relevant issues

Fixes the litellm_request_queue_time_seconds Prometheus metric which was previously reporting as 0/None because arrival_time was being accessed before it was initialized in the request metadata.

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have added testing in the tests/litellm/ directory (at least 1 test added)
    - Added test_queue_time_seconds_is_set_in_metadata in
    tests/test_litellm/proxy/test_common_request_processing.py
  • My PR passes all unit tests on make test-unit
  • My PR’s scope is isolated and solves only 1 specific problem
  • I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting maintainer review

CI (LiteLLM team)

  • Branch creation CI run
    Link:
  • CI run for the last commit
    Link:
  • Merge / cherry-pick CI run
    Links:

Type

🐛 Bug Fix
✅ Test

Changes

  • Logic Reordering: Modified litellm/proxy/common_request_processing.py to move the arrival_time initialization (via add_litellm_data_to_request) before the calculation of queue_time_seconds.
  • Dependency Addition: Added import time to common_request_processing.py to support high-precision timestamp calculations.
  • New Unit Test: Added a regression test in tests/test_litellm/proxy/test_common_request_processing.py that mocks the request flow and verifies queue_time_seconds is correctly populated and non-negative in the request metadata.
  • Formatting: Performed minor linting/formatting fixes in the touched files to align with project standards.

Verification:

Verified by sending a request via Postman to the proxy and then checking the /metrics endpoint. The metric litellm_request_queue_time_seconds_sum now correctly records the processing delay instead of being empty or zero

image

@vercel
Copy link
Copy Markdown

vercel Bot commented Feb 23, 2026 •
edited
Loading

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
litellm Ready Ready Preview, Comment Feb 23, 2026 11:48am

Request Review

@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented Feb 23, 2026

Greptile Summary

This PR fixes a bug where the litellm_request_queue_time_seconds Prometheus metric was always reporting as 0/None. The root cause was that arrival_time was being read from proxy_server_request before add_litellm_data_to_request() was called — the very function that populates arrival_time.

  • Core fix: Moved the queue_time_seconds calculation block in common_processing_pre_call_logic to after the add_litellm_data_to_request() call, so arrival_time is properly available when the calculation runs.
  • import time addition: The old code used start_time.timestamp() (from datetime) but was never actually reached due to the bug. The new code uses time.time(), which is consistent with how arrival_time is set in litellm_pre_call_utils.py:842.
  • Regression test: Added test_queue_time_seconds_is_set_in_metadata that mocks the request flow and verifies queue_time_seconds is correctly computed and stored.
  • Formatting: Minor whitespace/line-wrapping reformats applied by a linter — no functional changes beyond the fix.

Confidence Score: 5/5

  • This PR is safe to merge — it corrects a clear ordering bug with a minimal, well-tested change.
  • The fix is straightforward and correct: the queue time calculation was reading arrival_time before it was set, and now it reads it after. The time.time() call is consistent with how arrival_time is recorded. A regression test is included. The remaining changes are auto-formatter whitespace adjustments with no functional impact.
  • No files require special attention.

Important Files Changed
Filename Overview
litellm/proxy/common_request_processing.py Correct bug fix: moved queue_time_seconds calculation after add_litellm_data_to_request which sets arrival_time. Added import time at module level. Remaining changes are formatting-only (line wrapping, trailing whitespace).
tests/test_litellm/proxy/test_common_request_processing.py Added regression test for queue_time_seconds. Test uses proper mocking (no real network calls). Minor style issue: import time inside a mock function instead of at module level. Remaining changes are formatting-only.

Sequence Diagram

sequenceDiagram
    participant Client
    participant Proxy as common_processing_pre_call_logic
    participant Utils as add_litellm_data_to_request
    participant Metrics as Prometheus Metrics

    Client->>Proxy: HTTP Request
    Proxy->>Proxy: start_time = datetime.now()

    Note over Proxy: BEFORE FIX: tried to read arrival_time here (always None)

    Proxy->>Utils: add_litellm_data_to_request(data)
    Utils->>Utils: arrival_time = time.time()
    Utils->>Utils: data["proxy_server_request"]["arrival_time"] = arrival_time
    Utils-->>Proxy: return data (with arrival_time set)

    Note over Proxy: AFTER FIX: reads arrival_time here (correctly populated)
    Proxy->>Proxy: queue_time = time.time() - arrival_time
    Proxy->>Proxy: metadata["queue_time_seconds"] = queue_time

    Proxy->>Metrics: Record litellm_request_queue_time_seconds
Loading

Last reviewed commit: 9fc3c77

greptile-apps[bot]
greptile-apps Bot reviewed Feb 23, 2026
View reviewed changes
Copy link
Copy Markdown
Contributor

@greptile-apps greptile-apps Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2 files reviewed, 1 comment

Edit Code Review Agent Settings | Greptile

Comment thread tests/test_litellm/proxy/test_common_request_processing.py

async def mock_add_litellm_data_to_request(*args, **kwargs):
data = kwargs.get("data", args[0] if args else {})
# Simulate what add_litellm_data_to_request does: set arrival_time
Copy link
Copy Markdown
Contributor

@greptile-apps greptile-apps Bot Feb 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Move import time to module level

Per project guidelines (CLAUDE.md: "Avoid imports within methods — place all imports at the top of the file"), import time should be at the top of the file rather than inside this mock function.

Suggested change
# Simulate what add_litellm_data_to_request does: set arrival_time

Context Used: Context from dashboard - CLAUDE.md (source)

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

@Harshit28j Harshit28j merged commit 4c487ac into BerriAI:main Feb 23, 2026
30 of 31 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer

@greptile-apps greptile-apps[bot] greptile-apps[bot] left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@Harshit28j