[Serve] Using RAY_SERVE_REQUEST_PATH_LOG_BUFFER_SIZE reuses replica context across logs #55851
Description
What happened + What you expected to happen
When using buffered logs, serves replica context will get reused across logs
. As a result multiple log lines will have the same request_id.Only effects file handler logs because memoery buffer is only used on file handler.
This doesn't effect access logs in latest master because access logs now dont depend on ServeContextFilter.
But this could still effect other logs.
Versions / Dependencies
2.49 -> does not effect access logs
2.48 -> effects all logs
Reproduction script
from fastapi import FastAPI
from starlette.responses import StreamingResponse
from ray import serve
from ray.serve.api import get_deployment_handle
@serve.deployment(max_ongoing_requests=10000)
class ChildDeployment:
async def __call__(self):
return "Hello, world!"
@serve.deployment(max_ongoing_requests=10000)
class MyDeployment:
def __init__(self, child):
self.child = child
async def __call__(self):
return await self.child.remote()
app = MyDeployment.bind(ChildDeployment.bind())
config.yaml
# This file was generated using the `serve build` command on Ray v3.0.0.dev0.
proxy_location: EveryNode
http_options:
host: 0.0.0.0
port: 8000
grpc_options:
port: 9000
grpc_servicer_functions: []
logging_config:
encoding: JSON
log_level: INFO
logs_dir: null
enable_access_log: true
additional_log_standard_attrs: []
applications:
- name: app1
route_prefix: /
import_path: app:app
deployments:
- name: ChildDeployment
max_ongoing_requests: 1000
- name: MyDeployment
max_ongoing_requests: 1000
run using RAY_SERVE_REQUEST_PATH_LOG_BUFFER_SIZE=1000 serve run config.yaml
Issue Severity
None