Points: 3 days ddl@6.14

Description: Establish the foundational architecture for request handling in the serving layer.

Deliverables:

• Create a serving_engine.py module with common serving functionality:

  • Error response creation (standardized error formats)
  • Request field validation helpers
  • Common utilities for request preprocessing and postprocessing (if any)

• Define protocol models for OpenAI-compatible API, copy protocol.py from the current openai_api directory.

• Add utils.py with helper functions

Testing:

• Create test_serving_engine.py to test:

  • Error response formatting
  • Field validation logic
  • Proper handling of edge cases

• Integration tests for the serving layer components