Skip to content

[Feature] Further support for Lora Radix Cache #9144

@Fridge003

Description

@Fridge003

Checklist

Motivation

In #7216, a customized Lora radix cache is implemented to support lora serving. However, it only includes basic features of radix cache. The following advanced features are still not supported:

  • Page Size > 1
  • Schedule policies other than fcfs
  • Hierarchical cache

This issue serves as a tracker for these advanced supports.

Related resources

In #10317, an extra key is appended to original radix tree key, which can be applied to lora_id. If #10317 is merged, then all the cases mentioned above are automatically supported.

No response

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions