Skip to content

merging trained lora model from unsloth into original mistralai/MistralAI-7B-v0.1 for vllm serving #95

@DoctorSlimm

Description

@DoctorSlimm

Understanding the need to serve efficiently, I was wondering if anybody here has experience with merging the mistral model trained with unsloth using PEFT back into the original mistral ai model for compatibility and serving via vllm framework?

Best Regards, Lukas

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions