Summary

WasmEdge is a lightweight inference runtime for AI and LLM applications. We want to build specialized and finetuned models for WasmEdge community. The model should be supported by WasmEdge and its applications should benefit the WasmEdge community.

In this project, we will build and compare two finetuned model for Rust coding assistance.

• A code review model. It aims to be a new backend for the PR review bot we currently use in the community.
• A QA model. It should be able to answer user questions about the Rust language and provide explanations. Our goal is to provide an alternative to our Learn Rust app.

Details

Objective 1: Code review model

Create a dataset with the following two fields

We are looking for at least 200 Q and A pairs. The total length of each QA should be less than 3000 words.

Q: a code segment
A: explanation / review of the code

The QA could come from Rust documentation such as Rust by Example and The Rust Programming Language.

Assemble the dataset into the llama3 chat template

It is similar to the following. Each entry should be all in one line with linebreaks denoted as \n.

<|begin_of_text|><|start_header_id|>system<|end_header_id|>

You are a reviewer of Rust source code.<|eot_id|><|start_header_id|>user<|end_header_id|>

{{ a code segment }}<|eot_id|><|start_header_id|>assistant<|end_header_id|>

{{ explanation / review of the code }}<|eot_id|>

Finetune

we will finetune based on the llama3-8b-instruct model.

You are free to use any finetune tools. But if you are unsure, we recommend using llama.cpp's finetune utility. See an example. It can run on CPUs. We will provide the computing resources required for the finetuing.

Objective 2: Code QA model

Create a dataset with the following three fields

We are looking for at least 100 chapter + Q + A rows.

C: A chapter from a Rust book
Q: A question related to the chapter
A: Explanation / answer for the question

You could use ChatGPT to generate these questions and answers based on the chapter content.

Assemble the dataset into the llama3 chat template

It is similar to the following. Each entry should be all in one line with linebreaks denoted as \n.

<|begin_of_text|><|start_header_id|>system<|end_header_id|>

You are an expert of the Rust language. Please answer question based on the context below.\n---------\n{{ book chapter }}<|eot_id|><|start_header_id|>user<|end_header_id|>

{{ a code segment }}<|eot_id|><|start_header_id|>assistant<|end_header_id|>

{{ explanation / review of the code }}<|eot_id|>

Finetune

Due to the chapter-long context length used in this dataset, we will finetune based on the 262k long context length llama3-8b-instruct model.

You are free to use any finetune tools. But if you are unsure, we recommend using llama.cpp's finetune utility. See an example. It can run on CPUs. We will provide the computing resources required for the finetuing.

Objective 3: Compare the two finetuned models

Start the finetuned models using the LlamaEdge API server, and test them on commonly used scenarios.

LFX

Expected outcome: Two finetuned models based on Llama3-8b for Rust code review and QA.

Recommended skills:

• Rust language
• ChatGPT and Claude
• LlamaEdge
• llama.cpp

Mentor:

• Michael Yuan @juntao michael@secondstate.io

Application link: ttps://mentorship.lfx.linuxfoundation.org/project/d52d172e-598d-4817-be97-3338d5b96432

Appendix

• Rust by Example
• The Rust Programming Language