-
Notifications
You must be signed in to change notification settings - Fork 4.5k
Closed
Labels
.NETIssue or Pull requests regarding .NET codeIssue or Pull requests regarding .NET codebugSomething isn't workingSomething isn't working
Description
Describe the bug
When trying to use semantic kernel with llama cpp server, i am running into an error with function calling. The plugin is executed as expected but it looks like some step with adding the plugin response to the chat history is failing.
To Reproduce
Steps to reproduce the behavior:
Project File (csproj)
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>net9.0</TargetFramework>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Microsoft.SemanticKernel" Version="1.40.1" />
<PackageReference Include="Microsoft.SemanticKernel.Agents.Core" Version="1.40.1-preview" />
<PackageReference Include="Microsoft.SemanticKernel.Connectors.OpenAI" Version="1.40.1" />
</ItemGroup>
</Project>
Program.cs
using System.ComponentModel;
using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.Agents;
using Microsoft.SemanticKernel.ChatCompletion;
using Microsoft.SemanticKernel.Connectors.OpenAI;
namespace FunctionError;
#pragma warning disable SKEXP0001
#pragma warning disable SKEXP0010
class Program
{
static async Task Main(string[] args)
{
var builder = Kernel.CreateBuilder();
builder = builder.AddOpenAIChatCompletion(modelId: "any-name", apiKey: null, endpoint: new Uri("http://localhost:8000/v1"));
var kernel = builder.Build();
var chatCompletionAgent = new ChatCompletionAgent
{
Name = "FunctionCaller",
Instructions = "You call functions",
Kernel = kernel,
Arguments = new KernelArguments (
new OpenAIPromptExecutionSettings()
{
FunctionChoiceBehavior = FunctionChoiceBehavior.Auto(),
}
),
};
chatCompletionAgent.Kernel.Plugins.AddFromType<CurrenTimePlugin>();
ChatHistory chat = [];
await InvokeAgentAsync("What is the current time?");
async Task InvokeAgentAsync(string input)
{
ChatMessageContent message = new(AuthorRole.User, input);
chat.Add(message);
WriteAgentChatMessage(message);
await foreach (ChatMessageContent response in chatCompletionAgent.InvokeAsync(chat))
{
chat.Add(response);
WriteAgentChatMessage(response);
}
}
}
private static void WriteAgentChatMessage(ChatMessageContent message)
{
string authorExpression = message.Role == AuthorRole.User ? string.Empty : $" - {message.AuthorName ?? "*"}";
string contentExpression = string.IsNullOrWhiteSpace(message.Content) ? string.Empty : message.Content;
Console.WriteLine($"\n# {message.Role}{authorExpression}:{contentExpression}");
}
}
public class CurrenTimePlugin
{
[KernelFunction, Description("Get the current local time")]
[return: Description("The current local time")]
public DateTime GetCurrentTime()
{
var currentTime = DateTime.Now;
Console.WriteLine($"Current Time: {currentTime}");
return currentTime;
}
}
Llama cpp server docker compose file
services:
llama-cpp-server:
image: ghcr.io/ggml-org/llama.cpp:server
ports:
- "8000:8000"
command: >
--model-url https://huggingface.co/bartowski/Qwen2.5-Coder-1.5B-Instruct-GGUF/resolve/main/Qwen2.5-Coder-1.5B-Instruct-Q4_K_M.gguf
--port 8000
--host 0.0.0.0
--jinja
-n 512
Expected behavior
The result of the plugin is added to chat history.
Screenshots
Platform
- Language: C#
- Source: See above for nuget versions
- AI model: recreated with qwen qwq 32b and qwen coder 1.5b instruct
- IDE: VS Code
- OS: POP OS and Ubuntu Server
Additional context
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
.NETIssue or Pull requests regarding .NET codeIssue or Pull requests regarding .NET codebugSomething isn't workingSomething isn't working
Type
Projects
Status
Sprint: Done

