Python: Add parallel_tool_calls attribute to OpenAI chat prompt execution settings by moonbox3 · Pull Request #9479 · microsoft/semantic-kernel

moonbox3 · 2024-10-30T19:29:06Z

Motivation and Context

OpenAI / Azure OpenAI released the ability to specify the parallel_tool_calls = True | False boolean attribute on the prompt execution settings in July. This was never brought into SK Python.

Further, there appears to be a pesky bug related to function calling where enabling parallel tool calls can cause 500s. The only way to get around this is to disable parallel function calling. This has been both tested on the Azure Chat Completion and OpenAI Chat Completion code and works well -- the chat history shows the synchronous function calls if multiple tool calls are required.

Description

Add the parallel_tool_calls attribute on the OpenAIChatPromptExecutionExecution settings, which are also used by the AzureChatCompletionClass.

Closes Python: Bug: How do I disable parallel function calling in Semantic Kernel (Python)? #9478
Adds unit tests for this prompt execution setting

Contribution Checklist

The code builds clean without any errors or warnings
The PR follows the SK Contribution Guidelines and the pre-submission formatting script raises no violations
All unit tests pass, and I have added new tests where possible
I didn't break anyone 😄

…tings. Add tests.

markwallace-microsoft · 2024-10-30T19:43:15Z

Python Test Coverage Report

File	Stmts	Miss	Cover	Missing
semantic_kernel
kernel.py	199	47	76%	148, 159, 163, 313–316, 423, 437–480
semantic_kernel/agents/group_chat
agent_chat.py	124	2	98%	78, 171
agent_group_chat.py	100	2	98%	151, 201
broadcast_queue.py	72	1	99%	35
semantic_kernel/agents/open_ai
assistant_content_generation.py	141	9	94%	97–98, 329–337, 379, 381
azure_assistant_agent.py	106	2	98%	285, 305
open_ai_assistant_agent.py	105	2	98%	252, 272
open_ai_assistant_base.py	467	8	98%	260, 338–339, 747, 868, 871, 945, 1007
semantic_kernel/connectors/ai
chat_completion_client_base.py	116	2	98%	382, 392
completion_usage.py	8	1	88%	17
semantic_kernel/connectors/ai/anthropic/services
anthropic_chat_completion.py	176	5	97%	147, 165, 169, 223, 419
semantic_kernel/connectors/ai/azure_ai_inference/services
azure_ai_inference_chat_completion.py	119	7	94%	120, 146–149, 159, 180, 202
azure_ai_inference_text_embedding.py	41	1	98%	87
semantic_kernel/connectors/ai/bedrock/services
bedrock_chat_completion.py	136	14	90%	117, 138, 163, 167–170, 228, 246–265, 324
bedrock_text_completion.py	57	2	96%	95, 118
bedrock_text_embedding.py	45	1	98%	94
semantic_kernel/connectors/ai/bedrock/services/model_provider
bedrock_ai21_labs.py	13	1	92%	67
bedrock_anthropic_claude.py	12	1	92%	54
bedrock_cohere.py	20	1	95%	75
utils.py	80	20	75%	68, 71, 102, 106–115, 132–150, 171–174
semantic_kernel/connectors/ai/embeddings
embedding_generator_base.py	8	1	88%	50
semantic_kernel/connectors/ai/google/google_ai/services
google_ai_chat_completion.py	119	4	97%	126, 152, 175, 177
google_ai_text_completion.py	63	2	97%	98, 121
utils.py	65	3	95%	139, 159–164
semantic_kernel/connectors/ai/google/vertex_ai/services
utils.py	66	3	95%	140, 160–165
vertex_ai_chat_completion.py	119	4	97%	121, 147, 170, 172
vertex_ai_text_completion.py	62	2	97%	95, 116
semantic_kernel/connectors/ai/hugging_face/services
hf_text_completion.py	60	3	95%	103, 112, 127
hf_text_embedding.py	32	5	84%	79–83
semantic_kernel/connectors/ai/mistral_ai/services
mistral_ai_chat_completion.py	118	7	94%	118–121, 307–310
semantic_kernel/connectors/ai/ollama/services
ollama_chat_completion.py	107	11	90%	114, 139, 143–144, 154, 187, 224, 234–235, 257, 284
ollama_text_completion.py	57	3	95%	93, 103, 131
utils.py	46	25	46%	29, 44–52, 64–86, 98–102, 119–122
semantic_kernel/connectors/ai/onnx
utils.py	53	3	94%	50–51, 112
semantic_kernel/connectors/ai/onnx/services
onnx_gen_ai_chat_completion.py	72	7	90%	67–68, 98, 122, 167, 173, 179
onnx_gen_ai_completion_base.py	58	21	64%	59–71, 79–90
onnx_gen_ai_text_completion.py	46	5	89%	54–55, 87, 117, 133
semantic_kernel/connectors/ai/open_ai/prompt_execution_settings
open_ai_prompt_execution_settings.py	95	1	99%	113
semantic_kernel/connectors/ai/open_ai/services
azure_chat_completion.py	107	5	95%	118, 123, 157, 166, 169
azure_text_completion.py	28	2	93%	82, 87
azure_text_embedding.py	30	2	93%	84, 89
open_ai_chat_completion_base.py	127	5	96%	71, 121, 141, 177, 287
open_ai_handler.py	63	3	95%	86, 95–96
open_ai_text_completion_base.py	80	2	98%	56, 161
semantic_kernel/connectors/ai/open_ai/settings
azure_open_ai_settings.py	22	1	95%	99
semantic_kernel/connectors/memory/azure_ai_search
azure_ai_search_collection.py	87	2	98%	150, 152
semantic_kernel/connectors/memory/redis
redis_collection.py	160	2	99%	146, 316
utils.py	45	11	76%	145–146, 164, 166, 173–188
semantic_kernel/connectors/memory/weaviate
utils.py	60	4	93%	78, 82, 110, 249
weaviate_collection.py	115	25	78%	144–153, 157–177, 181–186
weaviate_store.py	44	9	80%	106–114, 118–123
semantic_kernel/connectors/openapi_plugin
openapi_manager.py	58	2	97%	110–111
openapi_parser.py	88	2	98%	71, 128
openapi_runner.py	105	2	98%	181–182
semantic_kernel/connectors/openapi_plugin/models
rest_api_operation.py	129	1	99%	242
semantic_kernel/contents
function_call_content.py	100	2	98%	185, 213
streaming_chat_message_content.py	68	1	99%	210
streaming_content_mixin.py	39	2	95%	37, 64
semantic_kernel/core_plugins/sessions_python_tool
sessions_python_plugin.py	134	8	94%	69, 82–91, 99
sessions_python_settings.py	39	4	90%	84–87
semantic_kernel/data
vector_store_record_collection.py	249	19	92%	408, 468–472, 480–484, 524–527, 534–537
vector_store_record_utils.py	26	2	92%	50, 52
semantic_kernel/functions
kernel_function_decorator.py	98	1	99%	102
kernel_function_from_method.py	96	1	99%	153
kernel_function_from_prompt.py	154	7	95%	165–166, 180, 201, 219, 239, 322
kernel_function_log_messages.py	36	6	83%	37–43
kernel_plugin.py	187	2	99%	472, 475
semantic_kernel/planners
plan.py	234	45	81%	54, 163–165, 197, 214–227, 264, 269, 277–278, 288–291, 308, 313, 329, 332–337, 355, 360, 363, 365, 372, 386–388, 393–397
semantic_kernel/planners/function_calling_stepwise_planner
function_calling_stepwise_planner.py	116	4	97%	145, 189–190, 198
semantic_kernel/planners/sequential_planner
sequential_planner.py	64	6	91%	71, 75, 109, 125, 134–135
sequential_planner_extensions.py	50	9	82%	31–32, 56, 110–124
sequential_planner_parser.py	77	12	84%	66–74, 93, 117–120
semantic_kernel/processes
process_builder.py	68	39	43%	43–52, 56–58, 64–74, 78–85, 89–92, 96–100, 105, 109–114
process_end_step.py	19	2	89%	37, 41
process_function_target_builder.py	25	3	88%	37–40
process_step_builder.py	105	24	77%	44, 89, 103, 110–123, 135–142, 151, 160–169, 178, 192, 209
process_step_edge_builder.py	34	3	91%	31, 46, 56
process_types.py	25	1	96%	35
semantic_kernel/processes/kernel_process
kernel_process_step_context.py	17	1	94%	37
semantic_kernel/processes/local_runtime
local_kernel_process.py	20	2	90%	23, 30
local_kernel_process_context.py	32	2	94%	66–67
local_process.py	134	52	61%	92, 102, 120–130, 163–190, 194–199, 203, 207–213, 217–227, 231–232
local_step.py	178	114	36%	61, 72, 81–169, 173, 177, 181–182, 187–249, 253–270, 274–277, 281–284, 288–297, 303–306, 310–312
semantic_kernel/schema
kernel_json_schema_builder.py	129	9	93%	53, 90, 186, 194, 205, 213, 228, 232–233
semantic_kernel/services
ai_service_client_base.py	22	1	95%	64
semantic_kernel/template_engine/blocks
code_block.py	77	1	99%	119
named_arg_block.py	43	1	98%	98
semantic_kernel/utils/authentication
entra_id_authentication.py	15	2	87%	26, 38
semantic_kernel/utils/telemetry
user_agent.py	16	2	88%	18–19
semantic_kernel/utils/telemetry/model_diagnostics
decorators.py	171	4	98%	372–375
TOTAL	13416	713	95%

Python Unit Test Overview

Tests	Skipped	Failures	Errors	Time
2771	4 💤	0 ❌	0 🔥	1m 7s ⏱️

..._kernel/connectors/ai/open_ai/prompt_execution_settings/open_ai_prompt_execution_settings.py

moonbox3 added 2 commits October 30, 2024 15:22

Add parallel_tool_calls attribute to OpenAI chat prompt execution set…

d4be3d4

…tings. Add tests.

Remove unnecessary Pydantic field.

98bf49a

moonbox3 requested a review from a team as a code owner October 30, 2024 19:29

markwallace-microsoft added the python Pull requests for the Python Semantic Kernel label Oct 30, 2024

moonbox3 self-assigned this Oct 30, 2024

moonbox3 added the ai connector Anything related to AI connectors label Oct 30, 2024

alliscode approved these changes Oct 30, 2024

View reviewed changes

esttenorio reviewed Oct 30, 2024

View reviewed changes

..._kernel/connectors/ai/open_ai/prompt_execution_settings/open_ai_prompt_execution_settings.py Show resolved Hide resolved

esttenorio approved these changes Oct 30, 2024

View reviewed changes

moonbox3 added this pull request to the merge queue Oct 30, 2024

github-merge-queue bot removed this pull request from the merge queue due to failed status checks Oct 30, 2024

Don't send parallel tool calls if no tools are specified.

2012a8f

moonbox3 enabled auto-merge October 30, 2024 21:07

alliscode approved these changes Oct 30, 2024

View reviewed changes

moonbox3 added this pull request to the merge queue Oct 30, 2024

Merged via the queue into microsoft:main with commit 5e632fd Oct 30, 2024

moonbox3 deleted the auto-func-calling-attribute branch October 30, 2024 22:45

moonbox3 mentioned this pull request Oct 31, 2024

Python: Bug: extension_data doesn't seem to propagate in chat completions API #9456

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Python: Add parallel_tool_calls attribute to OpenAI chat prompt execution settings#9479

Python: Add parallel_tool_calls attribute to OpenAI chat prompt execution settings#9479
moonbox3 merged 3 commits intomicrosoft:mainfrom
moonbox3:auto-func-calling-attribute

moonbox3 commented Oct 30, 2024

Uh oh!

markwallace-microsoft commented Oct 30, 2024 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

moonbox3 commented Oct 30, 2024

Motivation and Context

Description

Contribution Checklist

Uh oh!

markwallace-microsoft commented Oct 30, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Python Unit Test Overview

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

markwallace-microsoft commented Oct 30, 2024 •

edited

Loading