Problems with the is_native function in functions_view.py #1212
Description
Describe the bug
There is a problem with FunctionsView's native function validation.
def is_semantic(self, skill_name: str, function_name: str) -> bool:
as_sf = self._semantic_functions.get(skill_name, [])
as_sf = any(f.name == function_name for f in as_sf)
as_nf = self._native_functions.get(skill_name, [])
as_nf = any(f.name == function_name for f in as_nf)
if as_sf and as_nf:
raise KernelException(
KernelException.ErrorCodes.AmbiguousImplementation,
f"There are 2 functions with the same name: {function_name}."
f"One is native and the other semantic.",
)
return as_sf
def is_native(self, skill_name: str, function_name: str) -> bool:
return not self.is_semantic(skill_name, function_name)The result we want to achieve by using that method should look like this.
| as_sf | as_nf | is_semantic Result | is_native Result |
|---|---|---|---|
| True | True | KernelException | KernelException |
| True | False | True | False |
| False | True | False | True |
| False | False | False | False |
However, with the code above, the result is as follows:
the is_native function will return True even if as_nf is False, i.e., the given native function was not checked.
| as_sf | as_nf | is_semantic Result | is_native Result |
|---|---|---|---|
| True | True | KernelException | KernelException |
| True | False | True | False |
| False | True | False | True |
| False | False | False | True |
To Reproduce
Here is my test code.
Prepare in advance
import semantic_kernel as sk
from semantic_kernel.core_skills import MathSkill
import semantic_kernel.connectors.ai.open_ai as sk_oai
kernel = sk.Kernel()
model = "text-davinci-002"
service_id = model
# Configure AI service used by the kernel
api_key, org_id = sk.openai_settings_from_dot_env()
kernel.add_text_completion_service(
service_id, sk_oai.OpenAITextCompletion(model, api_key, org_id)
)
# Import Semantic and Native skill
kernel.import_semantic_skill_from_directory("../samples/skills", "FunSkill")
kernel.import_skill(MathSkill(), "math")
# Verify that the skill has been imported
print(kernel.skills.get_functions_view()._semantic_functions)
print(kernel.skills.get_functions_view()._native_functions)Output
{'FunSkill': [<semantic_kernel.skill_definition.function_view.FunctionView object at 0x7f8ba39068b0>, <semantic_kernel.skill_definition.function_view.FunctionView object at 0x7f8ba39067f0>, <semantic_kernel.skill_definition.function_view.FunctionView object at 0x7f8ba3906820>]}
{'math': [<semantic_kernel.skill_definition.function_view.FunctionView object at 0x7f8ba39067f0>, <semantic_kernel.skill_definition.function_view.FunctionView object at 0x7f8ba3906820>]}✅ Normal Case
# Determine if the skill is semantic or native
print(kernel.skills.get_functions_view().is_semantic("FunSkill", "Joke"))
print(kernel.skills.get_functions_view().is_native("math", "Add"))In this case, both the semantic and native functions exist within the given skill, so it outputs True, True.
True
True❌Problem Case
# Determine if the skill is semantic or native
print(kernel.skills.get_functions_view().is_semantic("FunSkill", "Joke"))
print(kernel.skills.get_functions_view().is_native("math", "Missing Function"))In this case, is_native is returning True even though function not in math(which is natvie skill).
True
TrueExpected behavior
Therefore, the native function check logic should be modified as follows:
def is_semantic(self, skill_name: str, function_name: str) -> bool:
as_sf = self._semantic_functions.get(skill_name, [])
as_sf = any(f.name == function_name for f in as_sf)
as_nf = self._native_functions.get(skill_name, [])
as_nf = any(f.name == function_name for f in as_nf)
if as_sf and as_nf:
raise KernelException(
KernelException.ErrorCodes.AmbiguousImplementation,
f"There are 2 functions with the same name: {function_name}."
f"One is native and the other semantic.",
)
return as_sf
def is_native(self, skill_name: str, function_name: str) -> bool:
as_sf = self._semantic_functions.get(skill_name, [])
as_sf = any(f.name == function_name for f in as_sf)
as_nf = self._native_functions.get(skill_name, [])
as_nf = any(f.name == function_name for f in as_nf)
if as_sf and as_nf:
raise KernelException(
KernelException.ErrorCodes.AmbiguousImplementation,
f"There are 2 functions with the same name: {function_name}."
f"One is native and the other semantic.",
)
return as_nfScreenshots
Desktop (please complete the following information):
- OS: macOS
- IDE: VSCode
- NuGet Package Version [e.g. 0.1.0]
Additional context
Except, I had one question. In _normalize_names in the skill_collection, it performs lower_case on skill_name and function_name to check for case insensitive skill and function names.
However, inside the functions_view, it performs an exact match without normalizing. I think these two processes are inconsistent.
Shouldn't the same process be applied in functions_view? Please comment if you agree.