Describe the bug
https://github.com/deepset-ai/haystack-core-integrations/actions/runs/21953077883/job/63409084941?pr=2835
To Reproduce
def test_live_run_with_guided_json_schema(self):
json_schema = {
"type": "object",
"properties": {"title": {"type": "string"}, "rating": {"type": "number"}},
"required": ["title", "rating"],
}
chat_messages = [
ChatMessage.from_user(
"""
Return the title and the rating based on the following movie review according
to the provided json schema.
Review: Inception is a really well made film. I rate it four stars out of five."""
)
]
component = NvidiaChatGenerator(
model="meta/llama-3.1-70b-instruct",
generation_kwargs={"extra_body": {"guided_json": json_schema}, "temperature": 0.0},
)
results = component.run(chat_messages)
assert len(results["replies"]) == 1
message = results["replies"][0].text
output = json.loads(message)
assert output["title"] == "Inception"
assert "rating" in output
FAILED tests/test_nvidia_chat_generator.py::TestNvidiaChatGenerator::test_live_run_with_guided_json_schema - json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
What I tried
- Switching to new NIM syntax:
extra_body={"guided_json": json_schema}, (no nvext key)
- Changing the model
This does not work and my impression is that the JSON schema is not correctly passed.
I can make the test pass including the schema in the prompt and telling the model to just return a JSON and nothing else, but at this point, the test would be meaningless.
Describe the bug
https://github.com/deepset-ai/haystack-core-integrations/actions/runs/21953077883/job/63409084941?pr=2835
To Reproduce
What I tried
extra_body={"guided_json": json_schema},(nonvextkey)This does not work and my impression is that the JSON schema is not correctly passed.
I can make the test pass including the schema in the prompt and telling the model to just return a JSON and nothing else, but at this point, the test would be meaningless.