Skip to content

Nvidia guided JSON test fails #2838

@anakin87

Description

@anakin87

Describe the bug
https://github.com/deepset-ai/haystack-core-integrations/actions/runs/21953077883/job/63409084941?pr=2835

To Reproduce

def test_live_run_with_guided_json_schema(self):
      json_schema = {
          "type": "object",
          "properties": {"title": {"type": "string"}, "rating": {"type": "number"}},
          "required": ["title", "rating"],
      }
      chat_messages = [
          ChatMessage.from_user(
              """
          Return the title and the rating based on the following movie review according
          to the provided json schema.
          Review: Inception is a really well made film. I rate it four stars out of five."""
          )
      ]

      component = NvidiaChatGenerator(
          model="meta/llama-3.1-70b-instruct",
          generation_kwargs={"extra_body": {"guided_json": json_schema}, "temperature": 0.0},
      )

      results = component.run(chat_messages)
      assert len(results["replies"]) == 1
      message = results["replies"][0].text
      output = json.loads(message)
      assert output["title"] == "Inception"
      assert "rating" in output
FAILED tests/test_nvidia_chat_generator.py::TestNvidiaChatGenerator::test_live_run_with_guided_json_schema - json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

What I tried

  • Switching to new NIM syntax: extra_body={"guided_json": json_schema}, (no nvext key)
  • Changing the model

This does not work and my impression is that the JSON schema is not correctly passed.

I can make the test pass including the schema in the prompt and telling the model to just return a JSON and nothing else, but at this point, the test would be meaningless.

Metadata

Metadata

Assignees

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions