chore(internal): simplify http snapshots by karpetrosyan · Pull Request #1092 · anthropics/anthropic-sdk-python

karpetrosyan · 2025-12-03T18:09:31Z

Before:

    @pytest.mark.respx(base_url=base_url)
    def test_basic_call_sync(self, client: Anthropic, respx_mock: MockRouter, monkeypatch: pytest.MonkeyPatch) -> None:
        @beta_tool
        def get_weather(location: str, units: Literal["c", "f"]) -> BetaFunctionToolResultType:
            """Lookup the weather for a given city in either celsius or fahrenheit

            Args:
                location: The city and state, e.g. San Francisco, CA
                units: Unit for the output, either 'c' for celsius or 'f' for fahrenheit
            Returns:
                A dictionary containing the location, temperature, and weather condition.
            """
            return json.dumps(_get_weather(location, units))

        message = make_snapshot_request(
            lambda c: c.beta.messages.tool_runner(
                max_tokens=1024,
                model="claude-haiku-4-5",
                tools=[get_weather],
                messages=[{"role": "user", "content": "What is the weather in SF?"}],
            ).until_done(),
            content_snapshot=snapshots["basic"]["responses"],
            path="/v1/messages",
            mock_client=client,
            respx_mock=respx_mock,
        )

After:

    @pytest.mark.parametrize(
        "http_snapshot",
        [cast(Any, external("uuid:ad005e89-f72b-4f0e-941a-9ec3473c1bc8.json"))],
    )
    def test_basic_call_sync(self, snapshot_client: Anthropic) -> None:
        @beta_tool
        def get_weather(location: str, units: Literal["c", "f"]) -> BetaFunctionToolResultType:
            """Lookup the weather for a given city in either celsius or fahrenheit

            Args:
                location: The city and state, e.g. San Francisco, CA
                units: Unit for the output, either 'c' for celsius or 'f' for fahrenheit
            Returns:
                A dictionary containing the location, temperature, and weather condition.
            """
            return json.dumps(_get_weather(location, units))

        message = snapshot_client.beta.messages.tool_runner(
            max_tokens=1024,
            model="claude-haiku-4-5",
            tools=[get_weather],
            messages=[{"role": "user", "content": "What is the weather in SF?"}],
        ).until_done()

karpetrosyan · 2025-12-05T15:15:24Z

.../__inline_snapshot__/test_runners/TestSyncRunTools/ff848716-2309-477f-8e90-39877809aaad.json

+        "x-robots-tag": "none",
+        "server": "cloudflare"
+      },
+      "body": "event: message_start\ndata: {\"type\":\"message_start\",\"message\":{\"model\":\"claude-haiku-4-5-20251001\",\"id\":\"msg_01SAGfXDtnWyD3j4h5Wz3kky\",\"type\":\"message\",\"role\":\"assistant\",\"content\":[],\"stop_reason\":null,\"stop_sequence\":null,\"usage\":{\"input_tokens\":656,\"cache_creation_input_tokens\":0,\"cache_read_input_tokens\":0,\"cache_creation\":{\"ephemeral_5m_input_tokens\":0,\"ephemeral_1h_input_tokens\":0},\"output_tokens\":26,\"service_tier\":\"standard\"}}       }\n\nevent: content_block_start\ndata: {\"type\":\"content_block_start\",\"index\":0,\"content_block\":{\"type\":\"tool_use\",\"id\":\"toolu_01LZ7WbFLVzFd16Ur9RHXeGz\",\"name\":\"get_weather\",\"input\":{}}   }\n\nevent: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"input_json_delta\",\"partial_json\":\"\"}}\n\nevent: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"input_json_delta\",\"partial_json\":\"{\\\"loca\"}             }\n\nevent: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"input_json_delta\",\"partial_json\":\"tion\\\": \\\"Sa\"}         }\n\nevent: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"input_json_delta\",\"partial_json\":\"n Fran\"}          }\n\nevent: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"input_json_delta\",\"partial_json\":\"cisco, CA\\\"\"}        }\n\nevent: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"input_json_delta\",\"partial_json\":\", \\\"units\\\"\"}            }\n\nevent: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"input_json_delta\",\"partial_json\":\": \\\"f\\\"}\"}              }\n\nevent: content_block_stop\ndata: {\"type\":\"content_block_stop\",\"index\":0 }\n\nevent: message_delta\ndata: {\"type\":\"message_delta\",\"delta\":{\"stop_reason\":\"tool_use\",\"stop_sequence\":null},\"usage\":{\"input_tokens\":656,\"cache_creation_input_tokens\":0,\"cache_read_input_tokens\":0,\"output_tokens\":74}              }\n\nevent: message_stop\ndata: {\"type\":\"message_stop\"        }\n\n"


I'll add a serializer for text/event-stream content-type, so it'll be pretty printed too

You planning on doing that in a follow-on? or in this PR too

I'll do it in a different one

RobertCraigie · 2025-12-09T12:45:21Z

tests/lib/_parse/test_beta_messages.py

+        response = await async_stream_parse(async_snapshot_client)

-        assert response.content[0].text == snapshot("[12345,67890]")
+        assert response.content[0].text == snapshot("[12345,67890]")  # type: ignore


q: why do we need the type ignore now?

Ah, that's for accessing the text property, since it might be missing. Idk what's better an extra line to check for it, or just type: ignore. pytest would anyway fail with the good message if there would not be .text 🤷‍♂️

RobertCraigie · 2025-12-09T12:46:47Z

.../tools/__inline_snapshot__/test_runners/__module__/c4061e82-ea59-4756-9549-71a35da24499.json

+  {
+    "response": {
+      "status_code": 200,
+      "headers": {


q: can / should we filter out a bunch of these headers? some of them will frequently update and aren't helpful, e.g. date

That's a good point. I can add a whitelist/blacklist for headers, or an option to exclude them entirely. I think in the SDKs, we're more interested in the request headers, not the response headers

Yeah can you update the default header blacklist to include the following?

date

request-id

anthropic-organization-id

x-envoy-upstream-service-time

cf-ray

RobertCraigie · 2025-12-09T12:47:32Z

.../tools/__inline_snapshot__/test_runners/__module__/c4061e82-ea59-4756-9549-71a35da24499.json

@@ -0,0 +1,93 @@
+[
+  {


q: can / should we include the request here as well?

Hmmm. I initially disabled it by default for more compact snapshots. Since we won't need request snapshots in every test, maybe it makes more sense to enable it per-test when needed

ooc how do you enable it for a test? I think we should probably include the requests for the tool runner tests?

I do...

@pytest.mark.parametrize( "http_snapshot, http_snapshot_serializer_options", [ ( inline_snapshot.external("uuid:my-test-snapshot.json"), SnapshotSerializerOptions( exclude_request_headers=["X-API-Key"], include_request=True, # Include request details in snapshot ), ), ], )

@RobertCraigie Okay, I’ve updated it so that all snapshots now include the request, and it doesn’t seem too messy.

RobertCraigie · 2025-12-09T12:48:00Z

.../__inline_snapshot__/test_runners/TestSyncRunTools/ff848716-2309-477f-8e90-39877809aaad.json

+        "x-robots-tag": "none",
+        "server": "cloudflare"
+      },
+      "body": "event: message_start\ndata: {\"type\":\"message_start\",\"message\":{\"model\":\"claude-haiku-4-5-20251001\",\"id\":\"msg_01SAGfXDtnWyD3j4h5Wz3kky\",\"type\":\"message\",\"role\":\"assistant\",\"content\":[],\"stop_reason\":null,\"stop_sequence\":null,\"usage\":{\"input_tokens\":656,\"cache_creation_input_tokens\":0,\"cache_read_input_tokens\":0,\"cache_creation\":{\"ephemeral_5m_input_tokens\":0,\"ephemeral_1h_input_tokens\":0},\"output_tokens\":26,\"service_tier\":\"standard\"}}       }\n\nevent: content_block_start\ndata: {\"type\":\"content_block_start\",\"index\":0,\"content_block\":{\"type\":\"tool_use\",\"id\":\"toolu_01LZ7WbFLVzFd16Ur9RHXeGz\",\"name\":\"get_weather\",\"input\":{}}   }\n\nevent: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"input_json_delta\",\"partial_json\":\"\"}}\n\nevent: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"input_json_delta\",\"partial_json\":\"{\\\"loca\"}             }\n\nevent: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"input_json_delta\",\"partial_json\":\"tion\\\": \\\"Sa\"}         }\n\nevent: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"input_json_delta\",\"partial_json\":\"n Fran\"}          }\n\nevent: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"input_json_delta\",\"partial_json\":\"cisco, CA\\\"\"}        }\n\nevent: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"input_json_delta\",\"partial_json\":\", \\\"units\\\"\"}            }\n\nevent: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"input_json_delta\",\"partial_json\":\": \\\"f\\\"}\"}              }\n\nevent: content_block_stop\ndata: {\"type\":\"content_block_stop\",\"index\":0 }\n\nevent: message_delta\ndata: {\"type\":\"message_delta\",\"delta\":{\"stop_reason\":\"tool_use\",\"stop_sequence\":null},\"usage\":{\"input_tokens\":656,\"cache_creation_input_tokens\":0,\"cache_read_input_tokens\":0,\"output_tokens\":74}              }\n\nevent: message_stop\ndata: {\"type\":\"message_stop\"        }\n\n"


You planning on doing that in a follow-on? or in this PR too

RobertCraigie · 2025-12-09T12:49:04Z

tests/lib/utils.py

        return self.__class__.__name__.split("[", maxsplit=1)[0]

-    with monkeypatch.context() as m:
+    with pytest.MonkeyPatch.context() as m:


q: does this still work as a test-scoped patch? I did the forward monkeypatch setup before to make sure any patches are cleaned up properly; If this works too that would be cool.

I did the forward monkeypatch setup before to make sure any patches are cleaned up properly

It cleans up immediately after exiting the context manager, in both cases

oh very nice, I missed the context manager 🤦

karpetrosyan · 2026-02-24T12:50:49Z

@claude can I get your review?

karpetrosyan · 2026-02-24T13:19:11Z

@claude review this PR please

RobertCraigie

I only really skimmed but this looks great!

karpetrosyan requested a review from a team as a code owner December 3, 2025 18:09

karpetrosyan marked this pull request as draft December 3, 2025 18:09

karpetrosyan force-pushed the improve-requests-snapshots branch 4 times, most recently from 73d3f9a to 87759a3 Compare December 5, 2025 15:12

karpetrosyan commented Dec 5, 2025

View reviewed changes

karpetrosyan marked this pull request as ready for review December 5, 2025 15:41

karpetrosyan requested a review from RobertCraigie December 5, 2025 15:41

karpetrosyan added the ready-to-merge label Dec 9, 2025

RobertCraigie reviewed Dec 9, 2025

View reviewed changes

stainless-app bot force-pushed the next branch from 5302f27 to 9b5ab24 Compare January 13, 2026 18:40

stainless-app bot force-pushed the next branch from 1dd6119 to cd1b39b Compare February 7, 2026 18:05

stainless-app bot force-pushed the next branch from f6ce67c to 4de03c2 Compare February 17, 2026 19:25

dtmeadows requested a review from RobertCraigie February 19, 2026 14:12

karpetrosyan force-pushed the improve-requests-snapshots branch 2 times, most recently from 11678b3 to 1eaae19 Compare February 20, 2026 09:43

karpetrosyan marked this pull request as draft February 20, 2026 09:45

karpetrosyan force-pushed the improve-requests-snapshots branch 2 times, most recently from 3c684e1 to 6eaf2e3 Compare February 24, 2026 13:16

chore(internal): simplify http snapshots

b066374

karpetrosyan force-pushed the improve-requests-snapshots branch from 6eaf2e3 to b066374 Compare February 24, 2026 13:42

karpetrosyan marked this pull request as ready for review February 24, 2026 13:49

RobertCraigie approved these changes Feb 24, 2026

View reviewed changes

RobertCraigie merged commit 4a4dc9f into anthropics:next Feb 24, 2026
5 checks passed

stainless-app bot mentioned this pull request Feb 24, 2026

release: 0.84.0 #1193

Merged

stainless-app bot pushed a commit that referenced this pull request Feb 25, 2026

chore(internal): simplify http snapshots (#1092)

c3017a2

Conversation

karpetrosyan commented Dec 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

karpetrosyan Dec 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

karpetrosyan commented Feb 24, 2026

Uh oh!

karpetrosyan commented Feb 24, 2026

Uh oh!

RobertCraigie left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

karpetrosyan commented Dec 3, 2025 •

edited

Loading

karpetrosyan Dec 9, 2025 •

edited

Loading