Skip to content

pi-estimate-user-image-issue #4983

@yuzheng310

Description

@yuzheng310

What happened?

estimateTokens() counts image blocks in toolResult messages as 1200 tokens, but counts image blocks in user messages as 0.

Code path:

  • packages/coding-agent/src/core/compaction/compaction.ts: the user branch only sums text blocks.
  • The toolResult branch adds 4800 chars for block.type === "image", which becomes 1200 tokens.

This can make context estimates too low after user image attachments. I can implement the fix if maintainers approve.

Steps to reproduce

Add this minimal regression test as packages/coding-agent/test/user-image-estimate-repro.test.ts:

import type { AgentMessage } from "@earendil-works/pi-agent-core";
import type { ImageContent } from "@earendil-works/pi-ai";
import { describe, expect, it } from "vitest";
import { estimateTokens } from "../src/core/compaction/index.ts";

describe("estimateTokens user image repro", () => {
	it("counts user image blocks consistently with tool result image blocks", () => {
		const image: ImageContent = {
			type: "image",
			data: "base64-image-data",
			mimeType: "image/png",
		};

		const userMessage: AgentMessage = {
			role: "user",
			content: [image],
			timestamp: Date.now(),
		};

		const toolResultMessage: AgentMessage = {
			role: "toolResult",
			toolCallId: "toolu_test",
			toolName: "read",
			content: [image],
			isError: false,
			timestamp: Date.now(),
		};

		expect(estimateTokens(toolResultMessage)).toBe(1200);
		expect(estimateTokens(userMessage)).toBe(1200);
	});
});

Then run:

cd packages/coding-agent
node ../../node_modules/vitest/dist/cli.js --run test/user-image-estimate-repro.test.ts

Actual local output:

FAIL  test/user-image-estimate-repro.test.ts > estimateTokens user image repro > counts user image blocks consistently with tool result image blocks
AssertionError: expected +0 to be 1200 // Object.is equality

- Expected
+ Received

- 1200
+ 0

Expected behavior

User image blocks should be counted consistently with tool result image blocks in estimateTokens(), so UI context usage and error-path auto-compaction do not undercount sessions with user images.

Version

@earendil-works/pi-coding-agent 0.75.5, commit 30f48fea.

Metadata

Metadata

Assignees

Labels

bugSomething isn't workinginprogressIssue is being worked on

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions