Skip to content

fix(gateway): use filtered history length for transcript message extraction#395

Merged
teknium1 merged 1 commit into
NousResearch:mainfrom
PercyDikec:fix/transcript-history-offset
Mar 5, 2026
Merged

fix(gateway): use filtered history length for transcript message extraction#395
teknium1 merged 1 commit into
NousResearch:mainfrom
PercyDikec:fix/transcript-history-offset

Conversation

@PercyDikec

Copy link
Copy Markdown
Contributor

The transcript extraction used len(history) to find new messages from each agent turn, but history includes session_meta entries that are stripped before passing to the agent. This mismatch caused 1 message to be lost from the transcript on every turn after the first.

Changes:

  • _run_agent now returns history_offset (the filtered history length actually passed to the agent) in its result dict
  • The caller uses history_offset instead of len(history) for the slice
  • Changed the else branch from returning all agent_messages to returning [], preventing full-history duplication when session_meta count equals agent message count

Tests

Added tests/gateway/test_transcript_offset.py with 6 tests:

  • session_meta causes offset mismatch - core bug: 1 session_meta entry causes 1 message loss
  • no session_meta gives same result - first turn (no session_meta) works correctly with both approaches
  • multiple session_meta entries - 2 session_meta entries cause the else branch to duplicate the whole history
  • system messages also filtered - system messages in history also contribute to the offset drift
  • else branch returns empty list - verifies the else branch fix returns [] instead of all messages
  • tool call messages preserved in filter - tool_calls and tool results pass through the filter correctly, keeping the offset accurate

Closes #394

…action

The transcript extraction used len(history) to find new messages, but
history includes session_meta entries that are stripped before passing
to the agent. This mismatch caused 1 message to be lost from the
transcript on every turn after the first, because the slice offset
was too high. Use the filtered history length (history_offset) returned
by _run_agent instead.

Also changed the else branch from returning all agent_messages to
returning an empty list, so compressed/shorter agent output does not
duplicate the entire history into the transcript.
@teknium1 teknium1 merged commit bd3025d into NousResearch:main Mar 5, 2026
teknium1 added a commit that referenced this pull request Mar 5, 2026
The error return (no final_response) was missing history_offset,
falling back to len(history) which has the same session_meta offset
bug fixed in PR #395. Now both return paths include the correct
filtered history length.
angelburgosrosado pushed a commit to angelburgosrosado/hermes-agent that referenced this pull request Apr 27, 2026
…for transcript message extraction

Authored by PercyDikec. Fixes NousResearch#394.

The transcript extraction used len(history) to find new messages, but
history includes session_meta entries stripped before reaching the agent.
This caused 1 message lost per turn from turn 2 onwards. Fix returns
history_offset (filtered length) from _run_agent and uses it for the slice.
angelburgosrosado pushed a commit to angelburgosrosado/hermes-agent that referenced this pull request Apr 27, 2026
The error return (no final_response) was missing history_offset,
falling back to len(history) which has the same session_meta offset
bug fixed in PR NousResearch#395. Now both return paths include the correct
filtered history length.
olympus-terminal pushed a commit to olympus-terminal/hermes-agent that referenced this pull request May 16, 2026
…for transcript message extraction

Authored by PercyDikec. Fixes NousResearch#394.

The transcript extraction used len(history) to find new messages, but
history includes session_meta entries stripped before reaching the agent.
This caused 1 message lost per turn from turn 2 onwards. Fix returns
history_offset (filtered length) from _run_agent and uses it for the slice.
olympus-terminal pushed a commit to olympus-terminal/hermes-agent that referenced this pull request May 16, 2026
The error return (no final_response) was missing history_offset,
falling back to len(history) which has the same session_meta offset
bug fixed in PR NousResearch#395. Now both return paths include the correct
filtered history length.
rafe-walker added a commit to rafe-walker/kora that referenced this pull request May 20, 2026
…ll — closes D-kr2-st4 (#14)

emit_kora_event body swap: mcp_client.invoke('kora__append_event', {workspace_id, event_type, payload}) returns substrate event_id. K-9 contract honored. Defensive guards: None mcp_client → ValueError; unexpected response shape → RuntimeError. provider._attempt_chain_event_emit fetches IsoKronMCPClient via KR-7a-wired get_mcp_client(); catches IsoKronMCPInvocationError at lifecycle boundary; ERROR-logs [kora.chain.emit.failed] (operator-visible per PM-lean substrate-side-issue-worth-surfacing). Successful emits INFO-log [kora.chain.emit] with event_id. iso_node_supersede refactored to route kora.node.superseded emit through _attempt_chain_event_emit (unified error handling + cache invalidation surface). Deprecation runway: ChainEventEmitNotAvailableError class kept exported tagged [kora.isokron.deprecated] for one release. Production-test posture: K-9 handler is notImplementedHandler stub on substrate main; dispatch tier (IsoKron task NousResearch#395) un-stubs + bridges Layer-A→Layer-B actor_kind='kora'. KR-7 code shape sound; mock tests pass; production deploys wait. Verify-at-first-live-emit step (actor_id resolution to 0076-seeded canonical Kora actor) documented. ty zero-delta. 351/351 plugins/memory tests pass. Open deferrals 4→3.
rafe-walker pushed a commit to rafe-walker/kora that referenced this pull request May 20, 2026
…pad MCP call — closes D-kr2-st3

Mechanical follow-on swap unlocked by K-8 (substrate `bd165eb2`) +
KR-7a's MCP client transport. Same pattern as KR-7 chain-emit:
write_scratchpad_entry body replaces `raise
ScratchpadWriteNotAvailableError()` with
`await mcp_client.invoke('kora__write_agent_scratchpad', {...})`,
returning the substrate-assigned scratchpad_entry_id. ~40 LOC across
scratchpad.py + provider.py + tools/iso_node.py + tests + docs.

Production-test posture per IsoKron PM #27:
* K-8's kora__write_agent_scratchpad handler is a
  notImplementedHandler stub on substrate main; dispatch tier (task
  NousResearch#395) un-stubs + bridges Layer-A wsk_* auth → Layer-B actor_kind.
* KR-8 code shape is sound; mock tests verify correctness.
* Production deploys wait on dispatch tier landing — identical
  posture to KR-7 / KR-7b / KR-7a.

K-DG note: K-8's internal flow emits the substrate-canonical chain
literal `kronicle.agent_scratchpad.created` (NOT
`kora.scratchpad.entry.created` as some prior specs may suggest).
The runtime doesn't pass an event_type — the substrate emits
internally as part of the SECDEF flow. Verify-at-first-live-emit:
confirm event_log.actor_id resolves to the 0076-seeded canonical
Kora actor.

scratchpad.py:
* write_scratchpad_entry body: live call to mcp_client.invoke. K-8
  contract output: {'scratchpad_entry_id': '<uuid>',
  'approved_event_id': '<uuid>'} — projected to a plain str
  scratchpad_entry_id return.
* BLAKE3 content_hash computed via existing
  compute_scratchpad_content_hash() helper; passed to MCP tool.
* Defensive: None mcp_client raises ValueError; unexpected response
  shape raises RuntimeError (same pattern as KR-7's emit_kora_event).
* mcp_client param now required (was Optional with deferred-error
  default). Forward signature stays caller-stable from KR-2 ST3.
* ScratchpadWriteNotAvailableError class kept exported tagged
  [kora.isokron.deprecated] for one release. Original deferred-tag
  preserved inside the message for grep stability.

provider.py:_attempt_scratchpad_write:
* Stop catching ScratchpadWriteNotAvailableError (no longer raised).
* Fetch IsoKronMCPClient via self._connection.get_mcp_client();
  failure logs ERROR [kora.scratchpad.write.failed] + returns.
* Catch IsoKronMCPInvocationError + defensive Exception at the
  lifecycle boundary — log at ERROR so failures are operator-visible
  (same posture as KR-7's chain emit). Sessions don't crash; writes
  drop with a loud log line.
* Successful writes log INFO [kora.scratchpad.write] with the
  substrate-assigned entry_id and invalidate the per-workspace
  own-scratchpad cache so the next read re-fetches.

tools/iso_node.py:
* _handle_iso_node_create + _handle_iso_node_supersede: dropped
  the deferred-envelope return path. On success return
  {"ok": True, "entry_id": <substrate-uuid>}. On
  IsoKronMCPInvocationError return
  {"ok": False, "substrate_error": True, "tool_name": ...,
   "message": ...} — structured signal for the model that maps to
  substrate-level failures distinct from validation errors.
* iso_node_supersede also passes the new entry_id into the
  kora.node.superseded chain event payload so audit consumers can
  trace the supersession chain.
* Removed unused ScratchpadWriteNotAvailableError import.

Tests (5 new + 9 updated):
* test_scratchpad.py:
  - Replaced test_write_scratchpad_entry_raises_deferred_write_error
    with 4 MCP-call-path tests (happy/error/None-client/bad-shape)
    + 1 deprecation-runway test.
  - _FakeProviderConnection now exposes get_mcp_client() returning a
    _FakeMcpClient; sync_turn/on_memory_write tests flipped from
    deferred-WARNING assertions to success-INFO + entry_id checks.
  - New test_sync_turn_substrate_error_logs_at_error_session_continues
    verifies the IsoKronMCPInvocationError → ERROR-log path.

* test_iso_node_tools.py:
  - _FakeProviderConnection gains get_mcp_client(); new _FakeMcpClient
    routes by tool_name (kora__write_agent_scratchpad → spe-mock-N;
    kora__append_event → evt-mock-N; kora__read_kora_capability_row
    → minimal matrix).
  - test_iso_node_create_returns_deferred_payload →
    test_iso_node_create_returns_ok_envelope_with_substrate_entry_id;
    new test_iso_node_create_substrate_error_surfaces_structured_envelope.
  - test_iso_node_supersede_inherits_node_kind_from_original flipped
    to assert success + verify the inherited node_kind in the new
    content_inline header + verify the kora.node.superseded emit
    payload carries the new entry_id.

* test_tool_finalize.py: round-trip tests flipped from deferred to
  success-envelope assertions. _FakeProviderConnection gains
  get_mcp_client().

* test_provider_end_to_end.py: _FakeMcpClient extended to handle
  kora__write_agent_scratchpad (returns spe-mock-N). E2E assertions
  flipped from 3 deferred-WARNINGs to 3 [kora.scratchpad.write] INFO
  logs + 3 scratchpad_write_calls on the fake + 2 append_event_calls.
  Assertion narrowed to NOT count the cap-matrix fetch (KR-7b adds
  that; this PR stays narrow per parallel-mergeability).

BUILD_DEVIATIONS:
* D-kr2-st3-no-scratchpad-write-mcp-tool moved Open → Closed with
  Rule-5 spec-quote, KR-8's call-site refactor inventory, production-
  test posture, deprecation-runway note, verify-at-first-live-emit
  step.
* 1 deferral remains open (D-kr3-st2 — KR-9 dispatchable).

README "Operator pitfalls":
* Deferred-surface table trimmed 2 → 1 row (D-kr3-st2 only).
* D-kr2-st3 moves to Recently-closed list.
* Individual "Scratchpad writes are currently deferred" pitfall
  rewritten as "Scratchpad writes route via kora__write_agent_
  scratchpad (KR-8)" with success/failure log paths + operator
  grep pointer.

Local gates:
* ty check — 7,337 diagnostics, zero-delta vs KR-7 baseline.
* pytest tests/plugins/memory/ — 357/357 passing.
* Full suite via xdist (-n auto): 24,654 / 168 failed / 129 skipped.
  Δ vs KR-7b (24,652/175/129): +2 passed, -7 failed. Same
  tests/tools/* + tests/tui_gateway/* xdist isolation noise; none
  touch isokron.

1 deferral still open pending CC#1 substrate:
* D-kr3-st2-no-relationlink-write-mcp-tool ← KR-9 (K-10 merged; PM dispatched)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
rafe-walker added a commit to rafe-walker/kora that referenced this pull request May 20, 2026
…pad MCP call — closes D-kr2-st3 (#16)

Mechanical follow-on swap unlocked by K-8 (substrate bd165eb2) + KR-7a's MCP client transport. Same pattern as KR-7 chain-emit: write_scratchpad_entry body replaces raise ScratchpadWriteNotAvailableError with await mcp_client.invoke('kora__write_agent_scratchpad', {...}), returning the substrate-assigned scratchpad_entry_id. Production-test posture per IsoKron PM #27: K-8 handler is notImplementedHandler stub on substrate main; dispatch tier (task NousResearch#395) un-stubs + bridges Layer-A→Layer-B; KR-8 code-shape sound; mock tests verify correctness; production deploys wait on dispatch tier. K-DG note: substrate-canonical chain literal kronicle.agent_scratchpad.created (NOT kora.scratchpad.entry.created). ScratchpadWriteNotAvailableError class kept exported tagged [kora.isokron.deprecated] for one release. Defensive guards: None mcp_client → ValueError; unexpected response shape → RuntimeError. tools/iso_node.py iso_node_create + iso_node_supersede dropped deferred-envelope return path; on success returns {ok:true, entry_id}; on IsoKronMCPInvocationError returns {ok:false, substrate_error:true, tool_name, message}. 5 new + 9 updated tests; ty zero-delta; 357/357 plugins/memory tests pass. Cherry-pick rebased onto post-KR-7 + KR-7b main.
rafe-walker pushed a commit to rafe-walker/kora that referenced this pull request May 20, 2026
…ink MCP call — closes D-kr3-st2

Fourth + final mechanical follow-on swap unlocked by K-10 + 0083
(IsoKron PM #32). Same pattern as KR-7 (chain-emit) and KR-8
(scratchpad-write): create_relationlink body replaces `raise
RelationLinkWriteNotAvailableError()` with
`await mcp_client.invoke('kora__create_relationlink', {...})`,
returning the substrate-assigned link_id. ~30 LOC across
relationlink.py + tools/iso_link.py + tests + docs.

All three KR-3 ST2 substrate blockers are now resolved substrate-side:
* actor_kind CHECK extended to include 'kora' via 0083 (prod-applied)
* public.kora_create_relationlink SECDEF function exists
* kora.relationlink.created event literal in event_log_event_type_check
  (300th literal in the set post-0083)
* kora__create_relationlink Sea MCP tool registered

Production-test posture per IsoKron PM #27:
* K-10's kora__create_relationlink handler is a notImplementedHandler
  stub on substrate main; substrate-team's dispatch tier (task NousResearch#395)
  un-stubs + bridges Layer-A wsk_* auth → Layer-B actor_kind='kora'.
* KR-9 code shape is sound; mock tests verify correctness.
* Production deploys wait on dispatch tier landing — identical
  posture to KR-7 / KR-7b / KR-8.

Verify-at-first-live-emit per spec § 3:
* event_log row with event_type = 'kora.relationlink.created'
* event_log.actor_id = 0076-seeded canonical Kora actor (NOT
  token-UUIDv5)
* relationlink row with chain_event_id matching the emitted event +
  created_by_actor_kind = 'kora'

relationlink.py:
* create_relationlink body: live call to mcp_client.invoke. K-10
  contract output: {'link_id': '<uuid>', 'chain_event_id': '<uuid>'}
  — projected to a plain str link_id return.
* mcp_client param now required (was Optional with deferred-error
  default).
* New rationale_block_id + evidence_block_ids params match K-10's
  input schema. Legacy `rationale` parameter preserved for one-
  release back-compat (silently dropped — superseded by
  rationale_block_id).
* Defensive: None mcp_client → ValueError; unexpected response shape
  → RuntimeError (mirrors KR-7/KR-8 pattern).
* RelationLinkWriteNotAvailableError class kept exported tagged
  [kora.isokron.deprecated] for one release. Original three-blocker
  message preserved inside the message body for grep stability.

tools/iso_link.py:_handle_iso_link_create:
* Dropped the deferred-envelope path. Success →
  {"ok": True, "link_id": <substrate-uuid>}. Substrate failure →
  {"ok": False, "substrate_error": True, "tool_name", "message"} —
  same structured signal shape as KR-8's iso_node_create.
* Fetches IsoKronMCPClient via get_mcp_client(); MCP-client-
  unavailable returns {"ok": False, "error": ...} envelope.
* Removed unused RelationLinkWriteNotAvailableError import.

Tests (7 new + 2 updated):
* test_iso_link_tools.py:
  - test_create_relationlink_raises_deferred_write_error →
    test_create_relationlink_invokes_kora__create_relationlink:
    happy path asserts spec-pinned arg shape (workspace_id,
    from/to entity_id + kind, link_type, evidence_block_ids=[]).
  - New test_create_relationlink_propagates_mcp_invocation_error
    (substrate active-edge uniqueness violation surfaces).
  - New test_create_relationlink_rejects_none_mcp_client.
  - New test_create_relationlink_rejects_unexpected_response_shape.
  - New test_create_relationlink_passes_rationale_block_id_when_present
    (optional args pass-through).
  - New test_relationlink_write_not_available_error_still_importable_post_kr9
    (deprecation runway).
  - test_iso_link_create_handler_returns_deferred_envelope →
    _returns_ok_envelope_with_substrate_link_id: success-envelope
    + verifies spec-pinned tool name in invoke recording.
  - New test_iso_link_create_handler_surfaces_substrate_error_envelope:
    IsoKronMCPInvocationError flips to structured envelope.
  - _FakeMcpClient routes by tool_name (returns link-N for
    kora__create_relationlink). _FakeProviderConnection gains
    get_mcp_client().

BUILD_DEVIATIONS:
* D-kr3-st2-no-relationlink-write-mcp-tool moved Open → Closed with
  Rule-5 spec-quote, call-site refactor inventory, production-test
  posture, all-three-blockers-resolved-substrate-side note,
  deprecation-runway note, verify-at-first-live-emit step.

README "Operator pitfalls":
* "Deferred-surface summary" table replaced with
  "All BUILD_DEVIATIONS closed code-side as of KR-9 (parallel-merged
  with KR-7b + KR-8)" note. All 5 deferral closures listed under
  Recently closed.
* Individual "iso_link_create writes are blocked" pitfall rewritten
  as "RelationLink writes route via kora__create_relationlink (KR-9)"
  with success/failure envelope shapes + operator grep pointer.

Parallel-mergeability note: KR-7b + KR-8 + KR-9 modify disjoint files
(KR-7b: capability_matrix_mirror.py + provider.initialize; KR-8:
scratchpad.py + provider sync_turn/on_memory_write; KR-9:
relationlink.py + tools/iso_link.py). Merge interleaves; each PR
narrows its E2E assertions to its own surface so test fixture
counts don't fight. PM handles README + BUILD_DEVIATIONS final
reconciliation on merge order.

Local gates:
* ty check — 7,337 diagnostics, zero-delta vs KR-7 baseline.
* pytest tests/plugins/memory/ — 357/357 passing.
* Full suite via xdist (-n auto): 24,629 / 181 failed / 12 errors /
  129 skipped. Same tests/tools/* xdist isolation noise as documented
  across prior PRs; none touch isokron.

After KR-9 + KR-7b + KR-8 merge, all four BUILD_DEVIATIONS are
closed code-side. Standing-by state becomes dispatch-tier-gated
(waiting on substrate-team task NousResearch#395 + service-token mint for
production deploys).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
rafe-walker added a commit to rafe-walker/kora that referenced this pull request May 20, 2026
…ink MCP call — closes D-kr3-st2 (#17)

4th and FINAL mechanical closure swap. Mechanical follow-on swap unlocked by K-10 (substrate 35e67f18 + 0083 applied to prod) + KR-7a's MCP client transport. create_relationlink body replaces raise NotAvailableError with await mcp_client.invoke('kora__create_relationlink', {...}). Production-test posture: K-10's kora__create_relationlink handler is notImplementedHandler stub on substrate main; dispatch tier (task NousResearch#395) un-stubs. Forward-stability touch: added rationale_block_id + evidence_block_ids params matching K-10 input schema; legacy rationale param preserved for one-release back-compat (silently dropped — superseded by rationale_block_id). Original exception class kept exported tagged [kora.isokron.deprecated]. Verify-at-first-live-emit: event_log.actor_id resolves to 0076-seeded canonical Kora actor; public.relationlink row chain_event_id matches; created_by_actor_kind='kora'. ty zero-delta; 357/357 plugins/memory tests pass. Cherry-pick rebased onto post-KR-8 main. ALL 4 Kora-runtime BUILD_DEVIATIONS now closed code-side.
Egavasyug pushed a commit to Egavasyug/hermes-agent that referenced this pull request Jun 10, 2026
…for transcript message extraction

Authored by PercyDikec. Fixes NousResearch#394.

The transcript extraction used len(history) to find new messages, but
history includes session_meta entries stripped before reaching the agent.
This caused 1 message lost per turn from turn 2 onwards. Fix returns
history_offset (filtered length) from _run_agent and uses it for the slice.
Egavasyug pushed a commit to Egavasyug/hermes-agent that referenced this pull request Jun 10, 2026
The error return (no final_response) was missing history_offset,
falling back to len(history) which has the same session_meta offset
bug fixed in PR NousResearch#395. Now both return paths include the correct
filtered history length.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Gateway transcript loses 1 message per turn due to session_meta offset mismatch

2 participants