Fix find_last_user_message_index skipping skill content injections#21119
Merged
alkispoly-db merged 1 commit intomlflow:masterfrom Feb 27, 2026
Merged
Fix find_last_user_message_index skipping skill content injections#21119alkispoly-db merged 1 commit intomlflow:masterfrom
alkispoly-db merged 1 commit intomlflow:masterfrom
Conversation
Contributor
🛠 DevTools 🛠
Install mlflow from this PRFor Databricks, use the following command: |
Contributor
|
Documentation preview for 7b45a6c is available at: More info
|
378eaa8 to
954c7a9
Compare
smoorjani
approved these changes
Feb 27, 2026
mlflow/claude_code/tracing.py
Outdated
| @@ -222,6 +222,15 @@ def find_last_user_message_index(transcript: list[dict[str, Any]]) -> int | None | |||
| for i in range(len(transcript) - 1, -1, -1): | |||
| entry = transcript[i] | |||
| if entry.get(MESSAGE_FIELD_TYPE) == MESSAGE_TYPE_USER and not entry.get("toolUseResult"): | |||
Collaborator
There was a problem hiding this comment.
tiny nit: let's move toolUseResult (and maybe commandName) to constants
When Claude Code invokes a skill, the transcript contains a skill content injection — a user message with text content that immediately follows the skill tool result. This message passes all existing filters in `find_last_user_message_index()`, causing the trace input to show thousands of characters of skill text instead of the actual user prompt. Detect and skip these injections by checking whether the previous transcript entry is a skill tool result (a user message with `toolUseResult` containing `commandName`). Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Alkis Polyzotis <alkis.polyzotis@databricks.com>
954c7a9 to
7b45a6c
Compare
daniellok-db
pushed a commit
to daniellok-db/mlflow
that referenced
this pull request
Mar 5, 2026
…lflow#21119) Signed-off-by: Alkis Polyzotis <alkis.polyzotis@databricks.com> Co-authored-by: Claude <noreply@anthropic.com>
daniellok-db
pushed a commit
to daniellok-db/mlflow
that referenced
this pull request
Mar 5, 2026
…lflow#21119) Signed-off-by: Alkis Polyzotis <alkis.polyzotis@databricks.com> Co-authored-by: Claude <noreply@anthropic.com>
daniellok-db
pushed a commit
that referenced
this pull request
Mar 5, 2026
…21119) Signed-off-by: Alkis Polyzotis <alkis.polyzotis@databricks.com> Co-authored-by: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Related Issues/PRs
#xxxWhat changes are proposed in this pull request?
find_last_user_message_index()incorrectly identifies skill content injection messages as the last user prompt. When Claude Code invokes a skill, the transcript contains ausermessage with the full skill instructions that passes all existing filters (notoolUseResult, content type istext, not empty). The trace input then shows thousands of characters of skill text instead of the actual user prompt.This PR adds a check that detects skill content injections by inspecting the previous transcript entry. If the previous entry is a skill tool result (a
usermessage withtoolUseResultcontainingcommandName), the current entry is skipped.How is this PR tested?
Three new tests for
find_last_user_message_index:test_find_last_user_message_skips_skill_injection— single skill invocation, verifies the function returns the actual user prompttest_find_last_user_message_index_basic— normal multi-turn conversation without skills (no false positives)test_find_last_user_message_skips_consecutive_skill_injections— two back-to-back skill invocations, verifies both injections are skippedDoes this PR require documentation update?
Does this PR require updating the MLflow Skills repository?
Release Notes
Is this a user-facing change?
Fix Claude Code tracing to correctly identify the user prompt when skills are invoked, instead of capturing skill injection content as the trace input.
What component(s), interfaces, languages, and integrations does this PR affect?
Components
area/tracing: MLflow Tracing features, tracing APIs, and LLM tracing functionalityHow should the PR be classified in the release notes? Choose one:
rn/bug-fix- A user-facing bug fix worth mentioning in the release notesShould this PR be included in the next patch release?
🤖 Generated with Claude Code