[Agent Builder] Agent skills implementation by KDKHD · Pull Request #251209 · elastic/kibana

KDKHD · 2026-02-02T10:38:35Z

Summary

Add Skills Support to Filestore

Fix https://github.com/elastic/search-team/issues/12744
Fix https://github.com/elastic/search-team/issues/12655

Summary

Adds skills support to the filestore system introduced in #250043. Skills are stored as file entries in a virtual filesystem, enabling agents to discover and access skill instructions via standard filestore operations (read, ls, glob, grep).

Changes

SkillsStore Implementation (skills_store.ts):
- SkillsStoreImpl manages skills in a MemoryVolume.
- Converts SkillTypeDefinition objects to file entries.
- Supports add/delete operations with volume synchronization.
- Provides a readonly interface for safe access.
Skills Utilities (utils.ts):
- createSkillEntries(): Converts skills to FileEntry objects with metadata.
- getSkillEntryPath(): Generates standardized paths for skill files.
- getSkillReferencedContentEntryPath(): Handles referenced content paths.
- getSkillPlainText(): Formats skills with frontmatter.
- isSkillFileEntry(): Type guard for skill file entries.
Skills Prompts (prompts.ts):
- getSkillsInstructions(): Generates XML-formatted skill listings for agent prompts.
- Discovers skills via glob pattern matching.
- Sorts skills by path for consistent ordering.
Store Integration (create_store.ts):
- Mounts skills volume into the virtual filesystem.
- Makes skills accessible through the filestore API.
Type Definitions (types.ts):
- SkillFileEntry: File entry type for skill files.
- SkillReferencedContentFileEntry: File entry type for referenced content.

Skill-Bound Tools

Dynamic Tool Binding: Skills can define associated tools via getAllowedTools and getInlineTools.
Automatic Provisioning: When an agent reads a skill file from the filestore, the associated tools are automatically bound to the agent's current session, making them available for immediate use in the conversation.

Technical Details

Skills are stored as SKILL.md files in paths like skills/{basePath}/{name}/SKILL.md.
Referenced content is stored alongside skills with proper path resolution.
File entries include metadata (skill name, description, ID, token count).
Skills are marked as readonly in the filestore.
Integration with existing FileEntryType.skill enum.

Related PRs

Builds on the filestore infrastructure from #250043.

Checklist

Check the PR satisfies following conditions.

Reviewers should verify this PR satisfies this list as well.

Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
Documentation was added for features that require explanation or tutorials
Unit or functional tests were updated or added to match the most common scenarios
If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the docker list
This was checked for breaking HTTP API changes, and any breaking changes have been approved by the breaking-change committee. The release_note:breaking label should be applied in these situations.
Flaky Test Runner was used on any tests changed
The PR description includes the appropriate Release Notes section, and the correct release_note:* label is applied per the guidelines
Review the backport guidelines and apply applicable backport:* labels.

Identify risks

Does this PR introduce any risks? For example, consider risks like hard to test bugs, performance regression, potential of data loss.

Describe the risk, its severity, and mitigation for each identified risk. Invite stakeholders and evaluate how to proceed before merging.

See some risk examples
...

KDKHD · 2026-02-02T14:37:02Z

...tions/security/plugins/security_solution/server/agent_builder/skills/alert_analysis_skill.ts

+/**
+ * DISCLAIMER: This skill is a sample skill.
+ *
+ * Threat analysis skill for security operations.
+ * This skill helps analyze security threats, investigate alerts, and assess risk.
+ */


Replace this skill example with a better one

We should remove that skill yeah, or at least remove the registration from the security plugin ihmo (or at least remove the inline tool with stub data)

unregistered here: https://github.com/elastic/kibana/pull/251209/files#diff-d8d3f1d06e2eddeae71eee2c5702d15155a10157588ea0ad9054654c36710324R14

KDKHD · 2026-02-02T14:59:14Z

x-pack/platform/plugins/shared/agent_builder/server/services/runner/store/tools/read.ts

+      if (isSkillFileEntry(entry)) {
+        const skill = skillsService.getSkillDefinition(entry.metadata.skill_id);
+        if (skill) {
+          const inlineTools = (await skill.getInlineTools?.()) ?? [];
+          const inlineExecutableTools = inlineTools.map((tool) =>
+            skillsService.convertSkillTool(tool)
+          );
+
+          const allowedTools = skill.getAllowedTools?.() ?? [];
+          const registryExecutableTools = await pickTools({
+            toolProvider,
+            selection: [{ tool_ids: allowedTools }],
+            request,
+          });
+
+          await toolManager.addTool(
+            {
+              type: ToolManagerToolType.executable,
+              tools: [...inlineExecutableTools, ...registryExecutableTools],
+              logger,
+            },
+            {
+              dynamic: true,
+            }
+          );
+        } else {
+          logger.debug(`Skill '${entry.metadata.skill_id}' not found in registry.`);
+        }
+      }


This is the code that adds skill tools to the tool manager. I think it is clearest if this logic lives here. Maybe eventually it should be moved into a hook, though.

Agreed, reading the code before your comment I was thinking the exact same thing - should live in a hook ideally. But also agreed that until we do have hooks, this is the best place for the logic to live.

NIT: I would however extract the logic living in that if (isSkillFileEntry(entry)) block to a dedicated static loadSkill helper already.

KDKHD · 2026-02-02T18:07:00Z

x-pack/platform/plugins/shared/agent_builder/server/services/conversation/client/converters.ts

      ...base,
      rounds: roundsWithRefs,
      attachments: existingAttachments,
+      ...(document._source!.state && { state: document._source!.state }),


Previously state was only returned by the final case. Assuming it is alright to return state when hasLegacyRoundAttachments or existingAttachments && existingAttachments.length > 0 is true.

KDKHD · 2026-02-03T09:31:41Z

x-pack/platform/plugins/shared/agent_builder/server/services/tool_manager/tool_manager.ts

+export const createToolManager = (): ToolManager => {
+  return new ToolManager({
+    dynamicToolCapacity: 10,
+  });
+};


Currently, the tool manager has a dynamic tool capacity of 10. That means, if an 11th tool is added to the tool manager due to a skill being loaded, the least recently used tool would be evicted, and the agent wouldn't be able to call that tool anymore. Dynamic tool IDs are persisted at the end of a round, and the next round is initialised with the same dynamic tools so the agent could call those tools again without needing to load the skill.

Maybe this logic should be changed so it works like this: ToolManager has no (or a much higher) dynamic tool capacity, which would essentially result in no tools ever getting evicted within a round. To ensure the number of tools bound does not get out of hand, I could add a constraint that limits the number of tools carried over to the next round to, e.g. the 5 most recently used tools.

Or maybe we don't have to carry over tools at all.

I think what you did is fine for an initial implementation. Agreed that we may want to get to a point where the set of dynamic tools "kept" in current round isn't the same as the list we "store" for re-use in the next rounds. As you said, we may even not need to carry over tools at all, given we may want to assume that the agent will read the skill again if necessary in next rounds.

But tldr, the approach you use is fine for now ihmo.

pgayvallet

Implementation looks good, solid for a first milestone.

A few remarks, and a bunch of NITs, but overall no major concerns or problems.

Pre-approving (don't feel forced to address all the NITs, only the ones which make sense to you)

The things I think would be good to address before merging:

replacing the LRUMap implementation with the lru-cache package comment
implementing "raw" content in the skill's file entry comment
disabling/removing the "test" skill from security comment

pgayvallet · 2026-02-03T11:42:20Z

x-pack/platform/packages/shared/agent-builder/agent-builder-server/runner/skills_store.ts

+import type { SkillTypeDefinition } from '../skills';
+
+/**
+ * Store to access tool results during execution


NIT: typo "tool results" -> "skills" (+ below too)

pgayvallet · 2026-02-03T11:45:32Z

x-pack/platform/packages/shared/agent-builder/agent-builder-server/runner/tool_manager.ts

+   * @param input - The tool input configuration (executable or browser)
+   * @param options - Optional configuration for tool storage (static vs dynamic)
+   */
+  addTool(input: AddToolInput, options?: AddToolOptions): Promise<void>;


NIT: addTools given the input defines multiple

pgayvallet · 2026-02-03T11:46:42Z

x-pack/platform/packages/shared/agent-builder/agent-builder-server/runner/tool_manager.ts

+export type {
+  ToolManager,
+  ToolManagerParams,
+  ToolName,
+  AddToolOptions,
+  ExecutableToolInput,
+  BrowserToolInput,
+  AddToolInput,
+};


NIT: consistency, use export directly on definitions

pgayvallet · 2026-02-03T11:51:40Z

x-pack/platform/packages/shared/agent-builder/agent-builder-server/skills/tools.ts

+export type SkillBoundedTool<RunInput extends ZodObject<any> = ZodObject<any>> =
+  | BuiltinSkillBoundedTool<RunInput>
+  | StaticEsqlSkillBoundedTool
+  | IndexSearchSkillBoundedTool
+  | WorkflowSkillBoundedTool;


Note for myself: need to factorize that with the attachment bounded tools later (perfectly fine to keep it that way in the PR, I'll handle it later)

pgayvallet · 2026-02-03T11:56:14Z

x-pack/platform/packages/shared/agent-builder/agent-builder-server/skills/type_definition.ts

+/**
+ * Skill directory structure - explicit about how skills are organized.
+ *
+ * The purpose of this is to encourage skills to be well organized, grouped logically and consistently.
+ * This also requires code owners to review directory changes to ensure skills are discoverable.
+ *
+ * In order to store skills in a new directory, you need to add the directory to the structure.
+ */
+export type SkillsDirectoryStructure = Directory<{
+  skills: Directory<{
+    platform: FileDirectory;
+    observability: FileDirectory<{}>;
+    security: FileDirectory<{
+      alerts: FileDirectory<{
+        rules: FileDirectory;
+      }>;
+      entities: FileDirectory;
+    }>;
+    search: FileDirectory<{}>;
+  }>;
+}>;


I agree about having guideline and a "strict" structure to avoid chaos, but idk if the hardcoded structure type is the right approach (death by a thousand review request kind of thing).

Fine keeping it that way for now though, we'll see later if we need to revisit.

(the DirectoryPath type util is quite nice though, I like it)

pgayvallet · 2026-02-03T14:59:14Z

x-pack/platform/plugins/shared/agent_builder/server/services/skills/prompts.ts

+          '<available_skills>',
+          ...skillsFileEntries.flatMap((skillFileEntry) => [
+            `    <skill path="${skillFileEntry.path}">`,
+            `      <name>${skillFileEntry.metadata.skill_name}</name>`,
+            `      <description>${skillFileEntry.metadata.skill_description}</description>`,
+            `    </skill>`,
+          ]),
+          '</available_skills>',


NIT: consistency - please use generateXmlTree for xml formatted content (x-pack/platform/packages/shared/agent-builder/agent-builder-genai-utils/tools/utils/formatting/xml.ts)

pgayvallet · 2026-02-03T15:01:26Z

x-pack/platform/plugins/shared/agent_builder/server/services/skills/prompts.ts

+  const fileEntries = await filesystem.glob('/**/SKILL.md');
+  const skillsFileEntries = fileEntries
+    .filter(isSkillFileEntry)
+    .toSorted((a, b) => a.path.localeCompare(b.path));


Is there an upside retrieving the skill via the filestore here? I would have used the skill service / store "directly" instead, even if that doesn't change much

No real benefit or difference to doing it with the filestore. I did it like this just to be confident that only the skills that actually exist in the volume are added into the system prompt.

pgayvallet · 2026-02-03T15:10:49Z

x-pack/platform/plugins/shared/agent_builder/server/services/skills/skill_type_registry.ts

+  return new SkillTypeRegistryImpl();
+};
+
+class SkillTypeRegistryImpl implements SkillTypeRegistry {


NIT: why skillTypeRegistry?

pgayvallet · 2026-02-03T15:22:17Z

x-pack/platform/plugins/shared/agent_builder/server/services/tool_manager/tool_manager.ts

+export const createToolManager = (): ToolManager => {
+  return new ToolManager({
+    dynamicToolCapacity: 10,
+  });
+};


I think what you did is fine for an initial implementation. Agreed that we may want to get to a point where the set of dynamic tools "kept" in current round isn't the same as the list we "store" for re-use in the next rounds. As you said, we may even not need to carry over tools at all, given we may want to assume that the agent will read the skill again if necessary in next rounds.

But tldr, the approach you use is fine for now ihmo.

pgayvallet · 2026-02-03T15:36:03Z

...tions/security/plugins/security_solution/server/agent_builder/skills/alert_analysis_skill.ts

+/**
+ * DISCLAIMER: This skill is a sample skill.
+ *
+ * Threat analysis skill for security operations.
+ * This skill helps analyze security threats, investigate alerts, and assess risk.
+ */


We should remove that skill yeah, or at least remove the registration from the security plugin ihmo (or at least remove the inline tool with stub data)

elasticmachine · 2026-02-04T17:36:09Z

⏳ Build in-progress, with failures

Buildkite Build
Commit: 6b1f94a

Failed CI Steps

Test Failures

[job] [logs] FTR Configs #116 / "before all" hook in "{root}"
[job] [logs] FTR Configs #66 / Serverless Observability - Deployment-agnostic Streams API integration tests Streams Endpoints Failure Store Classic streams failure store "after each" hook for "inherit defaults to disabled for classic streams without index template failure store configuration"
[job] [logs] Scout: [ platform / streams_app-serverless-oblt ] plugin / serverless-oblt - Stream data processing - data sources management - should allow adding a new kql data source
[job] [logs] Scout: [ platform / streams_app-serverless-oblt ] plugin / serverless-oblt - Stream data processing - data sources management - should allow adding a new kql data source
[job] [logs] Scout: [ platform / streams_app-stateful ] plugin / should allow adding a new kql data source
[job] [logs] Scout: [ platform / streams_app-serverless-oblt ] plugin / should allow adding a new kql data source
[job] [logs] Scout: [ platform / streams_app-stateful ] plugin / should allow adding a new kql data source
[job] [logs] Scout: [ platform / streams_app-serverless-oblt ] plugin / should allow adding a new kql data source
[job] [logs] Scout: [ platform / streams_app-stateful ] plugin / stateful - Stream data processing - data sources management - should allow adding a new kql data source
[job] [logs] Scout: [ platform / streams_app-stateful ] plugin / stateful - Stream data processing - data sources management - should allow adding a new kql data source

History

💔 Build #389838 failed 4a2c2c2
💔 Build #389778 failed e2db8fb
💔 Build #389758 failed 49b20da
💛 Build #388860 was flaky 81e74b7

* commit '5c0872d56bc0268177cd3c7150a1685481fb5238': (221 commits) Add .cursorignore file (elastic#251709) [Search] Add descriptions to semantic_text field inference endpoint select (elastic#249265) [Agent Builder] Agent skills implementation (elastic#251209) [Lens] [ES|QL] Improve types for ES|QL conversion. (elastic#251042) Update the trace waterfall to make it easy to understand (elastic#250442) [ES|QL] [Lens] Adds query stats (elastic#251029) [Lens] Fix KQL character escaping when query is generated from Top values column (breakdown). (elastic#250925) fix(kbn-elastic-assistant): fix a11y issue with missing label on flyout (elastic#251656) Update dependency @elastic/monaco-esql to v3.1.16 (main) (elastic#251666) [Automatic Import V2] Add langsmith tracing (elastic#251592) [scout] fix duplicated test failure reports in Buildkite annotations (elastic#251455) chore(NA): remove us-central1-b from gcp zones on high load jobs (elastic#251748) skip flaky suite (elastic#250973) [Lens] Allow read only view for users with write permissions but having no write access to the dashboard (elastic#247746) [CI] Increase artifacts disk to 180gb (elastic#251774) [content-list] 1. Provider Foundation (elastic#251344) [AI Infra] Add missing ES|QL commands and functions documentation for inference tasks (elastic#249089) [docs-utils] 4️⃣ pre-req: Prepare for new validations (elastic#250810) [APM] Extend React flow service map test coverage (elastic#251624) [scout] discover tests with custom server configs (elastic#251297) ... # Conflicts: # src/platform/plugins/shared/dashboard/tsconfig.json # x-pack/platform/plugins/shared/agent_builder_platform/server/tools/create_visualization/create_visualization.ts

pgayvallet added 30 commits January 21, 2026 09:40

add memfs to deps

3c18b6c

work in progress

e4879d6

I totally did that alone

c1bca2f

move FS files, adapt tool result store and runner

b136796

Merge remote-tracking branch 'upstream/main' into ab-12348-fs-store

0f22dcd

preparing store interface

9942e7e

implement fs_store, update read tool

d0d69ff

create base set of tools

710fe39

move types to server package

f71fe46

fix unit tests

97f1663

plug FS tools to agent

5e92995

Merge remote-tracking branch 'upstream/main' into ab-12348-fs-store

f97f223

remove memfs

477cef4

use constants for tool names

0b029fb

fix mocks

c17e3e8

export class to please TS

77bf6a0

add folder tree util

42402db

cleanup agent execution

5e51506

move timestamp to prompt factory

5351aaf

clean exports

0fd234b

fix types again

1b0f548

Merge remote-tracking branch 'upstream/main' into ab-12348-fs-store

8534c6d

rename filesystem to filestore + cleanup

d716cd4

change namespace

36dcac4

remove duplicate instructions

72fbe61

add kill switch

4a92f7d

add filestore to descriptions

1a1e930

update prompt

c7ffc30

update prompt again

447200b

last time

e9ce8e4

KDKHD commented Feb 2, 2026

View reviewed changes

KDKHD changed the title ~~[Agent Builder] skills: initial implementation~~ [Agent Builder] Agent skills Feb 2, 2026

KDKHD changed the title ~~[Agent Builder] Agent skills~~ [Agent Builder] Agent skills initial implementation Feb 2, 2026

KDKHD marked this pull request as ready for review February 2, 2026 14:38

KDKHD requested review from a team as code owners February 2, 2026 14:38

KDKHD commented Feb 2, 2026

View reviewed changes

pgayvallet self-requested a review February 2, 2026 15:12

fix dynamic tool carry over & limit to 7 skill tools

81e74b7

KDKHD force-pushed the feature/agent-builder-skills2 branch from cbe5029 to 81e74b7 Compare February 2, 2026 18:05

KDKHD commented Feb 2, 2026

View reviewed changes

KDKHD added release_note:skip Skip the PR/issue when compiling release notes backport:skip This PR does not require backporting labels Feb 3, 2026

KDKHD commented Feb 3, 2026

View reviewed changes

pgayvallet approved these changes Feb 3, 2026

View reviewed changes

code review

2d116d3

KDKHD requested a review from a team as a code owner February 4, 2026 10:43

KDKHD and others added 4 commits February 4, 2026 10:51

Merge branch 'main' into feature/agent-builder-skills2

49b20da

Changes from node scripts/eslint_all_files --no-cache --fix

e2db8fb

fix types

4a2c2c2

fix test

6b1f94a

Merge branch 'main' into feature/agent-builder-skills2

1d674fc

patrykkopycinski approved these changes Feb 5, 2026

View reviewed changes

joemcelroy approved these changes Feb 5, 2026

View reviewed changes

KDKHD changed the title ~~[Agent Builder] Agent skills initial implementation~~ [Agent Builder] Agent skills implementation Feb 5, 2026

KDKHD merged commit acf6780 into elastic:main Feb 5, 2026
17 checks passed

kibanamachine added the v9.4.0 label Feb 5, 2026

Conversation

KDKHD commented Feb 2, 2026 • edited by pgayvallet Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Add Skills Support to Filestore

Summary

Changes

Skill-Bound Tools

Technical Details

Related PRs

Checklist

Identify risks

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

KDKHD Feb 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pgayvallet left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pgayvallet Feb 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

elasticmachine commented Feb 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⏳ Build in-progress, with failures

Failed CI Steps

Test Failures

History

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

KDKHD commented Feb 2, 2026 •

edited by pgayvallet

Loading

KDKHD Feb 3, 2026 •

edited

Loading

pgayvallet left a comment •

edited

Loading

pgayvallet Feb 3, 2026 •

edited

Loading

elasticmachine commented Feb 4, 2026 •

edited

Loading