Skip to content

fix(deps): add hindsight-client as optional dependency#9194

Closed
jbotwell wants to merge 7 commits into
NousResearch:mainfrom
jbotwell:fix/hindsight-optional-dep
Closed

fix(deps): add hindsight-client as optional dependency#9194
jbotwell wants to merge 7 commits into
NousResearch:mainfrom
jbotwell:fix/hindsight-optional-dep

Conversation

@jbotwell

Copy link
Copy Markdown

Title: fix(deps): add hindsight-client as optional dependency

Body:

What does this PR do?

Adds hindsight-client as an optional dependency ([hindsight] extra) so it can be included at build time by package managers that pre-build the venv (NixOS/uv2nix, Docker, Homebrew, etc.

Currently the hindsight memory plugin lists hindsight-client>=0.4.22 in plugins/memory/hindsight/plugin.yaml which triggers a runtime pip install. This works for standard `pip install

Related Issue

Fixes #8873

Type of Change

  • Bug fix (non-breaking change that fixes an issue)

Changes Made

  • pyproject.toml: Added hindsight = ["hindsight-client>=0.4.22"] optional dependency group, matching the version already required by the plugin
  • pyproject.toml: Added hermes-agent[hindsight] to the all extra
  • uv.lock: Regenerated — adds hindsight-client v0.5.0 and its transitive deps

How to Test

  1. uv sync --extra hindsight — verify hindsight-client is installed
  2. uv sync --extra all — verify hindsight-client is included
  3. Configure hindsight as memory provider (hermes config set memory.provider hindsight) and start a session — the hindsight_client import should resolve without runtime pip

Tested on NixOS 26.05 with a uv2nix-built venv including the [hindsight] extra. Hindsight memory provider initializes and works correctly.

Checklist

Code

  • I've read the Contributing Guide
  • My commit messages follow Conventional Commits (fix(deps):)
  • I searched for existing PRs to make sure this isn't a duplicate
  • My PR contains only changes related to this fix/feature (no unrelated commits)
  • I've run pytest tests/ -q and all tests pass
  • I've added tests for my changes (not applicable — dependency declaration only)
  • I've tested on my platform: NixOS 26.05 (uv2nix)

Documentation & Housekeeping

  • N/A — no config keys, architecture changes, or tool behavior changes

@NateH2O

NateH2O commented Apr 14, 2026

Copy link
Copy Markdown

I'm running the agent using the official docker image and hindsight in another container (local external) and can't find a way to get hindsight-client to install on boot. Hoping this solves it.

@mrhwick

mrhwick commented Apr 15, 2026

Copy link
Copy Markdown
Contributor

Thanks for working on this. I wanted to add a data point from an immutable container/Kubernetes deployment use case, because this is a real packaging problem for us.

We run Hermes as a derived container image built from nousresearch/hermes-agent:<tag> and then deploy that image into Kubernetes. In that model, dependencies for supported features need to be present at image build time rather than installed after startup.

Our current workaround is to bake hindsight-client into the wrapper image explicitly:

FROM nousresearch/hermes-agent:v2026.4.8

RUN python3 -m pip install --no-cache-dir --break-system-packages "hindsight-client==0.5.0"

That has worked well for us. It makes hindsight support reproducible and keeps restarts and rollbacks predictable.

From our perspective, the important requirement is:

If memory.provider = hindsight is part of the supported container story, there should be a supported way to build an image where hindsight-client is already present before the container starts.

Why this matters for us:

  • We do not want to rely on runtime pip install in a running container.
  • We want the image artifact itself to describe the full dependency set.
  • We want restart behavior to be independent of writable runtime state or manual operator changes.
  • This is especially important in Kubernetes and other environments with ephemeral runtime filesystems.

I do not have a strong opinion on whether the right answer is this PR specifically (it seems to be slightly incomplete?) or some adjacent packaging change, but I do want to confirm that the underlying problem is real and that a solution for build-time inclusion would solve it cleanly for our deployment model.

Comment thread pyproject.toml Outdated
@NateH2O

NateH2O commented Apr 18, 2026

Copy link
Copy Markdown

This looks related to support thread: "Hermes Docker does not contain hindsight pip package" in discord
https://discord.com/channels/1053877538025386074/1494880345915654246/1494880345915654246

@jbotwell jbotwell force-pushed the fix/hindsight-optional-dep branch from 599e86e to cae1928 Compare April 19, 2026 01:12
@jbotwell

Copy link
Copy Markdown
Author

Ok. I just went ahead and regenerated the uv lock and commited on top of a more recent commit. In terms of it being incomplete, this is all I need to unblock what I am trying to do, but happy to look at other stuff that would need to happen if I'm missing anything

@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/plugins Plugin system and bundled plugins tool/memory Memory tool and memory providers area/nix Nix flake, NixOS module, container packaging labels Apr 22, 2026
@jbotwell jbotwell closed this May 4, 2026
@descention

descention commented May 5, 2026

Copy link
Copy Markdown

@jbotwell Why did this get closed?

@jbotwell

jbotwell commented May 5, 2026

Copy link
Copy Markdown
Author

@descention , It was waiting for a maintainer review for a long time, and in the meantime they added a services.hermes-agent.extraPythonPackages option that allows nix people to include Python packages declaratively without needing to be in container mode. I figured that was going to be the preferred way to do it and I interpreted the maintainer review lag as reluctance to put that dependency in the pyproject. All that said, I'm happy to reopen it if you think it makes sense

@descention

descention commented May 5, 2026

Copy link
Copy Markdown

@jbotwell I'd prefer to see it natively supported. I don't like having to manually add additional packages to use built-in features. Your PR addresses that well.

@jbotwell jbotwell reopened this May 6, 2026
@jbotwell

jbotwell commented May 6, 2026

Copy link
Copy Markdown
Author

Cool. Should be good to go again? It'll keep getting into conflicts with uv.lock, though. Easy to fix, but heads-up that that divergence is a thing

@teknium1

teknium1 commented May 7, 2026

Copy link
Copy Markdown
Contributor

@alt-glitch can you take a look at this one?

Putting hindsight-client into the [all] extra means every user doing pip install 'hermes-agent[all]' pulls hindsight-client in, even though the vast majority aren't using hindsight as their memory provider. We'd rather not require it in everyone's install just to fix the NixOS build path.

Would prefer to keep the hindsight extra itself (that's the right fix for #8873 — packagers can opt in at build time), but drop it from [all]. NixOS/uv2nix/Docker users who want hindsight can add [hindsight] to their own extras list. Same pattern we use for modal, daytona, vercel, rl, yc-bench — present as extras, absent from [all].

@jbotwell

jbotwell commented May 7, 2026

Copy link
Copy Markdown
Author

@teknium1 , this is all:

all = [
  "hermes-agent[modal]",
  "hermes-agent[daytona]",
  "hermes-agent[vercel]",
  "hermes-agent[messaging]",
  # matrix: python-olm (required by matrix-nio[e2e]) is upstream-broken on
  # modern macOS (archived libolm, C++ errors with Clang 21+).  On Linux the
  # [matrix] extra's own marker pulls in the [e2e] variant automatically.
  "hermes-agent[matrix]; sys_platform == 'linux'",
  "hermes-agent[cron]",
  "hermes-agent[cli]",
  "hermes-agent[dev]",
  "hermes-agent[tts-premium]",
  "hermes-agent[slack]",
  "hermes-agent[pty]",
  "hermes-agent[hindsight]",
  "hermes-agent[honcho]",
  "hermes-agent[mcp]",
  "hermes-agent[homeassistant]",
  "hermes-agent[sms]",
  "hermes-agent[acp]",
  "hermes-agent[voice]",
  "hermes-agent[dingtalk]",
  "hermes-agent[feishu]",
  "hermes-agent[google]",
  "hermes-agent[mistral]",
  "hermes-agent[bedrock]",
  "hermes-agent[web]",
]

modal, daytona, and vercel are in all. There seem to be a number of packages in there that are only likely to be useful for a subset of users. I mean, I agree it would make sense to have packages that are declarable on their own without having to pull in a whole universe of packages but I'm a little fuzzy on the criteria for a package being in that list, though. Currently I'm grabbing hindsight by just declaring its build in my nix config and putting it in extraPythonPackages (which is super useful, so I'm glad that was added). I don't think there's a way to declare which pyproject dependency groups are passed to the virtual environment in nix, though, is there? As per a conversation with hermes:

Yes, the infrastructure is there but it's not wired through. Here's what I found:

nix/python.nix already accepts dependency-groups ? [ "all" ] and passes it to mkVirtualEnv:

pythonSet.mkVirtualEnv "hermes-agent-env" {
  hermes-agent = dependency-groups;
}

But nix/hermes-agent.nix doesn't expose it — it calls python.nix without passing dependency-groups, so it always defaults to [ "all" ]:

hermesVenv = callPackage ./python.nix {
  inherit uv2nix pyproject-nix pyproject-build-systems;
  # dependency-groups is NOT passed — stuck at default ["all"]
};

if that understanding is flawed or there is a different way we're not thinking of, please correct me. If not, it might be that another issue/PR would be useful to allow dependency groups to be declared

@benfeather

Copy link
Copy Markdown

Is this PR going to be merged? I'm having this exact issue with the Hermes agent on NixOS.

@alt-glitch

alt-glitch commented May 8, 2026

Copy link
Copy Markdown
Collaborator

Hey @jbotwell, @descention, @mrhwick! Thanks for the patience on this one and for pushing on the issue. The discussion here (especially the comment about dependency-groups not being wired through) was really helpful in clarifying what the right fix actually looks like.

We've split this into two PRs that solve the problem without polluting [all]:

1. #21818feat(deps): add hindsight-client as optional dependency

Adds hindsight = ["hindsight-client>=0.4.22"] to pyproject.toml optional deps. Deliberately NOT in [all] — memory providers are plugins, and if we let hindsight in then supermemory, mem0, openviking all have equal claim and [all] becomes unbounded. This gives Docker/k8s users:

pip install hermes-agent[hindsight]

2. #21817feat(nix): add extraDependencyGroups for sealed venv extras

This addresses the gap @jbotwell identified — dependency-groups existed in python.nix but wasn't wired through hermes-agent.nix. Now it is. NixOS users can opt into any pyproject.toml extra inside the sealed venv (single uv resolver pass, no PYTHONPATH append, no collision checker):

services.hermes-agent.extraDependencyGroups = [ "hindsight" ];

This is cleaner than the extraPythonPackages workaround because uv resolves the full graph in one pass — no risk of transitive dep collisions with the sealed venv.


The two together give every environment a clean path:

  • Nix: extraDependencyGroups = [ "hindsight" ]
  • Docker/k8s: pip install hermes-agent[hindsight]
  • Regular pip/uv: uv sync --extra hindsight

Would appreciate your eyes on both if you get a chance — especially @jbotwell since you traced the dependency-groups wiring gap that led to #21817.

@jbotwell

jbotwell commented May 8, 2026

Copy link
Copy Markdown
Author

Hey @jbotwell, @descention, @mrhwick! Thanks for the patience on this one and for pushing on the issue. The discussion here (especially the comment about dependency-groups not being wired through) was really helpful in clarifying what the right fix actually looks like.

We've split this into two PRs that solve the problem without polluting [all]:

1. #21818feat(deps): add hindsight-client as optional dependency

Adds hindsight = ["hindsight-client>=0.4.22"] to pyproject.toml optional deps. Deliberately NOT in [all] — memory providers are plugins, and if we let hindsight in then supermemory, mem0, openviking all have equal claim and [all] becomes unbounded. This gives Docker/k8s users:

pip install hermes-agent[hindsight]

2. #21817feat(nix): add extraDependencyGroups for sealed venv extras

This addresses the gap @jbotwell identified — dependency-groups existed in python.nix but wasn't wired through hermes-agent.nix. Now it is. NixOS users can opt into any pyproject.toml extra inside the sealed venv (single uv resolver pass, no PYTHONPATH append, no collision checker):

services.hermes-agent.extraDependencyGroups = [ "hindsight" ];

This is cleaner than the extraPythonPackages workaround because uv resolves the full graph in one pass — no risk of transitive dep collisions with the sealed venv.

The two together give every environment a clean path:

* **Nix**: `extraDependencyGroups = [ "hindsight" ]`

* **Docker/k8s**: `pip install hermes-agent[hindsight]`

* **Regular pip/uv**: `uv sync --extra hindsight`

Would appreciate your eyes on both if you get a chance — especially @jbotwell since you traced the dependency-groups wiring gap that led to #21817.

Yeah, took a look at those. No issues jumping out at me. I think the separation makes a lot more sense. Thanks!

@alt-glitch

Copy link
Copy Markdown
Collaborator

Closed in favour of: #21818
Thank you @jbotwell @descention @mrhwick for the help!

@alt-glitch alt-glitch closed this May 11, 2026
alt-glitch added a commit that referenced this pull request May 11, 2026
Expose the dependency-groups parameter from python.nix through
hermes-agent.nix and the NixOS module, allowing users to opt into
pyproject.toml optional extras (e.g. hindsight, voice, matrix) that
are resolved by uv inside the sealed venv.

Unlike extraPythonPackages (which appends to PYTHONPATH and requires
collision checking), extraDependencyGroups resolves the full dependency
graph in a single uv pass — no PYTHONPATH patching, no version
conflicts, no collision risk.

When to use which:
- extraDependencyGroups: enable a pyproject.toml optional extra
- extraPythonPackages: add an external Python plugin not in pyproject.toml

Usage:
  services.hermes-agent.extraDependencyGroups = [ "hindsight" ];

Or via overlay:
  pkgs.hermes-agent.override { extraDependencyGroups = [ "hindsight" ]; }

Refs: #8873, #9194
rmulligan pushed a commit to rmulligan/hermes-agent that referenced this pull request May 11, 2026
…arch#21817)

Expose the dependency-groups parameter from python.nix through
hermes-agent.nix and the NixOS module, allowing users to opt into
pyproject.toml optional extras (e.g. hindsight, voice, matrix) that
are resolved by uv inside the sealed venv.

Unlike extraPythonPackages (which appends to PYTHONPATH and requires
collision checking), extraDependencyGroups resolves the full dependency
graph in a single uv pass — no PYTHONPATH patching, no version
conflicts, no collision risk.

When to use which:
- extraDependencyGroups: enable a pyproject.toml optional extra
- extraPythonPackages: add an external Python plugin not in pyproject.toml

Usage:
  services.hermes-agent.extraDependencyGroups = [ "hindsight" ];

Or via overlay:
  pkgs.hermes-agent.override { extraDependencyGroups = [ "hindsight" ]; }

Refs: NousResearch#8873, NousResearch#9194
JinyuID pushed a commit to JinyuID/hermes-agent that referenced this pull request May 11, 2026
…arch#21817)

Expose the dependency-groups parameter from python.nix through
hermes-agent.nix and the NixOS module, allowing users to opt into
pyproject.toml optional extras (e.g. hindsight, voice, matrix) that
are resolved by uv inside the sealed venv.

Unlike extraPythonPackages (which appends to PYTHONPATH and requires
collision checking), extraDependencyGroups resolves the full dependency
graph in a single uv pass — no PYTHONPATH patching, no version
conflicts, no collision risk.

When to use which:
- extraDependencyGroups: enable a pyproject.toml optional extra
- extraPythonPackages: add an external Python plugin not in pyproject.toml

Usage:
  services.hermes-agent.extraDependencyGroups = [ "hindsight" ];

Or via overlay:
  pkgs.hermes-agent.override { extraDependencyGroups = [ "hindsight" ]; }

Refs: NousResearch#8873, NousResearch#9194
02356abc pushed a commit to 02356abc/hermes-agent that referenced this pull request May 14, 2026
…arch#21817)

Expose the dependency-groups parameter from python.nix through
hermes-agent.nix and the NixOS module, allowing users to opt into
pyproject.toml optional extras (e.g. hindsight, voice, matrix) that
are resolved by uv inside the sealed venv.

Unlike extraPythonPackages (which appends to PYTHONPATH and requires
collision checking), extraDependencyGroups resolves the full dependency
graph in a single uv pass — no PYTHONPATH patching, no version
conflicts, no collision risk.

When to use which:
- extraDependencyGroups: enable a pyproject.toml optional extra
- extraPythonPackages: add an external Python plugin not in pyproject.toml

Usage:
  services.hermes-agent.extraDependencyGroups = [ "hindsight" ];

Or via overlay:
  pkgs.hermes-agent.override { extraDependencyGroups = [ "hindsight" ]; }

Refs: NousResearch#8873, NousResearch#9194
jsboige pushed a commit to jsboige/hermes-agent that referenced this pull request May 14, 2026
…arch#21817)

Expose the dependency-groups parameter from python.nix through
hermes-agent.nix and the NixOS module, allowing users to opt into
pyproject.toml optional extras (e.g. hindsight, voice, matrix) that
are resolved by uv inside the sealed venv.

Unlike extraPythonPackages (which appends to PYTHONPATH and requires
collision checking), extraDependencyGroups resolves the full dependency
graph in a single uv pass — no PYTHONPATH patching, no version
conflicts, no collision risk.

When to use which:
- extraDependencyGroups: enable a pyproject.toml optional extra
- extraPythonPackages: add an external Python plugin not in pyproject.toml

Usage:
  services.hermes-agent.extraDependencyGroups = [ "hindsight" ];

Or via overlay:
  pkgs.hermes-agent.override { extraDependencyGroups = [ "hindsight" ]; }

Refs: NousResearch#8873, NousResearch#9194
AlexFoxD pushed a commit to AlexFoxD/hermes-agent that referenced this pull request May 21, 2026
…arch#21817)

Expose the dependency-groups parameter from python.nix through
hermes-agent.nix and the NixOS module, allowing users to opt into
pyproject.toml optional extras (e.g. hindsight, voice, matrix) that
are resolved by uv inside the sealed venv.

Unlike extraPythonPackages (which appends to PYTHONPATH and requires
collision checking), extraDependencyGroups resolves the full dependency
graph in a single uv pass — no PYTHONPATH patching, no version
conflicts, no collision risk.

When to use which:
- extraDependencyGroups: enable a pyproject.toml optional extra
- extraPythonPackages: add an external Python plugin not in pyproject.toml

Usage:
  services.hermes-agent.extraDependencyGroups = [ "hindsight" ];

Or via overlay:
  pkgs.hermes-agent.override { extraDependencyGroups = [ "hindsight" ]; }

Refs: NousResearch#8873, NousResearch#9194
gweeteve pushed a commit to gweeteve/hermes-agent that referenced this pull request Jun 2, 2026
…arch#21817)

Expose the dependency-groups parameter from python.nix through
hermes-agent.nix and the NixOS module, allowing users to opt into
pyproject.toml optional extras (e.g. hindsight, voice, matrix) that
are resolved by uv inside the sealed venv.

Unlike extraPythonPackages (which appends to PYTHONPATH and requires
collision checking), extraDependencyGroups resolves the full dependency
graph in a single uv pass — no PYTHONPATH patching, no version
conflicts, no collision risk.

When to use which:
- extraDependencyGroups: enable a pyproject.toml optional extra
- extraPythonPackages: add an external Python plugin not in pyproject.toml

Usage:
  services.hermes-agent.extraDependencyGroups = [ "hindsight" ];

Or via overlay:
  pkgs.hermes-agent.override { extraDependencyGroups = [ "hindsight" ]; }

Refs: NousResearch#8873, NousResearch#9194
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/nix Nix flake, NixOS module, container packaging comp/plugins Plugin system and bundled plugins P2 Medium — degraded but workaround exists tool/memory Memory tool and memory providers type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Setup]: Hindsight-client required to use hindsight as memory provider, but can't be installed with non-container nixos config

8 participants