Skip to content

Compaction bypasses custom streamFn, causing problems when using a proxy #4484

@angelaloh09

Description

@angelaloh09

What happened?

We use pi with a proxy for the LLM providers, by supplying a custom streamFn when creating the agent. This works for the normal LLM calls, but compaction fails. It appears that compaction does not call streamFn but rather calls createSimple directly, which bypasses our proxy.

The correct thing to do, I think, would be to call streamFn in the compact logic as well.

Steps to reproduce

  1. Create an Agent with a custom streamFn.
  2. Do not supply an API key for any of the model providers.
  3. Converse with agent until you hit compaction.
  4. Compaction fails because it tries to call the model provider directly and we don't have the api key for them.

Expected behavior

Compaction should use our custom streamFn so it goes through our proxy which attaches the API key.

Version

0.70.6

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingclosed-because-refactorClosed while the project refactor is in progressinprogressIssue is being worked on

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions