Skip to content

bug: reasoning->effort parameter does not work  #1479

@dariuszkowalski-com

Description

@dariuszkowalski-com

Description
The reasoning: -> effort: parameter (low | medium | high) does not appear to work, especially in custom agents. Regardless of value, results remain the same in terms of time, speed, and number of thinking tokens.

Tested Models

  • gtp-oss-20
  • gtp-oss-120
  • grok-code-fast-1

Expected Behavior

  • Clear differences between low, medium, and high reasoning.
  • gtp-oss family normally shows large variation in thinking tokens:
    • low → ~58 tokens
    • medium → ~1500 tokens
    • high → 5000+ tokens

Actual Behavior

  • No significant change between reasoning levels.
  • Output corresponds roughly to medium level regardless of setting.
  • OpenRouter activity logs show no increase in thinking tokens when switching from low to high (should be ~10x difference).

Steps to Reproduce

  1. Run the same prompt with agents with different reasoning effort: low | medium | high.
  2. Compare runtime, speed, and thinking tokens.
  3. Observe logs for token usage.

Evidence

  • Local runs of gtp-oss models confirm expected differences.
  • Cloud service runs show no variation (all behave like medium).

Impact

  • Impossible to control reasoning depth in custom agents.
  • Misleads users expecting higher reasoning at higher effort levels.

Metadata

Metadata

Assignees

No one assigned

    Labels

    type: bugSomething isn't working.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions