Skip to content

CRITICAL: Image processing triggers infinite cache recursion causing massive token consumption #27602

@yanji84

Description

@yanji84

Priority

🔴 URGENT - HIGHEST

Summary

Multiple bots affected by cache system recursion bug triggered by image processing. Verified cases: prime-heron ($15.09 in 90 minutes) and sharp-otter ($8.58). System generates 26-33 million cache read tokens per image.

Verified Impact

Prime-heron (Jinbot):

Image processing period: 2026-02-26 05:40-07:10 (90 minutes)
Historical consumption: $15.30 (normal usage over days)  
Image bug consumption: $15.09 (90 minutes)
Cache read tokens: 33,150,331 ⚠️ CRITICAL
Cache write tokens: 5,084,601 ⚠️ CRITICAL
Request count: 847

Sharp-otter (Ace Bot):

Similar period consumption: $8.58
Cache read tokens: 26,461,934 ⚠️ CRITICAL
Cache write tokens: 2,182,666 ⚠️ CRITICAL  
Request count: 727

Technical Details

Environment:

  • OpenClaw Version: 2026.2.6-3
  • Channel: WhatsApp group chat
  • Affected: Multiple bots on same platform

Timeline (prime-heron):

  1. 05:40:09 - Received image message (129 chars)
  2. 05:40:11 - Image resized: 134.5KB → 96.4KB (-28.3%)
  3. 05:40:16 - Auto-replied to group
  4. 05:40 - 07:10 - Cache recursion loop begins
  5. 07:10 - Budget exhausted, service interruption

Root Cause Analysis:

  • Image processing function triggers infinite cache recursion
  • Each cache read operation generates more cache reads
  • Exponential cache operation growth: 26M-33M reads per image
  • Bug affects multiple bots consistently

Business Impact

  • Financial: $23+ lost across multiple bots
  • Service: User experience degraded, bots offline
  • Platform: Potential infrastructure strain from cache abuse
  • Trust: Users experiencing unexpected charges

Immediate Mitigations Applied

  • 🚨 User advisory: All users notified to avoid sending images
  • 📊 Monitoring: Enhanced tracking of cache metrics
  • 💰 Budget: Temporarily increased limits for affected bots

Requested Resolution

  1. Emergency fix: Identify and patch cache recursion in image pipeline
  2. Circuit breakers: Implement cache operation limits (e.g., max 1K reads per request)
  3. Monitoring: Real-time alerts for abnormal cache patterns
  4. Testing: Regression tests for image processing workflows
  5. Documentation: Post-incident analysis and prevention guidelines

Reproduction Steps

  1. Send any image to WhatsApp bot
  2. Monitor cache read/write metrics
  3. Observe exponential growth pattern
  4. Watch token consumption spike

Additional Data Available

  • Full usage metrics for both affected bots
  • Docker logs showing exact timing
  • Cache operation patterns

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions