Skip to content

Media: recurse TTL cleanup in nested subdirs#29211

Closed
jink-ucla wants to merge 6 commits intoopenclaw:mainfrom
jink-ucla:fix/media-cleanup-recursive-subdirs
Closed

Media: recurse TTL cleanup in nested subdirs#29211
jink-ucla wants to merge 6 commits intoopenclaw:mainfrom
jink-ucla:fix/media-cleanup-recursive-subdirs

Conversation

@jink-ucla
Copy link
Copy Markdown
Contributor

Summary

  • Problem: cleanOldMedia() only deleted expired files in the media root and first-level subdirectories.
  • Why it matters: stale files in nested subdirectories could accumulate indefinitely and waste disk.
  • What changed: media cleanup now recursively scans nested subdirectories and removes expired files at any depth.
  • What did NOT change: cleanup still only removes expired files (not directories), and media save semantics are unchanged.

Change Type

  • Bug fix
  • Feature
  • Refactor
  • Docs
  • Security hardening
  • Chore/infra

Scope

  • Gateway / orchestration
  • Skills / tool execution
  • Auth / tokens
  • Memory / storage
  • Integrations
  • API / contracts
  • UI / DX
  • CI/CD / infra

User-visible / Behavior Changes

  • Expired media files are now cleaned even when stored in nested media subdirectories.

Files Changed

  • src/media/store.ts
  • src/media/store.test.ts

Verification

  • Added regression test for nested subdirectory cleanup in src/media/store.test.ts.
  • pnpm check format stage passes; type-check currently fails on unrelated baseline error in src/agents/pi-embedded-runner-extraparams.test.ts
    (TS2352).
  • Full test/build execution in this environment is partially blocked by subprocess permission limits (spawn EPERM).

Risks and Mitigations

  • Risk: recursive traversal could affect behavior in unusual directory structures.
  • Mitigation: logic still gates deletion strictly on mtime > ttl and only removes files; directories are preserved.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f142a207b3

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread src/media/store.ts Outdated
@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented Feb 27, 2026

Greptile Summary

Implemented recursive TTL cleanup for media files in nested subdirectories, fixing a bug where files beyond the first level weren't removed. The solution properly handles arbitrary nesting depth while preserving directory structure.

  • Recursively removes expired files at all directory depths (previously only first-level subdirectories)
  • Added test coverage for nested subdirectory cleanup
  • Clean refactoring with improved variable naming (fullfullPath, entriesmediaEntries)

Confidence Score: 4/5

  • This PR is safe to merge with minimal risk
  • Well-implemented fix with appropriate test coverage. The recursive logic correctly handles nested directories and preserves the existing cleanup behavior. A minor suggestion about symlink handling is noted in comments but represents an edge case in the application-managed media directory.
  • No files require special attention

Last reviewed commit: 03e37ee

Copy link
Copy Markdown
Contributor

@greptile-apps greptile-apps Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2 files reviewed, 1 comment

Edit Code Review Agent Settings | Greptile

Comment thread src/media/store.ts Outdated
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6315604710

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread src/media/store.ts
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 75c287c9f3

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread src/media/store.ts
@jink-ucla
Copy link
Copy Markdown
Contributor Author

@codex reivew

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9cbf73c27f

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread src/media/store.ts
Comment on lines 123 to +124
if (stat.isDirectory()) {
await removeExpiredFilesInDir(full);
await removeExpiredFilesRecursively(fullPath);
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Skip symlinked top-level dirs before recursive cleanup

Using fs.stat on top-level media entries means symlinked directories are treated as real directories, and this new recursive call now walks the entire symlink target tree. In a setup like .openclaw/media/inbound -> /some/other/path, cleanOldMedia() can delete expired files outside the media store at arbitrary depth, which is a broader data-loss risk introduced by this recursion change; top-level entries should be lstated and symlink directories skipped.

Useful? React with 👍 / 👎.

@jink-ucla
Copy link
Copy Markdown
Contributor Author

Ready to merge. @vincentkoc could you merge this when you have a moment?

@jink-ucla
Copy link
Copy Markdown
Contributor Author

Ready to merge. @Takhoffman @Lukavyi could any of you merge this when you have a moment?

@Lukavyi
Copy link
Copy Markdown
Contributor

Lukavyi commented Feb 28, 2026

I'm not a maintainer....yet 😅

@jink-ucla
Copy link
Copy Markdown
Contributor Author

@vincentkoc @Takhoffman Bumping this as it's fully verified, all checks are passing, and it's marked as low-risk/XS. Would appreciate the merge when you have a second so I can clear these from my queue.

@vincentkoc
Copy link
Copy Markdown
Member

Folding this into #38292 so the media retention fix lands as one reviewable path.

That branch keeps the recursive cleanup coverage from here and combines it with the media.ttlHours config and gateway-owned startup/hourly cleanup needed to fully address #33078.

Keeping this linked for contributor credit because the nested-subdirectory cleanup work came from this PR.

@vincentkoc
Copy link
Copy Markdown
Member

Resolved in #38292

@vincentkoc vincentkoc closed this Mar 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants