Skip to content

fix(dreaming): use ingestion date for dayBucket instead of file date#67091

Merged
jalehman merged 2 commits into
openclaw:mainfrom
Bartok9:fix/67061-dreaming-daybucket-ingestion-date
Apr 15, 2026
Merged

fix(dreaming): use ingestion date for dayBucket instead of file date#67091
jalehman merged 2 commits into
openclaw:mainfrom
Bartok9:fix/67061-dreaming-daybucket-ingestion-date

Conversation

@Bartok9

@Bartok9 Bartok9 commented Apr 15, 2026

Copy link
Copy Markdown
Contributor

Summary

Fixes #67061.

When dreaming re-ingests the same daily memory file across multiple days, dailyCount stays stuck at 1 because dayBucket was set to batch.day (the file date) instead of the current ingestion date. Since dedupeByQueryPerDay checks whether todayBucket already appears in recallDays, using the file date causes every subsequent sweep to hit the dedupe guard — the file date was already recorded on the first ingestion.

This keeps signalCount (recallCount + dailyCount + groundedCount) below the default minRecallCount=3 promotion gate, so zero entries ever reach the scoring stage, and the phase reinforcement from #64068 can never take effect.

Root cause

In dreaming-phases.ts, all three ingestion functions pass the file date as dayBucket:

dayBucket: batch.day,  // e.g. "2026-04-11"

The dedupe logic in recordShortTermRecalls then checks:

const dedupeSignal =
  Boolean(params.dedupeByQueryPerDay) &&
  queryHashesBase.includes(queryHash) &&
  recallDaysBase.includes(todayBucket);

Since both queryHash and todayBucket (derived from dayBucket) are identical across sweeps, dedupeSignal is always true after the first ingestion, preventing dailyCount from incrementing.

Fix

Change dayBucket from the file date to the ingestion date (via formatMemoryDreamingDay(params.nowMs, params.timezone)) in all three call sites:

  1. ingestSessionTranscriptSignals
  2. ingestDailyMemorySignals
  3. seedHistoricalDailyMemorySignals

The query string still uses the file date for identification; only the dedupe bucket changes to reflect when the sweep actually ran. formatMemoryDreamingDay was already imported and params.nowMs/params.timezone were already available at all three sites.

Test

Added a regression test that:

  1. Ingests a daily memory file on day 1 (2026-04-05)
  2. Clears the checkpoint, then re-ingests the same file on day 2 (2026-04-06)
  3. Verifies dailyCount increments to 2 (was stuck at 1 before this fix)

All 22 tests in dreaming-phases.test.ts pass.

@greptile-apps

greptile-apps Bot commented Apr 15, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

Fixes a dedupe regression in the dreaming pipeline where dayBucket was set to the file's date rather than the ingestion date. Because recordShortTermRecalls uses dayBucket as todayBucket when evaluating dedupeSignal, passing the file date caused every re-ingestion sweep after the first to be treated as a same-day duplicate, keeping dailyCount permanently at 1 and blocking the promotion gate. The one-line fix at all three call sites (ingestSessionTranscriptSignals, ingestDailyMemorySignals, seedHistoricalDailyMemorySignals) is correct, and the regression test precisely covers the described scenario.

Confidence Score: 5/5

Safe to merge — targeted fix with a direct regression test and no unintended side effects.

All three changed call sites receive the same correct treatment (ingestion date instead of file date), same-day deduplication still works correctly, and the new test directly exercises the broken path. No P0/P1 issues found.

No files require special attention.

Reviews (1): Last reviewed commit: "fix(dreaming): use ingestion date for da..." | Re-trigger Greptile

@jalehman jalehman self-assigned this Apr 15, 2026
@jalehman jalehman force-pushed the fix/67061-dreaming-daybucket-ingestion-date branch from cb4ab80 to 7257b0c Compare April 15, 2026 18:17
bartokmagic and others added 2 commits April 15, 2026 11:54
When dreaming re-ingests the same daily memory file on a later day,
dayBucket was set to batch.day (the file date, e.g. '2026-04-11')
instead of the current ingestion date. Because dedupeByQueryPerDay
checks whether todayBucket already appears in recallDays, using the
file date caused all subsequent sweeps to hit the dedupe guard — the
file date was already in recallDays from the first ingestion. This
kept dailyCount stuck at 1, signalCount below the minRecallCount=3
promotion gate, and zero entries ever reached the scoring stage.

Fix: pass formatMemoryDreamingDay(params.nowMs, params.timezone) as
dayBucket in all three ingestion call sites (session transcripts,
daily memory, and historical seed). The query string still uses the
file date for identification; only the dedupe bucket changes to
reflect when the sweep actually ran.

Closes openclaw#67061
@jalehman jalehman force-pushed the fix/67061-dreaming-daybucket-ingestion-date branch from 7257b0c to 2df44e4 Compare April 15, 2026 18:55
@jalehman jalehman merged commit 4de56b1 into openclaw:main Apr 15, 2026
25 of 26 checks passed
@jalehman

Copy link
Copy Markdown
Contributor

Merged via squash.

Thanks @Bartok9!

xudaiyanzi pushed a commit to xudaiyanzi/openclaw that referenced this pull request Apr 17, 2026
…penclaw#67091)

Merged via squash.

Prepared head SHA: 2df44e4
Co-authored-by: Bartok9 <259807879+Bartok9@users.noreply.github.com>
Co-authored-by: jalehman <550978+jalehman@users.noreply.github.com>
Reviewed-by: @jalehman
kvnkho pushed a commit to kvnkho/openclaw that referenced this pull request Apr 17, 2026
…penclaw#67091)

Merged via squash.

Prepared head SHA: 2df44e4
Co-authored-by: Bartok9 <259807879+Bartok9@users.noreply.github.com>
Co-authored-by: jalehman <550978+jalehman@users.noreply.github.com>
Reviewed-by: @jalehman
lovewanwan pushed a commit to lovewanwan/openclaw that referenced this pull request Apr 28, 2026
…penclaw#67091)

Merged via squash.

Prepared head SHA: 2df44e4
Co-authored-by: Bartok9 <259807879+Bartok9@users.noreply.github.com>
Co-authored-by: jalehman <550978+jalehman@users.noreply.github.com>
Reviewed-by: @jalehman
ogt-redknie pushed a commit to ogt-redknie/OPENX that referenced this pull request May 2, 2026
…penclaw#67091)

Merged via squash.

Prepared head SHA: 2df44e4
Co-authored-by: Bartok9 <259807879+Bartok9@users.noreply.github.com>
Co-authored-by: jalehman <550978+jalehman@users.noreply.github.com>
Reviewed-by: @jalehman
github-actions Bot pushed a commit to Desicool/openclaw that referenced this pull request May 9, 2026
…penclaw#67091)

Merged via squash.

Prepared head SHA: 2df44e4
Co-authored-by: Bartok9 <259807879+Bartok9@users.noreply.github.com>
Co-authored-by: jalehman <550978+jalehman@users.noreply.github.com>
Reviewed-by: @jalehman
globalcaos pushed a commit to globalcaos/tinkerclaw that referenced this pull request May 13, 2026
…penclaw#67091)

Merged via squash.

Prepared head SHA: 2df44e4
Co-authored-by: Bartok9 <259807879+Bartok9@users.noreply.github.com>
Co-authored-by: jalehman <550978+jalehman@users.noreply.github.com>
Reviewed-by: @jalehman
github-actions Bot pushed a commit to Desicool/openclaw that referenced this pull request May 24, 2026
…penclaw#67091)

Merged via squash.

Prepared head SHA: 2df44e4
Co-authored-by: Bartok9 <259807879+Bartok9@users.noreply.github.com>
Co-authored-by: jalehman <550978+jalehman@users.noreply.github.com>
Reviewed-by: @jalehman
jameslcowan pushed a commit to jameslcowan/openclaw that referenced this pull request Jun 2, 2026
…penclaw#67091)

Merged via squash.

Prepared head SHA: 2df44e4
Co-authored-by: Bartok9 <259807879+Bartok9@users.noreply.github.com>
Co-authored-by: jalehman <550978+jalehman@users.noreply.github.com>
Reviewed-by: @jalehman
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Dreaming: dayBucket uses file date instead of ingestion date, preventing dailyCount from growing and blocking short-term promotion

3 participants