Skip to content

feat: Add source cache TTL for JIT-vendored components#2138

Merged
aknysh merged 6 commits intomainfrom
osterman/source-cache-ttl
Mar 5, 2026
Merged

feat: Add source cache TTL for JIT-vendored components#2138
aknysh merged 6 commits intomainfrom
osterman/source-cache-ttl

Conversation

@osterman
Copy link
Member

@osterman osterman commented Mar 4, 2026

Summary

Implement a ttl (time-to-live) field on component source configuration to control how long cached JIT-vendored sources are reused before re-pulling from the remote. This solves the problem of stale caches when using floating refs (branches) without requiring manual --force flags.

Key Feature: Declarative cache expiration policy. Set ttl: 0s for active development (always fresh), ttl: 1h for team collaboration (hourly refresh), or omit TTL for infinite cache (backward compatible).

Problem Solved

When JIT-vendored components use floating refs like version: "main", Atmos skips re-pulling because the version string in metadata hasn't changed—it's still "main" even though upstream content has. Developers must manually delete .workdir/ or run source pull --force.

Solution

Add optional ttl field to source configuration. When set, the source provisioner compares the workdir's update timestamp against the TTL. If expired, the source is re-pulled automatically.

# Per-component override (stack manifest)
components:
  terraform:
    my-module:
      source:
        uri: git::https://github.com/org/repo.git
        version: main
        ttl: "0s"  # Always re-pull

# Global default (atmos.yaml)
components:
  terraform:
    source:
      ttl: "1h"  # Re-pull if older than 1 hour

Changes

  • ✅ Add TTL field to VendorComponentSource schema
  • ✅ Add TerraformSourceSettings struct with global TTL default
  • ✅ Parse ttl from source maps in extract.go
  • ✅ Implement TTL expiration check in needsProvisioning()
  • ✅ Support per-component override and global defaults
  • ✅ Handle zero TTL explicitly (always expires)
  • ✅ Comprehensive unit tests for all TTL behaviors

Documentation

  • ✅ Updated terraform, helmfile, and packer source command docs with ttl field
  • ✅ Added "Cache TTL for Floating Refs" section to source-based versioning design pattern
  • ✅ Created PRD explaining problem, solution, and architecture (docs/prd/source-cache-ttl.md)
  • ✅ Created blog post with user-facing guidance (website/blog/2026-03-03-source-cache-ttl.mdx)
  • ✅ Updated roadmap with shipped milestone

Test Plan

  • Unit tests for TTL behavior: zero TTL, relative TTL with recent/old timestamps, no TTL
  • Integration tests for component sourcing with TTL
  • Tests for global TTL default merging
  • Code compiles: go build ./...
  • Tests pass: go test ./pkg/provisioner/source/...
  • Linting passes: make lint
  • Website builds: cd website && npm run build

Related

Fixes #2135

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features

    • Added source cache TTL for JIT‑vendored components. Configure per‑component or via a global default; "0s" forces re‑pull, other durations (e.g., "1h", "7d") trigger automatic refresh, unset = indefinite cache. Global default merges with per‑component TTL (per‑component wins).
  • Documentation

    • New docs, examples, a blog post, and CLI doc updates demonstrating TTL usage for Terraform, Helmfile, and Packer and recommended workflows.
  • Tests

    • Added unit tests for TTL parsing, invalid TTL handling, default merging, and TTL-driven provisioning decisions.

Implement a `ttl` (time-to-live) field on component source configuration to control how long cached JIT-vendored sources are reused before re-pulling from the remote.

## Changes

- Add `TTL` field to `VendorComponentSource` schema
- Add `TerraformSourceSettings` struct with global TTL default to `Terraform` config
- Parse `ttl` from source maps in extract.go
- Implement TTL expiration check in `needsProvisioning()` function
- Support per-component override and global default from atmos.yaml
- Handle zero TTL explicitly (always expires for active development)
- Add comprehensive unit tests for TTL behavior

## Configuration

```yaml
# Per-component (stack manifest)
components:
  terraform:
    my-module:
      source:
        uri: git::https://github.com/org/repo.git
        version: main
        ttl: "0s"  # Always re-pull

# Global default (atmos.yaml)
components:
  terraform:
    source:
      ttl: "1h"  # Re-pull if older than 1 hour
```

## Documentation

- Updated terraform, helmfile, and packer source command docs
- Added design pattern section on cache TTL for floating refs
- Created PRD explaining problem, solution, and architecture
- Created blog post with user-facing guidance
- Updated roadmap with shipped milestone

Fixes #2135

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
@osterman osterman requested a review from a team as a code owner March 4, 2026 03:13
@osterman osterman added feature New functionality documentation Improvements or additions to documentation labels Mar 4, 2026
@github-actions github-actions bot added the size/m Medium size PR label Mar 4, 2026
@github-actions
Copy link

github-actions bot commented Mar 4, 2026

Dependency Review

✅ No vulnerabilities or license issues found.

Scanned Files

None

@osterman osterman added the minor New features that do not break anything label Mar 4, 2026
MDX requires multi-line dd content to use block-level tags (dd on
its own line) rather than inline tags to handle blank lines correctly.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@coderabbitai
Copy link
Contributor

coderabbitai bot commented Mar 4, 2026

📝 Walkthrough

Walkthrough

Adds a source cache TTL mechanism for JIT-vendored components: schema fields for per-component and global TTL, parsing of ttl from source maps, TTL-aware provisioning logic that can trigger re-pulls when cache expires (including zero-ttl semantics), tests, and documentation/roadmap updates.

Changes

Cohort / File(s) Summary
Schema & Vendor Source
pkg/schema/vendor_component.go, pkg/schema/schema.go
Add TTL to VendorComponentSource and introduce SourceSettings/Source (with TTL) on Terraform, Helmfile, and Packer for global defaults.
Extraction & Tests
pkg/provisioner/source/extract.go, pkg/provisioner/source/extract_test.go
Parse ttl from source maps, validate duration format (error on invalid), preserve "0s" and absent TTL; add tests for valid, missing, zero, and invalid TTLs.
Provisioning Logic & Tests
pkg/provisioner/source/provision_hook.go, pkg/provisioner/source/provision_hook_test.go
Merge global/default TTL with per-component TTL, add isSourceCacheExpired() and isZeroTTL() helpers, evaluate TTL expiry inside needsProvisioning() and related flows; extensive tests for expiry, zero, invalid TTLs, and default merging.
Documentation & Blog
docs/prd/source-cache-ttl.md, website/blog/2026-03-03-source-cache-ttl.mdx, website/docs/.../terraform/source/source.mdx, website/docs/.../helmfile/source/source.mdx, website/docs/.../packer/source/source.mdx, website/docs/design-patterns/.../source-based-versioning.mdx
Add PRD, blog post, and CLI/docs pages describing TTL semantics, examples, supported formats, and global default behavior.
Roadmap
website/src/data/roadmap.js
Add shipped milestone entry "Source cache TTL for JIT-vendored components" in roadmap data.

Sequence Diagram(s)

sequenceDiagram
    participant Config as Config/Schema\n(atmos.yaml)
    participant Extract as Extract Layer
    participant Cache as Metadata Cache
    participant Provision as Provisioning Logic
    participant Source as Remote Source

    Config->>Extract: provide source config (uri, version, ttl)
    Extract->>Cache: persist source spec + TTL
    Provision->>Cache: read metadata (UpdatedAt, TTL)
    Provision->>Provision: evaluate isSourceCacheExpired(TTL, UpdatedAt)
    alt TTL expired or TTL == "0s"
        Provision->>Source: re-pull remote source
        Source-->>Provision: fresh content
        Provision->>Cache: update metadata.UpdatedAt
    else cache still valid
        Provision->>Provision: reuse cached workdir
    end
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

Suggested reviewers

  • osterman
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 76.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and accurately summarizes the main change: adding a source cache TTL feature for JIT-vendored components.
Linked Issues check ✅ Passed The implementation fully addresses issue #2135 by providing per-component and global TTL configuration to control re-pulling of JIT-vendored sources, replacing the proposed force_pull with a more flexible TTL mechanism.
Out of Scope Changes check ✅ Passed All changes are directly aligned with the TTL feature scope: schema updates, parsing logic, provisioning checks, comprehensive tests, and documentation. No unrelated modifications detected.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
  • 📝 Generate docstrings (stacked PR)
  • 📝 Generate docstrings (commit on current branch)
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch osterman/source-cache-ttl

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 5

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@pkg/provisioner/source/extract.go`:
- Around line 92-95: The ttl field must be validated during extraction: in the
block that reads sourceMap["ttl"] and assigns spec.TTL, attempt to
parse/validate the ttl string (e.g., with time.ParseDuration or the project’s
TTL parser) and on parse/validation failure return ErrSourceInvalidSpec with
context (include the invalid value and a brief hint about expected format)
instead of silently assigning; update the extraction path that sets spec.TTL
(where sourceMap, spec.TTL are referenced) to perform this validation and return
the error when invalid.

In `@pkg/provisioner/source/provision_hook.go`:
- Around line 341-345: The TTL parsing error is currently swallowed after
calling duration.ParseDuration(ttl) (variable ttlDuration), which hides
misconfiguration; instead surface the failure by logging or returning an
explicit error message: when duration.ParseDuration(ttl) returns err, call the
existing logger (or return a non-empty error string) with context like "invalid
TTL format" and the err details and avoid proceeding as if no TTL was provided;
update the clause that now returns (false, "") to return (false, "<clear error
message including err>") or propagate the error so callers of this function can
fail-fast and notify users (also update any tests that assumed silent skip).
- Around line 81-84: The TTL defaulting currently always uses
atmosConfig.Components.Terraform.Source.TTL; update AutoProvisionSource to
select the global TTL based on the componentType parameter instead of hardcoding
Terraform. Locate the TTL assignment that checks sourceSpec.TTL and replace the
hardcoded reference with a component-type-aware lookup on atmosConfig.Components
(e.g., switch or map keyed by componentType to retrieve the appropriate
Components.<Type>.Source.TTL), ensuring you still only set sourceSpec.TTL when
it is empty; reference symbols: AutoProvisionSource, sourceSpec.TTL,
componentType, and atmosConfig.Components.Source.TTL.

In `@pkg/schema/schema.go`:
- Around line 436-442: The PR adds TerraformSourceSettings (type
TerraformSourceSettings with TTL) but lacks equivalent typed global source
configs and merging logic for Helmfile and Packer, so add corresponding types
(e.g., HelmfileSourceSettings and PackerSourceSettings with the same TTL field
and tags), expose them in the top-level schema struct where
TerraformSourceSettings is declared, and update the provisioning/merge code that
applies global defaults (the routine that reads TerraformSourceSettings) to use
the appropriate global default per component type (helmfile ->
HelmfileSourceSettings, packer -> PackerSourceSettings) when resolving
component.source.ttl so components.helmfile.source.ttl and
components.packer.source.ttl are supported consistently with
Terraform.Source.TTL.

In `@website/src/data/roadmap.js`:
- Line 333: The shipped roadmap entry for the milestone with label "Source cache
TTL for JIT-vendored components" is missing the required pr field; update the
object (the milestone literal that currently has keys label, status, quarter,
changelog, prd, docs, description, benefits) to include a pr field set to the PR
number (e.g., pr: '###' or the actual PR string) so the milestone includes
status, changelog and pr as required by guidelines.

ℹ️ Review info

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Disabled knowledge base sources:

  • Linear integration is disabled

You can enable these sources in your CodeRabbit configuration.

Run ID: 18f715bb-d5c0-468a-9b4a-26657913dd44

📥 Commits

Reviewing files that changed from the base of the PR and between 427ce17 and 479fd20.

📒 Files selected for processing (13)
  • docs/prd/source-cache-ttl.md
  • pkg/provisioner/source/extract.go
  • pkg/provisioner/source/extract_test.go
  • pkg/provisioner/source/provision_hook.go
  • pkg/provisioner/source/provision_hook_test.go
  • pkg/schema/schema.go
  • pkg/schema/vendor_component.go
  • website/blog/2026-03-03-source-cache-ttl.mdx
  • website/docs/cli/commands/helmfile/source/source.mdx
  • website/docs/cli/commands/packer/source/source.mdx
  • website/docs/cli/commands/terraform/source/source.mdx
  • website/docs/design-patterns/version-management/source-based-versioning.mdx
  • website/src/data/roadmap.js

- Validate TTL at extraction time; invalid values now return
  ErrSourceInvalidSpec with context and hint
- Rename TerraformSourceSettings to SourceSettings and add Source
  field to Helmfile and Packer schema structs
- Use component-type-aware global TTL defaults instead of hardcoding
  Terraform
- Force re-provision on invalid TTL format instead of silently
  skipping
- Add pr: 2138 to roadmap milestone entry

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
pkg/provisioner/source/provision_hook.go (1)

348-351: Include parse-error context in the invalid TTL message.

Line 350 surfaces the invalid TTL, but adding the parse error (and a short format hint) would make troubleshooting faster.

Proposed tweak
-		return true, fmt.Sprintf("Invalid source TTL %q; forcing re-provision to avoid stale cache", ttl)
+		return true, fmt.Sprintf(
+			"Invalid source TTL %q (%v); forcing re-provision to avoid stale cache. Use formats like \"1h\", \"30m\", or \"0s\"",
+			ttl,
+			err,
+		)
As per coding guidelines, "Provide clear error messages to users, include troubleshooting hints when appropriate, and log detailed errors for debugging."
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@pkg/provisioner/source/provision_hook.go` around lines 348 - 351, The invalid
TTL branch should include the parse error and a short format hint in the
returned message: when duration.ParseDuration(ttl) fails (see ttlDuration and
duration.ParseDuration(ttl)), return a message that embeds err.Error() and a
brief example/hint such as "expected formats like '30s' or '5m'" so callers can
quickly diagnose and fix the TTL value.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@pkg/provisioner/source/provision_hook.go`:
- Around line 348-351: The invalid TTL branch should include the parse error and
a short format hint in the returned message: when duration.ParseDuration(ttl)
fails (see ttlDuration and duration.ParseDuration(ttl)), return a message that
embeds err.Error() and a brief example/hint such as "expected formats like '30s'
or '5m'" so callers can quickly diagnose and fix the TTL value.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 29ec299c-3ec2-4224-846c-2d5f2b23f14f

📥 Commits

Reviewing files that changed from the base of the PR and between 479fd20 and 61a0baa.

📒 Files selected for processing (6)
  • docs/prd/source-cache-ttl.md
  • pkg/provisioner/source/extract.go
  • pkg/provisioner/source/extract_test.go
  • pkg/provisioner/source/provision_hook.go
  • pkg/schema/schema.go
  • website/src/data/roadmap.js
🚧 Files skipped from review as they are similar to previous changes (3)
  • pkg/provisioner/source/extract_test.go
  • pkg/provisioner/source/extract.go
  • website/src/data/roadmap.js

coderabbitai[bot]
coderabbitai bot previously approved these changes Mar 4, 2026
… handling

Add Go doc comments to 5 new TTL test functions to improve docstring
coverage. Change Source fields from value to pointer types on Terraform,
Helmfile, and Packer structs so unset source config is properly nil
rather than a zero-value struct, with corresponding nil checks in the
provisioning logic.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (3)
pkg/provisioner/source/provision_hook.go (1)

346-369: Normalize TTL input before zero-value checks and parsing.

Right now zero TTL detection is exact-match only. Trimming/lowercasing first avoids accidental “invalid TTL” behavior for harmless formatting differences.

🛠️ Suggested adjustment
 func isSourceCacheExpired(ttl string, updatedAt time.Time) (bool, string) {
+	normalizedTTL := strings.ToLower(strings.TrimSpace(ttl))
+
 	// Handle zero TTL explicitly (always expired).
-	if isZeroTTL(ttl) {
-		return true, fmt.Sprintf("Source cache expired (TTL: %s, always re-pull)", ttl)
+	if isZeroTTL(normalizedTTL) {
+		return true, fmt.Sprintf("Source cache expired (TTL: %s, always re-pull)", normalizedTTL)
 	}
 
-	ttlDuration, err := duration.ParseDuration(ttl)
+	ttlDuration, err := duration.ParseDuration(normalizedTTL)
 	if err != nil {
-		return true, fmt.Sprintf("Invalid source TTL %q; forcing re-provision to avoid stale cache", ttl)
+		return true, fmt.Sprintf("Invalid source TTL %q; forcing re-provision to avoid stale cache", ttl)
 	}
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@pkg/provisioner/source/provision_hook.go` around lines 346 - 369, The TTL
string should be normalized (trimmed and lowercased) before being checked or
parsed to avoid accidental "invalid TTL" or missed zero checks; update
isSourceCacheExpired to call a normalizedTTL :=
strings.ToLower(strings.TrimSpace(ttl)) and use normalizedTTL for the isZeroTTL
check and for duration.ParseDuration, and either adjust isZeroTTL to expect a
normalized value or perform the same normalization inside isZeroTTL so values
like " 0S ", "0M", or " 0 " are correctly treated as zero TTL; keep all existing
return messages but continue to display the original ttl where desirable or
switch to normalizedTTL consistently.
pkg/provisioner/source/extract_test.go (1)

673-738: Consolidate TTL extraction cases into one table-driven test.

These four tests follow the same setup/assertion pattern; one table-driven test will reduce duplication and make future TTL cases easier to add.

♻️ Refactor sketch
-func TestExtractSource_WithTTL(t *testing.T) { ... }
-func TestExtractSource_WithoutTTL(t *testing.T) { ... }
-func TestExtractSource_WithInvalidTTL(t *testing.T) { ... }
-func TestExtractSource_WithZeroTTL(t *testing.T) { ... }
+func TestExtractSource_TTL(t *testing.T) {
+	tests := []struct {
+		name        string
+		source      map[string]any
+		expectErr   error
+		expectedTTL string
+	}{
+		// valid / missing / invalid / zero TTL cases
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			componentConfig := map[string]any{"source": tt.source}
+			result, err := ExtractSource(componentConfig)
+			// assertions...
+		})
+	}
+}

As per coding guidelines, "Use table-driven tests for testing multiple scenarios in Go."

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@pkg/provisioner/source/extract_test.go` around lines 673 - 738, The four TTL
tests for ExtractSource are duplicated and should be consolidated into a single
table-driven test: create a slice of test cases (with fields like name, ttlValue
(nil/""/string), expectedTTL, expectError bool, expectedErr) and iterate with
t.Run; for each case build the componentConfig with
"source":{"uri":"github.com/example/repo//module","version":..., "ttl":
ttlValue} (omit ttl when nil), call ExtractSource, and assert Uri and Version
always, then check TTL equals expectedTTL when expectError is false and assert
errors (require.Error / assert.ErrorIs with errUtils.ErrSourceInvalidSpec) and
nil result when expectError is true; replace the four individual
TestExtractSource_* functions with this single table-driven test to cover valid,
missing, invalid, and zero TTL scenarios.
pkg/provisioner/source/provision_hook_test.go (1)

335-402: Add an invalid-TTL case to lock in the defensive safety-net behavior.

This suite is strong; one more case (ttl: "invalid") asserting reprovision + reason containing Invalid source TTL would prevent regressions in the fallback path.

As per coding guidelines, "Every new feature must include comprehensive unit tests targeting >80% code coverage for all packages."

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@pkg/provisioner/source/provision_hook_test.go` around lines 335 - 402, Add a
test case to TestNeedsProvisioning_TTL that uses ttl: "invalid" to verify the
defensive fallback in needsProvisioning; update the tests slice in
TestNeedsProvisioning_TTL to include an entry with name "invalid TTL forces
reprovision", ttl: "invalid", any updatedAt (e.g., time.Now()), expected: true,
and expectExpired: true, then after calling needsProvisioning(dirPath,
sourceSpec, true) assert the result is true and assert.Contains(t, reason,
"Invalid source TTL") to lock in the error-path behavior for
schema.VendorComponentSource.TTL handling.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@pkg/provisioner/source/extract_test.go`:
- Around line 673-738: The four TTL tests for ExtractSource are duplicated and
should be consolidated into a single table-driven test: create a slice of test
cases (with fields like name, ttlValue (nil/""/string), expectedTTL, expectError
bool, expectedErr) and iterate with t.Run; for each case build the
componentConfig with
"source":{"uri":"github.com/example/repo//module","version":..., "ttl":
ttlValue} (omit ttl when nil), call ExtractSource, and assert Uri and Version
always, then check TTL equals expectedTTL when expectError is false and assert
errors (require.Error / assert.ErrorIs with errUtils.ErrSourceInvalidSpec) and
nil result when expectError is true; replace the four individual
TestExtractSource_* functions with this single table-driven test to cover valid,
missing, invalid, and zero TTL scenarios.

In `@pkg/provisioner/source/provision_hook_test.go`:
- Around line 335-402: Add a test case to TestNeedsProvisioning_TTL that uses
ttl: "invalid" to verify the defensive fallback in needsProvisioning; update the
tests slice in TestNeedsProvisioning_TTL to include an entry with name "invalid
TTL forces reprovision", ttl: "invalid", any updatedAt (e.g., time.Now()),
expected: true, and expectExpired: true, then after calling
needsProvisioning(dirPath, sourceSpec, true) assert the result is true and
assert.Contains(t, reason, "Invalid source TTL") to lock in the error-path
behavior for schema.VendorComponentSource.TTL handling.

In `@pkg/provisioner/source/provision_hook.go`:
- Around line 346-369: The TTL string should be normalized (trimmed and
lowercased) before being checked or parsed to avoid accidental "invalid TTL" or
missed zero checks; update isSourceCacheExpired to call a normalizedTTL :=
strings.ToLower(strings.TrimSpace(ttl)) and use normalizedTTL for the isZeroTTL
check and for duration.ParseDuration, and either adjust isZeroTTL to expect a
normalized value or perform the same normalization inside isZeroTTL so values
like " 0S ", "0M", or " 0 " are correctly treated as zero TTL; keep all existing
return messages but continue to display the original ttl where desirable or
switch to normalizedTTL consistently.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: fe76f4c8-c4e0-43c4-9ea7-fe42cecd52eb

📥 Commits

Reviewing files that changed from the base of the PR and between 61a0baa and 56f789b.

📒 Files selected for processing (4)
  • pkg/provisioner/source/extract_test.go
  • pkg/provisioner/source/provision_hook.go
  • pkg/provisioner/source/provision_hook_test.go
  • pkg/schema/schema.go

coderabbitai[bot]
coderabbitai bot previously approved these changes Mar 4, 2026
@codecov
Copy link

codecov bot commented Mar 4, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 76.62%. Comparing base (b80e946) to head (e74a5a9).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #2138      +/-   ##
==========================================
+ Coverage   76.59%   76.62%   +0.02%     
==========================================
  Files         833      833              
  Lines       79592    79635      +43     
==========================================
+ Hits        60963    61017      +54     
+ Misses      14825    14819       -6     
+ Partials     3804     3799       -5     
Flag Coverage Δ
unittests 76.62% <100.00%> (+0.02%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
pkg/provisioner/source/extract.go 100.00% <100.00%> (ø)
pkg/provisioner/source/provision_hook.go 86.51% <100.00%> (+2.44%) ⬆️
pkg/schema/schema.go 87.70% <ø> (ø)

... and 3 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@mergify
Copy link

mergify bot commented Mar 5, 2026

💥 This pull request now has conflicts. Could you fix it @osterman? 🙏

@mergify mergify bot added the conflict This PR has conflicts label Mar 5, 2026
aknysh and others added 2 commits March 5, 2026 14:54
Resolve roadmap.js conflict by keeping both milestones (source-cache-ttl and per-target version overrides).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Extract applyGlobalTTLDefault into a testable function and add comprehensive
tests for: global TTL defaults (terraform/helmfile/packer), per-component
override precedence, isSourceCacheExpired edge cases (invalid TTL, all zero
forms, 7d format), isZeroTTL variants, and TTL format validation in ExtractSource.

Coverage: 83.4% → 85.6% for source package. All TTL functions at 100%.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (2)
pkg/provisioner/source/provision_hook.go (1)

355-358: Include the parse error details in the invalid TTL reason.

Current message is good, but dropping err hides the exact parsing failure and slows troubleshooting.

Suggested tweak
 	ttlDuration, err := duration.ParseDuration(ttl)
 	if err != nil {
-		return true, fmt.Sprintf("Invalid source TTL %q; forcing re-provision to avoid stale cache", ttl)
+		return true, fmt.Sprintf(
+			"Invalid source TTL %q (%v); forcing re-provision to avoid stale cache",
+			ttl,
+			err,
+		)
 	}

As per coding guidelines, provide clear error messages to users, include troubleshooting hints when appropriate, and log detailed errors for debugging.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@pkg/provisioner/source/provision_hook.go` around lines 355 - 358, The invalid
TTL error message hides the parsing error; update the block that calls
duration.ParseDuration (assigning to ttlDuration, err) so the returned reason
includes the parse error details (err) alongside the TTL value — e.g., include
err.Error() or %v in the fmt.Sprintf used in the return statement where
ttlDuration and err are in scope — so the returned string shows both the invalid
TTL and the underlying parse error for debugging.
pkg/provisioner/source/provision_hook_test.go (1)

450-454: Tighten non-expired-path assertions with empty reason checks.

Right now non-expired cases only check expired/result == false; asserting reason is empty would catch noisy or stale reason regressions.

Optional hardening
 			result, reason := needsProvisioning(dirPath, sourceSpec, true)
 			assert.Equal(t, tt.expected, result)
 			if tt.expectExpired {
 				assert.Contains(t, reason, "Source cache expired")
+			} else {
+				assert.Empty(t, reason)
 			}
 			expired, reason := isSourceCacheExpired(tt.ttl, tt.updatedAt)
 			assert.Equal(t, tt.expected, expired)
 			if tt.expectedReason != "" {
 				assert.Contains(t, reason, tt.expectedReason)
+			} else {
+				assert.Empty(t, reason)
 			}

Also applies to: 1139-1142

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@pkg/provisioner/source/provision_hook_test.go` around lines 450 - 454, The
tests call needsProvisioning(dirPath, sourceSpec, true) and currently only
assert result/expired is false for non-expired cases; update the test assertions
to also assert that the returned reason string is empty when tt.expectExpired is
false to catch stray or stale reason messages—i.e., where the table-driven test
checks tt.expectExpired, add an assert.Equal(t, "", reason) (or assert.Empty)
for the non-expired branch; apply the same tightening to the other occurrence
around the 1139-1142 test block so both places validate an empty reason on
non-expired outcomes.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@pkg/provisioner/source/provision_hook_test.go`:
- Around line 450-454: The tests call needsProvisioning(dirPath, sourceSpec,
true) and currently only assert result/expired is false for non-expired cases;
update the test assertions to also assert that the returned reason string is
empty when tt.expectExpired is false to catch stray or stale reason
messages—i.e., where the table-driven test checks tt.expectExpired, add an
assert.Equal(t, "", reason) (or assert.Empty) for the non-expired branch; apply
the same tightening to the other occurrence around the 1139-1142 test block so
both places validate an empty reason on non-expired outcomes.

In `@pkg/provisioner/source/provision_hook.go`:
- Around line 355-358: The invalid TTL error message hides the parsing error;
update the block that calls duration.ParseDuration (assigning to ttlDuration,
err) so the returned reason includes the parse error details (err) alongside the
TTL value — e.g., include err.Error() or %v in the fmt.Sprintf used in the
return statement where ttlDuration and err are in scope — so the returned string
shows both the invalid TTL and the underlying parse error for debugging.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: e2d173ea-f2e8-403e-99dd-e9c82d622f8c

📥 Commits

Reviewing files that changed from the base of the PR and between 56f789b and e74a5a9.

📒 Files selected for processing (4)
  • pkg/provisioner/source/provision_hook.go
  • pkg/provisioner/source/provision_hook_test.go
  • pkg/schema/schema.go
  • website/src/data/roadmap.js

@aknysh aknysh merged commit f0ab0c7 into main Mar 5, 2026
64 checks passed
@aknysh aknysh deleted the osterman/source-cache-ttl branch March 5, 2026 22:37
@github-actions
Copy link

github-actions bot commented Mar 6, 2026

These changes were released in v1.208.1-rc.1.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation feature New functionality minor New features that do not break anything size/m Medium size PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

JIT: Option to always re-pull JIT-vendored sources via force_pull

2 participants