test: cdk watch integ tests for cli and toolkit lib by iankhou · Pull Request #1139 · aws/aws-cdk-cli

iankhou · 2026-02-12T20:38:23Z

Tests #1134

Description of tests

`cdk watch` in CLI

Test: Detects file changes for watched files
Test: Does NOT detect file changes for unwatched files

`Toolkit.watch()`

Test: Detects file changes for watched files
Test: Does NOT detect file changes for unwatched files

We used AI 🤖 to help with this contribution.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

codecov-commenter · 2026-02-12T20:50:09Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 87.76%. Comparing base (755842d) to head (a5bde3e).

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #1139   +/-   ##
=======================================
  Coverage   87.75%   87.76%           
=======================================
  Files          72       72           
  Lines       10137    10137           
  Branches     1339     1338    -1     
=======================================
+ Hits         8896     8897    +1     
+ Misses       1216     1215    -1     
  Partials       25       25

Flag	Coverage Δ
suite.unit	`87.76% <ø> (+<0.01%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

…tch integ test

mrgrain · 2026-02-19T15:45:52Z

...g/tests/cli-integ-tests/watch/cdk-watch-detects-file-changes-with-glob-patterns.integtest.ts

+      // Use separate output dir to avoid conflicts with lingering watch process
+      await fixture.cdkDestroy('test-1', { options: ['--output', 'cdk-destroy.out'] });


No need to manually delete. The rest harness is doing this for us.

Suggested change

// Use separate output dir to avoid conflicts with lingering watch process

await fixture.cdkDestroy('test-1', { options: ['--output', 'cdk-destroy.out'] });

mrgrain · 2026-02-19T15:46:23Z

...ges/@aws-cdk-testing/cli-integ/tests/toolkit-lib-integ-tests/toolkit-watch-glob.integtest.ts

+  }),
+);
+
+integTest(


Only one test per file. Pls move to separate file.

mrgrain · 2026-02-19T15:49:08Z

...g/tests/cli-integ-tests/watch/cdk-watch-detects-file-changes-with-glob-patterns.integtest.ts

+async function waitForOutput(getOutput: () => string, searchString: string, timeoutMs: number): Promise<void> {
+  const startTime = Date.now();
+  return new Promise((resolve, reject) => {
+    const check = () => {
+      if (getOutput().includes(searchString)) return resolve();
+      if (Date.now() - startTime > timeoutMs) {
+        return reject(new Error(`Timeout waiting for: "${searchString}"`));
+      }
+      setTimeout(check, 1000);
+    };
+    check();
+  });
+}
+
+async function waitForCondition(condition: () => boolean, timeoutMs: number, description: string): Promise<void> {
+  const startTime = Date.now();
+  return new Promise((resolve, reject) => {
+    const check = () => {
+      if (condition()) return resolve();
+      if (Date.now() - startTime > timeoutMs) {
+        return reject(new Error(`Timeout waiting for ${description}`));
+      }
+      setTimeout(check, 1000);
+    };
+    check();
+  });
+}


Are there really no existing helpers for this?

Nope, I couldn't find any that poll like this. I'm adding a negative test, though, so moving these to a shared file under the watch dir.

kaizencc · 2026-02-19T17:06:43Z

...g/tests/cli-integ-tests/watch/cdk-watch-detects-file-changes-with-glob-patterns.integtest.ts

+    });
+
+    try {
+      await waitForOutput(() => output, "Triggering initial 'cdk deploy'", 120000);


how did you come up with these timeout durations?

Not really based on anything in particular. The initial deployment takes longer than subsequent ones, so I was generous here.

kaizencc · 2026-02-19T17:08:23Z

...ges/@aws-cdk-testing/cli-integ/tests/toolkit-lib-integ-tests/toolkit-watch-glob.integtest.ts

+);
+
+integTest(
+  'toolkit watch excludes node_modules and dotfiles by default',


why is this test not part of the cli integ tests also?

Because it will add more time to the test, while offering limited utility IMO. Probably another minute or so. See my PR description where I addressed this discrepancy.

i dont understand why this would be the case. i think you need to
a) touch node_modules / dotfiles
b) assert that we do not see Detected change to '<file>' -- this happens nearly instantly if the file is being watched, so i dont see why it would take another minute.

as for whether this offers utility, i think its worse if we accidentally don't exclude files from watch than if we accidentally don't include files.

kaizencc · 2026-02-19T17:11:20Z

...g/tests/cli-integ-tests/watch/cdk-watch-detects-file-changes-with-glob-patterns.integtest.ts

+    const cdkJsonPath = path.join(fixture.integTestDir, 'cdk.json');
+    const cdkJson = JSON.parse(fs.readFileSync(cdkJsonPath, 'utf-8'));
+    cdkJson.watch = {
+      include: ['**/*.ts', '**/*.js'],


nit: including the .js files is a bad idea for some use cases (say you're also auto-compiling ts changes, now you're double counting). doesn't seem necessary in your test

I concur, removed **/*.js.

kaizencc · 2026-02-19T17:12:14Z

...g/tests/cli-integ-tests/watch/cdk-watch-detects-file-changes-with-glob-patterns.integtest.ts

+    const cdkJson = JSON.parse(fs.readFileSync(cdkJsonPath, 'utf-8'));
+    cdkJson.watch = {
+      include: ['**/*.ts', '**/*.js'],
+      exclude: ['node_modules/**', 'cdk.out/**', '**/*.d.ts'],


this part isn't tested. in this test or a new one, we should be testing that changes to exclude files do not trigger deployment.

I included this to model a default config, but you're right that we don't need it. Removed.

...g/tests/cli-integ-tests/watch/cdk-watch-detects-file-changes-with-glob-patterns.integtest.ts

…ch integ test

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

mrgrain · 2026-02-25T17:21:55Z

...g/tests/cli-integ-tests/watch/cdk-watch-detects-file-changes-with-glob-patterns.integtest.ts

+    await waitForOutput(() => output, "Triggering initial 'cdk deploy'", 120000);
+    fixture.log('✓ Watch started');
+
+    await waitForOutput(() => output, 'deployment time', 300000);
+    fixture.log('✓ Initial deployment completed');
+
+    // Modify the test file to trigger a watch event
+    fs.writeFileSync(testFile, 'export const modified = true;');
+
+    await waitForOutput(() => output, 'Detected change to', 60000);
+    fixture.log('✓ Watch detected file change');


Do we need individual timeout expectations for each of these? Feels like a top-level timeout would be enough to ensure it doesn't go overboard.

Removed individual timeouts, just left the file-level one.

mrgrain · 2026-02-25T17:23:26Z

...g/tests/cli-integ-tests/watch/cdk-watch-detects-file-changes-with-glob-patterns.integtest.ts

+    fixture.log('✓ Initial deployment completed');
+
+    // Modify the test file to trigger a watch event
+    fs.writeFileSync(testFile, 'export const modified = true;');


You could also just touch the file: child_process.spawn('touch', [testFile])

Changed to this approach.

...g/tests/cli-integ-tests/watch/cdk-watch-detects-file-changes-with-glob-patterns.integtest.ts

mrgrain · 2026-02-25T17:24:26Z

...g/tests/cli-integ-tests/watch/cdk-watch-detects-file-changes-with-glob-patterns.integtest.ts

+    });
+
+    await waitForOutput(() => output, "Triggering initial 'cdk deploy'", 120000);
+    fixture.log('✓ Watch started');


?

Suggested change

fixture.log('✓ Watch started');

fixture.log('✓ Watch start detected');

mrgrain · 2026-02-25T17:25:36Z

...g/tests/cli-integ-tests/watch/cdk-watch-detects-file-changes-with-glob-patterns.integtest.ts

+
+    await waitForOutput(() => output, 'Detected change to', 60000);
+    fixture.log('✓ Watch detected file change');
+  }),


Are we not checking that the second deployment runs?

Will add a check.

mrgrain · 2026-02-25T17:25:54Z

...g/tests/cli-integ-tests/watch/cdk-watch-detects-file-changes-with-glob-patterns.integtest.ts

+    await waitForOutput(() => output, 'Detected change to', 60000);
+    fixture.log('✓ Watch detected file change');
+  }),
+);


how is watch terminated here?

It's not, I'll end the process.

mrgrain · 2026-02-25T17:27:29Z

...li-integ-tests/watch/cdk-watch-detects-file-changes-with-glob-patterns-negative.integtest.ts

+    if (detectedExcluded) {
+      throw new Error('Watch should NOT have detected changes to excluded file');
+    }


Why not testing primitives like expect etc?

Will use expect instead, oops

mrgrain · 2026-02-25T17:27:41Z

...li-integ-tests/watch/cdk-watch-detects-file-changes-with-glob-patterns-negative.integtest.ts

+    // Verify deployment count hasn't increased
+    const deploymentsAfter = (output.match(/deployment time/g) || []).length;
+    if (deploymentsAfter > deploymentsBefore) {
+      throw new Error(`Unexpected deployment triggered. Before: ${deploymentsBefore}, After: ${deploymentsAfter}`);


mrgrain · 2026-02-25T17:28:20Z

...li-integ-tests/watch/cdk-watch-detects-file-changes-with-glob-patterns-negative.integtest.ts

+    // Start cdk watch with detached process group for clean termination
+    const watchProcess = child_process.spawn('cdk', [
+      'watch', '--hotswap', '-v', fixture.fullStackName('test-1'),
+    ], {
+      cwd: fixture.integTestDir,
+      shell: true,
+      detached: true,
+      env: { ...process.env, ...fixture.cdkShellEnv() },
+    });
+
+    watchProcess.stdout?.on('data', (data) => {
+      output += data.toString();
+      fixture.log(data.toString());
+    });
+    watchProcess.stderr?.on('data', (data) => {
+      output += data.toString();
+      fixture.log(data.toString());
+    });


nit: this could also be a helper on the fixture, similar to cdkDeploy

mrgrain · 2026-02-25T17:31:39Z

...li-integ-tests/watch/cdk-watch-detects-file-changes-with-glob-patterns-negative.integtest.ts

+    fixture.log('Created excluded file: should-be-ignored.excluded.ts');
+
+    // Wait a reasonable time for any potential (unwanted) detection
+    await sleep(5000);


mrgrain · 2026-02-25T17:31:55Z

...li-integ-tests/watch/cdk-watch-detects-file-changes-with-glob-patterns-negative.integtest.ts

+    const detectedExcluded = output.includes('Detected change to') &&
+    output.includes('excluded');


Suggested change

const detectedExcluded = output.includes('Detected change to') &&

output.includes('excluded');

const detectedExcluded = output.includes('Detected change to') && output.includes('excluded');

mrgrain · 2026-02-25T17:37:18Z

...cdk-testing/cli-integ/tests/toolkit-lib-integ-tests/toolkit-watch-glob-negative.integtest.ts

+      const configMsg = configMessages.find(m => m.includes("'exclude' patterns"));
+
+      if (!configMsg) {
+        throw new Error('Did not receive exclude patterns configuration message');


mrgrain · 2026-02-25T17:37:21Z

...cdk-testing/cli-integ/tests/toolkit-lib-integ-tests/toolkit-watch-glob-negative.integtest.ts

+
+      // Check that default excludes are present
+      if (!configMsg.includes('node_modules')) {
+        throw new Error('Default excludes should include node_modules');


mrgrain · 2026-02-25T17:37:24Z

...cdk-testing/cli-integ/tests/toolkit-lib-integ-tests/toolkit-watch-glob-negative.integtest.ts

+        throw new Error('Default excludes should include node_modules');
+      }
+      if (!configMsg.includes('.*')) {
+        throw new Error('Default excludes should include dotfiles (.*)');


mrgrain · 2026-02-25T17:38:21Z

...ges/@aws-cdk-testing/cli-integ/tests/toolkit-lib-integ-tests/toolkit-watch-glob.integtest.ts

+      );
+
+      if (!hasReadyOrObserving) {
+        throw new Error('Watcher did not emit ready/observing events');


Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

integ tests for cli and toolkit lib for cdk watch

3065b0f

iankhou had a problem deploying to automation February 12, 2026 20:38 — with GitHub Actions Failure

iankhou temporarily deployed to automation February 12, 2026 20:38 — with GitHub Actions Inactive

iankhou temporarily deployed to integ-approval February 12, 2026 20:38 — with GitHub Actions Inactive

github-actions bot added bug p0 labels Feb 12, 2026

aws-cdk-automation requested a review from a team February 12, 2026 20:39

iankhou temporarily deployed to run-tests February 12, 2026 20:48 — with GitHub Actions Inactive

iankhou temporarily deployed to run-tests February 12, 2026 20:52 — with GitHub Actions Inactive

iankhou had a problem deploying to run-tests February 12, 2026 20:52 — with GitHub Actions Failure

iankhou temporarily deployed to run-tests February 12, 2026 20:52 — with GitHub Actions Inactive

watch test fix

2b1ee05

iankhou had a problem deploying to integ-approval February 13, 2026 17:56 — with GitHub Actions Failure

iankhou and others added 4 commits February 13, 2026 16:14

format

d72ce2c

chore: self mutation

dd94cc5

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

fix async operations not completed before test teardown in toolkit wa…

768aa69

…tch integ test

Merge branch 'main' into iankhou-cdk-watch-integ

e5bafec

mrgrain reviewed Feb 19, 2026

View reviewed changes

mrgrain requested changes Feb 19, 2026

View reviewed changes

mrgrain reviewed Feb 19, 2026

View reviewed changes

kaizencc reviewed Feb 19, 2026

View reviewed changes

mrgrain reviewed Feb 20, 2026

View reviewed changes

...g/tests/cli-integ-tests/watch/cdk-watch-detects-file-changes-with-glob-patterns.integtest.ts Show resolved Hide resolved

iankhou and others added 4 commits February 24, 2026 13:30

review changes

7a05e51

Merge branch 'main' into iankhou-cdk-watch-integ

a157edd

supress eslint rule import/no-extraneous-dependencies for toolkit wat…

516f438

…ch integ test

chore: self mutation

4a3e3ac

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

mrgrain reviewed Feb 25, 2026

View reviewed changes

...g/tests/cli-integ-tests/watch/cdk-watch-detects-file-changes-with-glob-patterns.integtest.ts Outdated Show resolved Hide resolved

mrgrain reviewed Feb 25, 2026

View reviewed changes

mrgrain requested changes Feb 25, 2026

View reviewed changes

mrgrain reviewed Feb 25, 2026

View reviewed changes

iankhou and others added 2 commits February 25, 2026 14:33

review changes

7d749bc

chore: self mutation

a5bde3e

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

mrgrain approved these changes Feb 25, 2026

View reviewed changes

		// Use separate output dir to avoid conflicts with lingering watch process
		await fixture.cdkDestroy('test-1', { options: ['--output', 'cdk-destroy.out'] });

	fixture.log('✓ Watch started');
	fixture.log('✓ Watch start detected');

		const detectedExcluded = output.includes('Detected change to') &&
		output.includes('excluded');

Conversation

iankhou commented Feb 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description of tests

cdk watch in CLI

Toolkit.watch()

Uh oh!

codecov-commenter commented Feb 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mrgrain Feb 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

iankhou Feb 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

iankhou commented Feb 12, 2026 •

edited

Loading

`cdk watch` in CLI

`Toolkit.watch()`

codecov-commenter commented Feb 12, 2026 •

edited

Loading

mrgrain Feb 19, 2026 •

edited

Loading

iankhou Feb 19, 2026 •

edited

Loading