Skip to content

Conversation

@devversion
Copy link
Member

@devversion devversion commented Mar 2, 2023

The run-example-e2e script does not properly fail if configured
tests of examples are failing. This happens when a CLI example
configures multiple tests in the example-config. Due to incorrect
usage of promises in combination with reduce, only the last test
command had an effect on the overall test conclusion. A similar issue
seems to occur with SystemJS Protractor tests.

This commit fixes the problem and also cleans up the code a little
by switching it to async/await.

Additionally we fix the following two things:

  • Examples that were already broken but just didn't fail..
  • Local AIO tests had significant failures where incorrect transitive local dependencies were resolved..

@angular-robot angular-robot bot added the area: build & ci Related the build and CI infrastructure of the project label Mar 2, 2023
@ngbot ngbot bot added this to the Backlog milestone Mar 2, 2023
@jessicajaniuk jessicajaniuk removed the area: build & ci Related the build and CI infrastructure of the project label Mar 2, 2023
@ngbot ngbot bot removed this from the Backlog milestone Mar 2, 2023
@jessicajaniuk jessicajaniuk added the area: build & ci Related the build and CI infrastructure of the project label Mar 2, 2023
@ngbot ngbot bot added this to the Backlog milestone Mar 2, 2023
Formats the `run-example-e2e.mjs` file with prettier to ease
future diffs.
… situations

The `run-example-e2e` script does not properly fail if configured
tests of examples are failing. This happens when a CLI example
configures multiple tests in the `example-config`. Due to incorrect
usage of promises in combination with reduce, only the last test
command had an effect on the overall test conclusion.

A similar issue seems to occur with SystemJS Protractor tests.

This commit fixes the problem and also cleans up the code a little
by switching it to `async/await`.
… example

This causes the e2e tests to fail. The tests were compiled using a
tsconfig with `lib=dom`, but the tests explicitly tried to pull in the
Node types. This causes a conflict for e.g. `AbortController` types.
Whenever we run example tests using the local framework packages, the
e2e tests will have the local framework packages symlinked in the
`node_modules`.

This works well in general, but due to NodeJS by default resolving
symlinks to the target location, NodeJS will end up looking for
transitive dependencies in the `bazel-bin` instead of in the example
`node_modules` folder. This means that we end up incorrectly resolving
older versions of `@angular/core` that end up existing in the main
project dependencies. This causes errors like:

```
Error: ../../home/circleci/.cache/bazel/_bazel_circleci/9ce5c2144ecf75d11717c0aa41e45a8d/execroot/angular/bazel-out/k8-fastbuild/bin/packages/common/npm_package/http/testing/index.d.ts:12:21 - error TS2307: Cannot find module '@angular/common/http' or its corresponding type declarations.

12 import * as i1 from '@angular/common/http';
                       ~~~~~~~~~~~~~~~~~~~~~~

Error: ../../home/circleci/.cache/bazel/_bazel_circleci/9ce5c2144ecf75d11717c0aa41e45a8d/execroot/angular/bazel-out/k8-fastbuild/bin/packages/common/npm_package/index.d.ts:1630:18 - error TS2707: Generic type 'ɵɵDirectiveDeclaration' requires between 6 and 8 type arguments.

1630     static ɵdir: i0.ɵɵDirectiveDeclaration<NgClass, "[ngClass]", never, { "klass": "class"; "ngClass": "ngClass"; }, {}, never, never, true, never>;
                      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
```

We can fix this by properly ensuring that NodeJS does not resolve
symlinks, but rather preserves them.

In the error above, the e2e tests end up accidentally resolving
`@angular/core` v14 that comes from `@angular/benchpress`. Angular
Benchpress is installed via `@angular/build-tooling` in the project
root.
…AIO example e2e tests

Chromium is launched via Karma from within the Bazel AIO example e2e
tests. This breaks depending on the platform and sandbox mechanism used.

We should never use Chromium's sandbox on top of Bazel's sandbox
invocation. The Angular CLI exposes a browser exactly for this use-case.
Currently, examples with test commands like `ng build` are *never*
using the local version of `//packages/compiler-cli`. This is because
the CLI is invoked accidentally from within `external/aio_example_deps`.

Since the CLI relies on importing the compiler-cli, it will always
resolve the dependency from that directory- causing it to be always
the version installed via `aio/examples/tools/shared/package.json`.

We should never resolve symlinks and escape the e2e sandbox. That way
the compiler-cli would be resolved properly and could also become the
locally built one, depending on the test mode (i.e. npm or "local").
We recently migrated the testing example to use the router testing
harness. There was one instance of the previous non-`TestBed` examples
left that still relies on a stub that has already been removed.

This example is not used anywhere and we should rather encourage
a single pattern of testing. i.e. the harness as per recent changes.

This commit removes the broken file.
@devversion devversion force-pushed the fix-e2e-example-testing branch from 2bd3942 to 08dacb4 Compare March 3, 2023 16:10
@devversion devversion requested a review from atscott March 3, 2023 16:34
@devversion devversion added the action: review The PR is still awaiting reviews from at least one requested reviewer label Mar 3, 2023
@devversion devversion marked this pull request as ready for review March 3, 2023 16:34
@devversion
Copy link
Member Author

Note: Disabling pull approve since we have docs infra approval and LGTM the examples where actual docs content changed

@devversion devversion added action: merge The PR is ready for merge by the caretaker target: patch This PR is targeted for the next patch release and removed action: review The PR is still awaiting reviews from at least one requested reviewer labels Mar 3, 2023
@jessicajaniuk
Copy link
Contributor

This PR was merged into the repository by commit 47dda6e.

jessicajaniuk pushed a commit that referenced this pull request Mar 3, 2023
Formats the `run-example-e2e.mjs` file with prettier to ease
future diffs.

PR Close #49293
jessicajaniuk pushed a commit that referenced this pull request Mar 3, 2023
… situations (#49293)

The `run-example-e2e` script does not properly fail if configured
tests of examples are failing. This happens when a CLI example
configures multiple tests in the `example-config`. Due to incorrect
usage of promises in combination with reduce, only the last test
command had an effect on the overall test conclusion.

A similar issue seems to occur with SystemJS Protractor tests.

This commit fixes the problem and also cleans up the code a little
by switching it to `async/await`.

PR Close #49293
jessicajaniuk pushed a commit that referenced this pull request Mar 3, 2023
… example (#49293)

This causes the e2e tests to fail. The tests were compiled using a
tsconfig with `lib=dom`, but the tests explicitly tried to pull in the
Node types. This causes a conflict for e.g. `AbortController` types.

PR Close #49293
jessicajaniuk pushed a commit that referenced this pull request Mar 3, 2023
…49293)

Whenever we run example tests using the local framework packages, the
e2e tests will have the local framework packages symlinked in the
`node_modules`.

This works well in general, but due to NodeJS by default resolving
symlinks to the target location, NodeJS will end up looking for
transitive dependencies in the `bazel-bin` instead of in the example
`node_modules` folder. This means that we end up incorrectly resolving
older versions of `@angular/core` that end up existing in the main
project dependencies. This causes errors like:

```
Error: ../../home/circleci/.cache/bazel/_bazel_circleci/9ce5c2144ecf75d11717c0aa41e45a8d/execroot/angular/bazel-out/k8-fastbuild/bin/packages/common/npm_package/http/testing/index.d.ts:12:21 - error TS2307: Cannot find module '@angular/common/http' or its corresponding type declarations.

12 import * as i1 from '@angular/common/http';
                       ~~~~~~~~~~~~~~~~~~~~~~

Error: ../../home/circleci/.cache/bazel/_bazel_circleci/9ce5c2144ecf75d11717c0aa41e45a8d/execroot/angular/bazel-out/k8-fastbuild/bin/packages/common/npm_package/index.d.ts:1630:18 - error TS2707: Generic type 'ɵɵDirectiveDeclaration' requires between 6 and 8 type arguments.

1630     static ɵdir: i0.ɵɵDirectiveDeclaration<NgClass, "[ngClass]", never, { "klass": "class"; "ngClass": "ngClass"; }, {}, never, never, true, never>;
                      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
```

We can fix this by properly ensuring that NodeJS does not resolve
symlinks, but rather preserves them.

In the error above, the e2e tests end up accidentally resolving
`@angular/core` v14 that comes from `@angular/benchpress`. Angular
Benchpress is installed via `@angular/build-tooling` in the project
root.

PR Close #49293
jessicajaniuk pushed a commit that referenced this pull request Mar 3, 2023
…AIO example e2e tests (#49293)

Chromium is launched via Karma from within the Bazel AIO example e2e
tests. This breaks depending on the platform and sandbox mechanism used.

We should never use Chromium's sandbox on top of Bazel's sandbox
invocation. The Angular CLI exposes a browser exactly for this use-case.

PR Close #49293
jessicajaniuk pushed a commit that referenced this pull request Mar 3, 2023
)

Currently, examples with test commands like `ng build` are *never*
using the local version of `//packages/compiler-cli`. This is because
the CLI is invoked accidentally from within `external/aio_example_deps`.

Since the CLI relies on importing the compiler-cli, it will always
resolve the dependency from that directory- causing it to be always
the version installed via `aio/examples/tools/shared/package.json`.

We should never resolve symlinks and escape the e2e sandbox. That way
the compiler-cli would be resolved properly and could also become the
locally built one, depending on the test mode (i.e. npm or "local").

PR Close #49293
jessicajaniuk pushed a commit that referenced this pull request Mar 3, 2023
…49293)

We recently migrated the testing example to use the router testing
harness. There was one instance of the previous non-`TestBed` examples
left that still relies on a stub that has already been removed.

This example is not used anywhere and we should rather encourage
a single pattern of testing. i.e. the harness as per recent changes.

This commit removes the broken file.

PR Close #49293
jessicajaniuk pushed a commit that referenced this pull request Mar 3, 2023
… situations (#49293)

The `run-example-e2e` script does not properly fail if configured
tests of examples are failing. This happens when a CLI example
configures multiple tests in the `example-config`. Due to incorrect
usage of promises in combination with reduce, only the last test
command had an effect on the overall test conclusion.

A similar issue seems to occur with SystemJS Protractor tests.

This commit fixes the problem and also cleans up the code a little
by switching it to `async/await`.

PR Close #49293
jessicajaniuk pushed a commit that referenced this pull request Mar 3, 2023
… example (#49293)

This causes the e2e tests to fail. The tests were compiled using a
tsconfig with `lib=dom`, but the tests explicitly tried to pull in the
Node types. This causes a conflict for e.g. `AbortController` types.

PR Close #49293
jessicajaniuk pushed a commit that referenced this pull request Mar 3, 2023
…49293)

Whenever we run example tests using the local framework packages, the
e2e tests will have the local framework packages symlinked in the
`node_modules`.

This works well in general, but due to NodeJS by default resolving
symlinks to the target location, NodeJS will end up looking for
transitive dependencies in the `bazel-bin` instead of in the example
`node_modules` folder. This means that we end up incorrectly resolving
older versions of `@angular/core` that end up existing in the main
project dependencies. This causes errors like:

```
Error: ../../home/circleci/.cache/bazel/_bazel_circleci/9ce5c2144ecf75d11717c0aa41e45a8d/execroot/angular/bazel-out/k8-fastbuild/bin/packages/common/npm_package/http/testing/index.d.ts:12:21 - error TS2307: Cannot find module '@angular/common/http' or its corresponding type declarations.

12 import * as i1 from '@angular/common/http';
                       ~~~~~~~~~~~~~~~~~~~~~~

Error: ../../home/circleci/.cache/bazel/_bazel_circleci/9ce5c2144ecf75d11717c0aa41e45a8d/execroot/angular/bazel-out/k8-fastbuild/bin/packages/common/npm_package/index.d.ts:1630:18 - error TS2707: Generic type 'ɵɵDirectiveDeclaration' requires between 6 and 8 type arguments.

1630     static ɵdir: i0.ɵɵDirectiveDeclaration<NgClass, "[ngClass]", never, { "klass": "class"; "ngClass": "ngClass"; }, {}, never, never, true, never>;
                      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
```

We can fix this by properly ensuring that NodeJS does not resolve
symlinks, but rather preserves them.

In the error above, the e2e tests end up accidentally resolving
`@angular/core` v14 that comes from `@angular/benchpress`. Angular
Benchpress is installed via `@angular/build-tooling` in the project
root.

PR Close #49293
jessicajaniuk pushed a commit that referenced this pull request Mar 3, 2023
…AIO example e2e tests (#49293)

Chromium is launched via Karma from within the Bazel AIO example e2e
tests. This breaks depending on the platform and sandbox mechanism used.

We should never use Chromium's sandbox on top of Bazel's sandbox
invocation. The Angular CLI exposes a browser exactly for this use-case.

PR Close #49293
jessicajaniuk pushed a commit that referenced this pull request Mar 3, 2023
)

Currently, examples with test commands like `ng build` are *never*
using the local version of `//packages/compiler-cli`. This is because
the CLI is invoked accidentally from within `external/aio_example_deps`.

Since the CLI relies on importing the compiler-cli, it will always
resolve the dependency from that directory- causing it to be always
the version installed via `aio/examples/tools/shared/package.json`.

We should never resolve symlinks and escape the e2e sandbox. That way
the compiler-cli would be resolved properly and could also become the
locally built one, depending on the test mode (i.e. npm or "local").

PR Close #49293
jessicajaniuk pushed a commit that referenced this pull request Mar 3, 2023
…49293)

We recently migrated the testing example to use the router testing
harness. There was one instance of the previous non-`TestBed` examples
left that still relies on a stub that has already been removed.

This example is not used anywhere and we should rather encourage
a single pattern of testing. i.e. the harness as per recent changes.

This commit removes the broken file.

PR Close #49293
@angular-automatic-lock-bot
Copy link

This issue has been automatically locked due to inactivity.
Please file a new issue if you are encountering a similar or related problem.

Read more about our automatic conversation locking policy.

This action has been performed automatically by a bot.

@angular-automatic-lock-bot angular-automatic-lock-bot bot locked and limited conversation to collaborators Apr 3, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

action: merge The PR is ready for merge by the caretaker area: build & ci Related the build and CI infrastructure of the project PullApprove: disable target: patch This PR is targeted for the next patch release

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants