pkg(profiling-node): port profiling-node repo to monorepo#10151
pkg(profiling-node): port profiling-node repo to monorepo#10151
Conversation
10caf39 to
14d19b4
Compare
size-limit report 📦
|
8d98bdd to
febd09b
Compare
|
I think I spent like 1 day on the fact that running lerna run build between node-gyp configure node-gyp build somehow breaks gyp... |
|
Seems like we might be gucci on the prebuild, however fetching dep for the repo is so slow on windows it times out. Going to try and bump it |
dev-packages/e2e-tests/test-applications/node-profiling/build.mjs
Outdated
Show resolved
Hide resolved
| "build:transpile": { | ||
| "dependsOn": [ | ||
| "^build:transpile", | ||
| "^build:types" |
There was a problem hiding this comment.
Shouldn't this be included in the nx.json in root?
AbhiPrasad
left a comment
There was a problem hiding this comment.
CI timings will increase mostly because of the performance of windows runners
We just need bigger runners, but lets see how much this slows us down first after we merge.
AbhiPrasad
left a comment
There was a problem hiding this comment.
All good on my side! Let's not merge just yet - would like a team member to sanity check craft and CI config
| - name: Extract Profiling Node Prebuilt Binaries | ||
| # @TODO: v4 breaks convenient merging of same name artifacts | ||
| # https://github.com/actions/upload-artifact/issues/478 | ||
| uses: actions/download-artifact@v3 |
There was a problem hiding this comment.
Can we use the workaround in actions/upload-artifact#478 (comment) here?
There was a problem hiding this comment.
Generally looks good to me! Still had a couple of questions regarding publishing config.
As well as two requests:
- If not done yet, can you go through
docs/new-sdk-release-checklist.mdto make sure we're not missing something? Some points might not apply, given it's a migration rather than a first time publish but it probably makes sense to check anyway. - Also, I'd recommend pushing a
release/testbranch to the repo and checking the build artifacts to make sure that everything is in the tarball as expected. From what I've seen we should be good but it probably makes sense anyway to double check.
Last question: Once we cut the first release with this package, the version will jump from 1.x to 7.x. Are you fine with that or do we need to stay on 1.x for a while? (maybe that's the reason for the separate version bump files but I'm not sure if craft/our publishing can actually handle this).
Yes, definitely. I did not know it existed, but I'll go through :)
Good idea.
Yes that is ok. I also removed node 14 from the precompile matrix so a new major change will be welcome |
|
@lms @AbhiPrasad to minimize impact on CI times, I made the change to only run the compile bindings job if profiling or nodejs package has changed or if we are on a release branch. |
AbhiPrasad
left a comment
There was a problem hiding this comment.
Let's merge!
The amount of beers I owe you continue to climb @JonasBa - I'll be bankrupt at this rate 🤣
Lms24
left a comment
There was a problem hiding this comment.
Yup let's go! Thanks for applying my suggestions!
|
@lms @AbhiPrasad let me update this branch and merge it first thing start of next week. I will also update final artifact action to v4 |
0a9387a to
8fa4055
Compare
e10e477 to
0ba4761
Compare
Ports profiling-node to monorepo
This should only require CI step changes and no changes to the source
code (except inheriting from base configs in monorepo, but those were
already ported to profiling-node so they shouldn't result in any actual
changes to the codebase).
TODO:
- [x] verify pkg cmds work and we have no lint/ts issues.
- [x] verify tests pass
- [x] verify ci commands work
- [x] prebuild binaries
- [x] port build to rollup
- [x] create e2e verdaccio config
- [x] ensure e2e tests pass and the app can build.
I'm opening this as ready to review. Currently e2e test on profiling
fails as the package is missing. I'm not exactly sure why that is so
hoping I can get a helping hand on that. The condition for a successful
e2e test is to just execute a node script which triggers a profile and
attempt to bundle profiling-node. The bundler test is there to ensure
that we do in fact provide all of the statically required prebuild
binaries and ensure bundlers can resolve them - else we risk breaking
build tooling for folks bundling their code, which can be a common
optimization in serverless environments.
The good:
- Node profiling can resolve some issues around types which kept falling
out fo sync with the SDK
- We can follow the changes in JS SDK and core packages along as well as
their versioning
- The integration between the rest of the packages is tighter and safer,
as it should be impossible for the sentry-javascript packages we rely on
now to break in profiling-node.
The unfortunate:
- CI timings will increase mostly because of the performance of windows
runners. Installing repository dependencies on there takes roughly 10
minutes. I have tried leveraging the cache as much as I could, but since
our dep paths are marked as `~/path/to/cached_dep` and
`${{github.workspace}}/path/to/dep` it means they cannot be cached cross
os as paths differ. I did not change those paths in this PR, but it can
be an optimization as I suspect most of the packages are pure js and
nothing would break.
- When we pack artifacts in yarn build:tarball, we need skip
profiling-node and assemble it separately. This is because we need to
pull in the prebuilt binaries, else we'll only have the packed binary
for the os we are running the action on. This same step needs to be
replicated in e2e tests which prepare the tarball as well.
Couple of things I had to fix:
- Our build tooling was not windows compatible (util polyfill was using
awrong os path separator which wound up creating an incompatible build
output on windows)
- CI is finicky. I learned the hard way that node-gyp configure and
build need to be sequential, else all hell breaks loose and for reasons
which I didn't bother to investigate, python path and tooling is never
correctly resolved (even when specified via gyp arg)
Ports profiling-node to monorepo
This should only require CI step changes and no changes to the source code (except inheriting from base configs in monorepo, but those were already ported to profiling-node so they shouldn't result in any actual changes to the codebase).
TODO:
I'm opening this as ready to review. Currently e2e test on profiling fails as the package is missing. I'm not exactly sure why that is so hoping I can get a helping hand on that. The condition for a successful e2e test is to just execute a node script which triggers a profile and attempt to bundle profiling-node. The bundler test is there to ensure that we do in fact provide all of the statically required prebuild binaries and ensure bundlers can resolve them - else we risk breaking build tooling for folks bundling their code, which can be a common optimization in serverless environments.
The good:
The unfortunate:
~/path/to/cached_depand${{github.workspace}}/path/to/depit means they cannot be cached cross os as paths differ. I did not change those paths in this PR, but it can be an optimization as I suspect most of the packages are pure js and nothing would break.Couple of things I had to fix: