Fix a bug where errors could leak on extremely large stream chunks#386
Closed
haggholm wants to merge 1 commit intoadaltas:masterfrom
Closed
Fix a bug where errors could leak on extremely large stream chunks#386haggholm wants to merge 1 commit intoadaltas:masterfrom
haggholm wants to merge 1 commit intoadaltas:masterfrom
Conversation
Member
|
Thank you for reporting and preparing a fix, we will need a test case to reproduce the issue, do you think you could prepare one ? |
Contributor
Author
|
Maybe you can adapt something from something along these lines? Throws an unhandled exception before my fix, returns it to the callback with it. Of course it's kind of annoying because it's takes up a bunch of CPU and memory, but, well, it only happens with huge strings. const {stringify} = require("csv-stringify");
const {randomBytes} = require("crypto");
stringify(Array.from({length: 200_000}).map(
() => Array.from({length: 100}).map(
() => "ABCDEFGHIJKLMNOPQRSTUVXYZ0123456789")), (err, res) => {
if (err) {
console.error("Failed with error:", err);
} else {
console.info("Content length:", res.length);
}
}); |
Member
|
Thank you! |
salceson
added a commit
to evidenceprime/node-csv
that referenced
this pull request
Sep 22, 2023
* chore: latest dependencies * fix: uncaught errors with large stream chunks (fix adaltas#386) * chore(release): publish - csv-demo-browser@0.1.6 - csv-demo-cjs@0.2.4 - csv-demo-eslint@0.1.10 - csv-demo-esm@0.0.18 - csv-issues-cjs@0.1.5 - csv-issues-esm@0.0.9 - csv-demo-ts-moduleresolution-node16-cjs@0.2.4 - csv-demo-ts-module-node16@0.2.4 - csv-demo-webpack-ts@0.1.6 - csv-demo-webpack@0.1.8 - csv-generate@4.2.3 - csv-parse@5.3.7 - csv-stringify@6.3.1 - csv@6.2.9 - stream-transform@3.2.3 * test(csv-stringify): fix legacy * chore(release): publish - csv-demo-browser@0.1.7 - csv-demo-cjs@0.2.5 - csv-demo-eslint@0.1.11 - csv-demo-esm@0.0.19 - csv-issues-cjs@0.1.6 - csv-issues-esm@0.0.10 - csv-demo-ts-moduleresolution-node16-cjs@0.2.5 - csv-demo-ts-module-node16@0.2.5 - csv-demo-webpack-ts@0.1.7 - csv-demo-webpack@0.1.9 - csv-generate@4.2.4 - csv-parse@5.3.8 - csv-stringify@6.3.2 - csv@6.2.10 - stream-transform@3.2.4 * build: remove trailing slash in home url * chore: latest dependencies * fix(csv): fixed CJS types under modern `modernResolution` options (adaltas#388) * fix(csv): remove ts files in cjs dist * chore(release): publish - csv-demo-browser@0.1.8 - csv-demo-cjs@0.2.6 - csv-demo-eslint@0.1.12 - csv-demo-esm@0.0.20 - csv-issues-cjs@0.1.7 - csv-issues-esm@0.0.11 - csv-demo-ts-moduleresolution-node16-cjs@0.2.6 - csv-demo-ts-module-node16@0.2.6 - csv-demo-webpack-ts@0.1.8 - csv-demo-webpack@0.1.10 - csv-generate@4.2.5 - csv-parse@5.3.9 - csv-stringify@6.3.3 - csv@6.2.11 - stream-transform@3.2.5 * docs: minor upercase modification * chore: latest dependencies * chore(release): publish - csv-demo-browser@0.1.9 - csv-demo-cjs@0.2.7 - csv-demo-eslint@0.1.13 - csv-demo-esm@0.0.21 - csv-issues-cjs@0.1.8 - csv-issues-esm@0.0.12 - csv-demo-ts-moduleresolution-node16-cjs@0.2.7 - csv-demo-ts-module-node16@0.2.7 - csv-demo-webpack-ts@0.1.9 - csv-demo-webpack@0.1.11 - csv-generate@4.2.6 - csv-parse@5.3.10 - csv-stringify@6.3.4 - csv@6.2.12 - stream-transform@3.2.6 * feat: add unicode chars to formula escape (adaltas#387) * fix(csv-stringify): use switch in formula escaping * fix(csv-stringify): add unicode character equivalents in formula sanitization * chore: update tests * docs(csv-stringify): escape formulas references * chore(release): publish - csv-demo-browser@0.1.10 - csv-demo-cjs@0.2.8 - csv-demo-eslint@0.1.14 - csv-demo-esm@0.0.22 - csv-issues-cjs@0.1.9 - csv-issues-esm@0.0.13 - csv-demo-ts-moduleresolution-node16-cjs@0.2.8 - csv-demo-ts-module-node16@0.2.8 - csv-demo-webpack-ts@0.1.10 - csv-demo-webpack@0.1.12 - csv-stringify@6.4.0 - csv@6.3.0 * feat(csv-parse): add `columns` property in `Info` object type (adaltas#390) * fix(ts): Add `columns` property in `Info` object type * Add disabled options to columns type * build(csv-parse): build and write test after info ts definition * chore(release): publish - csv-demo-browser@0.1.11 - csv-demo-cjs@0.2.9 - csv-demo-esm@0.0.23 - csv-issues-cjs@0.1.10 - csv-issues-esm@0.0.14 - csv-demo-ts-moduleresolution-node16-cjs@0.2.9 - csv-demo-ts-module-node16@0.2.9 - csv-demo-webpack-ts@0.1.11 - csv-demo-webpack@0.1.13 - csv-parse@5.4.0 - csv@6.3.1 * docs: update build badge urls * docs(csv-generate): comment indentation in samples * refactor(csv-issues-cjs): code format * refactor(csv-issues-cjs): remove unused arguments * test(csv-issues-cjs): fix stdout maxBuffer length exceeded * test(csv-issues-esm): use spawn instead of exec * fix: commonjs types, run tsc and lint to validate changes (adaltas#397) * fix: types weren't working for commonjs. Run tsc and lint to validate changes * chore: needs to work on linux and BSD * chore: latest dependencies * chore(release): publish - csv-demo-browser@0.1.12 - csv-demo-cjs@0.2.10 - csv-demo-eslint@0.1.15 - csv-demo-esm@0.0.24 - csv-issues-cjs@0.1.11 - csv-issues-esm@0.0.15 - csv-demo-ts-moduleresolution-node16-cjs@0.2.10 - csv-demo-ts-module-node16@0.2.10 - csv-demo-webpack-ts@0.1.12 - csv-demo-webpack@0.1.14 - csv-generate@4.2.7 - csv-parse@5.4.1 - csv-stringify@6.4.1 - csv@6.3.2 - stream-transform@3.2.7 * feat(csv-issues-cjs): 399 issue * fix(csv-demo-ts-cjs-node16): upgrade module definition after latest typescript * feat(csv-parse): new comment_no_infix option (fix adaltas#325) * test(csv-issues-esm): reproduce issue adaltas#391 * refactor(csv-stringify): rename variable in sample * test(csv-issues-cjs): reproduce issue 327 * chore(release): publish - csv-demo-browser@0.1.13 - csv-demo-cjs@0.2.11 - csv-demo-eslint@0.1.16 - csv-demo-esm@0.0.25 - csv-issues-cjs@0.2.0 - csv-issues-esm@0.0.16 - csv-demo-ts-cjs-node16@0.2.11 - csv-demo-ts-module-node16@0.2.11 - csv-demo-webpack-ts@0.1.13 - csv-demo-webpack@0.1.15 - csv-generate@4.2.8 - csv-parse@5.5.0 - csv-stringify@6.4.2 - csv@6.3.3 - stream-transform@3.2.8 * docs(csv-parse): comment_no_infix sample --------- Co-authored-by: David Worms <david@adaltas.com> Co-authored-by: Petter <petter@petterhaggholm.net> Co-authored-by: Mateusz Burzyński <mateuszburzynski@gmail.com> Co-authored-by: Tom Emelko <tom.emelko@gmail.com> Co-authored-by: Elia Maino <eliamaino@gmail.com> Co-authored-by: David Tanner <darthtanner@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
I don't expect a lot of people are going to see this, but we have encountered an issue where an error is thrown as an unhandled exception:
This happens here:
Obviously, if
chunks.join()throws an error, it is leaked rather than passed to the callback. I suppose you don't normally expectchunks.join()to fail, but it is in fact posssible.I'm not sure what style you prefer for this sort of thing; this ought to work and be proof to double-invoking callbacks, but feel free to edit as needed or desired, obviously.