Fix stream piping #670
Merged
Fix stream piping #670
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Collaborator
|
lgtm, would be perfect if we can come up with a test case. |
Collaborator
|
looking at this again, I think we need more than just using |
Member
Author
|
I don't really know how to test this, looking at this comment we might not even be able to 😢 |
Richienb
reviewed
Sep 9, 2019
Member
Richienb
left a comment
There was a problem hiding this comment.
Are we planning to embed the code during compilation or depend on it?
Member
Author
|
@Richienb We should depend on it. |
Member
Author
Member
Author
|
I accidentally published a branch called Unfortunately because of that Travis build is broken and I'm not sure how to fix that 😢 |
Member
|
@xxczaki The best way to fix the Travis CI build is to trigger a build on a known good branch to restore the status. |
Signed-off-by: Richie Bendall <richiebendall@gmail.com>
Richienb
approved these changes
Sep 11, 2019
bitinn
approved these changes
Sep 12, 2019
Collaborator
bitinn
left a comment
There was a problem hiding this comment.
LGTM at a glimpse, my only concern is whether this will work in production without we testing it, if I had a lot of money I would put up a bounty for a test case :)
Member
Author
|
@bitinn Good idea, we can merge this, create an issue that we need tests for stream piping and then use issuehunt.io to request a bounty from them (they will use money from their membership subscribers to fund this issue). Even though they often reject bounty requests, I guess that considering the popularity of node-fetch (~10M monthly downloads) they will agree 😄 I can’t request a bounty from them, as I’m not a owner of this repository. |
Member
Author
|
I think it's time to merge this 😄 |
bitinn
added a commit
that referenced
this pull request
Mar 13, 2020
* feat: Migrate TypeScript types (#669) * style: Introduce linting via XO * fix: Fix tests * chore!: Drop support for nodejs 4 and 6 * chore: Fix Travis CI yml * Use old Babel (needs migration) * chore: lint everything * chore: Migrate to microbundle * Default response.statusText should be blank (#578) * fix: Use correct AbortionError message Signed-off-by: Richie Bendall <richiebendall@gmail.com> * chore: Use modern @babel/register Signed-off-by: Richie Bendall <richiebendall@gmail.com> * chore: Remove redundant packages Signed-off-by: Richie Bendall <richiebendall@gmail.com> * chore: Readd form-data Signed-off-by: Richie Bendall <richiebendall@gmail.com> * fix: Fix tests and force utf8-encoded urls Signed-off-by: Richie Bendall <richiebendall@gmail.com> * lint index.js * Update devDependencies & ignore `test` directory in linter options * Remove unnecessary eslint-ignore comment * Update the `lint` script to run linter on every file * Remove unused const & unnecessary import * TypeScript: Fix Body.blob() wrong type (DefinitelyTyped/DefinitelyTyped#33721) * chore: Lint as part of the build process * fix: Convert Content-Encoding to lowercase (#672) * fix: Better object checks (#673) * Fix stream piping (#670) * chore: Remove useless check Signed-off-by: Richie Bendall <richiebendall@gmail.com> * style: Fix lint Signed-off-by: Richie Bendall <richiebendall@gmail.com> * style: Fix lint Signed-off-by: Richie Bendall <richiebendall@gmail.com> * refactor: Modernise code Signed-off-by: Richie Bendall <richiebendall@gmail.com> * chore: Ensure all files are properly included Signed-off-by: Richie Bendall <richiebendall@gmail.com> * chore: Update deps and utf8 should be in dependencies Signed-off-by: Richie Bendall <richiebendall@gmail.com> * test: Drop Node v4 from tests Signed-off-by: Richie Bendall <richiebendall@gmail.com> * test: Modernise code Signed-off-by: Richie Bendall <richiebendall@gmail.com> * chore: Move errors to seperate directory Signed-off-by: Richie Bendall <richiebendall@gmail.com> * refactor: Add fetch-blob (#678) * feat: Migrate data uri integration Signed-off-by: Richie Bendall <richiebendall@gmail.com> * Allow setting custom highWaterMark via node-fetch options (#386) (#671) * Expose highWaterMark option to body clone function * Add highWaterMark to responseOptions * Add highWaterMark as node-fetch-only option * a way to silently pass highWaterMark to clone * Chai helper * Server helper * Tests * Remove debug comments * Document highWaterMark option * Add TypeScript types for the new highWaterMark option * feat: Include system error in FetchError if one occurs (#654) * style: Add editorconfig Signed-off-by: Richie Bendall <richiebendall@gmail.com> * chore!: Drop NodeJS v8 Signed-off-by: Richie Bendall <richiebendall@gmail.com> * chore: Remove legacy code for node < 8 Signed-off-by: Richie Bendall <richiebendall@gmail.com> * chore: Use proper checks for ArrayBuffer and AbortError Signed-off-by: Richie Bendall <richiebendall@gmail.com> * chore: Use explicitly set error name in checks Signed-off-by: Richie Bendall <richiebendall@gmail.com> * Propagate size and timeout to cloned response (#664) * Remove --save option as it isn't required anymore (#581) * Propagate size and timeout to cloned response Co-authored-by: Steve Moser <contact@stevemoser.org> Co-authored-by: Antoni Kepinski <xxczaki@pm.me> * Update Response types * Update devDependencies * feat: Fallback to blob type (Closes: #607) Signed-off-by: Richie Bendall <richiebendall@gmail.com> * style: Update formatting Signed-off-by: Richie Bendall <richiebendall@gmail.com> * style: Fix linting issues Signed-off-by: Richie Bendall <richiebendall@gmail.com> * docs: Add info on patching the global object * docs: Added non-globalThis polyfill * Replace deprecated `url.resolve` with the new WHATWG URL * Update devDependencies * Format code in examples to use `xo` style * Verify examples with RunKit and edit them if necessary * Add information about TypeScript support * Document the new `highWaterMark` option * Add Discord badge & information about Open Collective * Style change * Edit acknowledgement & add "Team" section * fix table * Format example code to use xo style * chore: v3 release changelog * Add the recommended way to fix `highWaterMark` issues * docs: Add simple Runkit example * fix: Properly set the name of the errors. Signed-off-by: Richie Bendall <richiebendall@gmail.com> * docs: Add AbortError to documented types Signed-off-by: Richie Bendall <richiebendall@gmail.com> * docs: AbortError proper typing parameters Signed-off-by: Richie Bendall <richiebendall@gmail.com> * docs: Add example code for Runkit Signed-off-by: Richie Bendall <richiebendall@gmail.com> * Replace microbundle with @pika/pack (#689) * gitignore the pkg/ directory * Move TypeScript types to the root of the project * Replace microbundle with @pika/pack * chore: Remove @pika/plugin-build-web and revert ./dist output directory Signed-off-by: Richie Bendall <richiebendall@gmail.com> Co-authored-by: Richie Bendall <richiebendall@gmail.com> * fix incorrect statement in changelog * chore: v3.x upgrade guide * Change the Open Collective button * docs: Encode support button as Markdown instead of HTML * chore: Ignore proper directory in xo * Add an "Upgrading" section to readme * Split the upgrade guide into 2 files & add the missing changes about v3.x * style: Lint test and example files Signed-off-by: Richie Bendall <richiebendall@gmail.com> * Move *.md files to the `docs` folder (except README.md) * Update references to files * Split LIMITS.md into 2 files (as of v2.x and v3.x) * chore: Remove logging statement Signed-off-by: Richie Bendall <richiebendall@gmail.com> * style: Fix lint * docs: Correct typings for systemError in FetchError (Fixes #697) * refactor: Replace `encoding` with `fetch-charset-detection`. (#694) * refactor: Replace `encoding` with `fetch-charset-detection`. Signed-off-by: Richie Bendall <richiebendall@gmail.com> * refactor: Move writing to stream back to body.js Signed-off-by: Richie Bendall <richiebendall@gmail.com> * refactor: Only put convertBody in fetch-charset-detection and refactor others. Signed-off-by: Richie Bendall <richiebendall@gmail.com> * test: Readd tests for getTotalBytes and extractContentType Signed-off-by: Richie Bendall <richiebendall@gmail.com> * chore: Revert package.json indention Signed-off-by: Richie Bendall <richiebendall@gmail.com> * chore: Remove optional dependency * docs: Replace code for fetch-charset-detection with documentation. Signed-off-by: Richie Bendall <richiebendall@gmail.com> * chore: Remove iconv-lite * fix: Use default export instead of named export for convertBody Signed-off-by: Richie Bendall <richiebendall@gmail.com> * chore: Remove unneeded installation of fetch-charset-detection in the build * docs: Fix typo * fix: Throw SyntaxError instead of FetchError in case of invalid… (#700) * fix: Throw SyntaxError instead of FetchError in case of invalid JSON Signed-off-by: Richie Bendall <richiebendall@gmail.com> * docs: Add to upgrade guide Signed-off-by: Richie Bendall <richiebendall@gmail.com> * Remove deprecated url.parse from test * Remove deprecated url.parse from server * fix: Proper data uri to buffer conversion (#703) Signed-off-by: Richie Bendall <richiebendall@gmail.com> * chore: Add funding info * fix: Flawed property existence test (#706) Fix a problem where not all prototype methods are copied from the Body via the mixin method due to a failure to properly detect properties in the target. The current code uses the `in` operator, which may return properties lower down the inheritance chain, thus causing them to fail the copy. The new code properly calls the `.hasOwnProperty()` method to make the determination. * fix: Properly handle stream pipeline double-fire Signed-off-by: Richie Bendall <richiebendall@gmail.com> * docs: Fix spelling Signed-off-by: Richie Bendall <richiebendall@gmail.com> * chore: Add `funding` field to package.json (#708) * Fix: Do not set ContentLength to NaN (#709) * do not set ContentLength to NaN * lint * docs: Add logo Signed-off-by: Richie Bendall <richiebendall@gmail.com> * chore: Update repository name from bitinn/node-fetch to node-fetch/node-fetch. Signed-off-by: Richie Bendall <richiebendall@gmail.com> * chore: Fix unit tests Signed-off-by: Richie Bendall <richiebendall@gmail.com> * chore(deps): Bump @pika/plugin-copy-assets from 0.7.1 to 0.8.1 (#713) Bumps [@pika/plugin-copy-assets](https://github.com/pikapkg/builders) from 0.7.1 to 0.8.1. - [Release notes](https://github.com/pikapkg/builders/releases) - [Commits](FredKSchott/pika-pack-builders@v0.7.1...v0.8.1) Signed-off-by: dependabot-preview[bot] <support@dependabot.com> * chore(deps): Bump @pika/plugin-build-types from 0.7.1 to 0.8.1 (#710) Bumps [@pika/plugin-build-types](https://github.com/pikapkg/builders) from 0.7.1 to 0.8.1. - [Release notes](https://github.com/pikapkg/builders/releases) - [Commits](FredKSchott/pika-pack-builders@v0.7.1...v0.8.1) Signed-off-by: dependabot-preview[bot] <support@dependabot.com> * Bump nyc from 14.1.1 to 15.0.0 (#714) Bumps [nyc](https://github.com/istanbuljs/nyc) from 14.1.1 to 15.0.0. - [Release notes](https://github.com/istanbuljs/nyc/releases) - [Changelog](https://github.com/istanbuljs/nyc/blob/master/CHANGELOG.md) - [Commits](istanbuljs/nyc@v14.1.1...v15.0.0) Signed-off-by: dependabot-preview[bot] <support@dependabot.com> * chore: Update travis ci url Signed-off-by: Richie Bendall <richiebendall@gmail.com> * chore(deps): Bump mocha from 6.2.2 to 7.0.0 (#711) Bumps [mocha](https://github.com/mochajs/mocha) from 6.2.2 to 7.0.0. - [Release notes](https://github.com/mochajs/mocha/releases) - [Changelog](https://github.com/mochajs/mocha/blob/master/CHANGELOG.md) - [Commits](mochajs/mocha@v6.2.2...v7.0.0) Signed-off-by: dependabot-preview[bot] <support@dependabot.com> * feat: Allow excluding a user agent in a fetch request by setting… (#715) Signed-off-by: Richie Bendall <richiebendall@gmail.com> * Bump @pika/plugin-build-node from 0.7.1 to 0.8.1 (#717) Bumps [@pika/plugin-build-node](https://github.com/pikapkg/builders) from 0.7.1 to 0.8.1. - [Release notes](https://github.com/pikapkg/builders/releases) - [Commits](FredKSchott/pika-pack-builders@v0.7.1...v0.8.1) Signed-off-by: dependabot-preview[bot] <support@dependabot.com> * Bump @pika/plugin-standard-pkg from 0.7.1 to 0.8.1 (#716) Bumps [@pika/plugin-standard-pkg](https://github.com/pikapkg/builders) from 0.7.1 to 0.8.1. - [Release notes](https://github.com/pikapkg/builders/releases) - [Commits](FredKSchott/pika-pack-builders@v0.7.1...v0.8.1) Signed-off-by: dependabot-preview[bot] <support@dependabot.com> * Bump form-data from 2.5.1 to 3.0.0 (#712) Bumps [form-data](https://github.com/form-data/form-data) from 2.5.1 to 3.0.0. - [Release notes](https://github.com/form-data/form-data/releases) - [Commits](form-data/form-data@v2.5.1...v3.0.0) Signed-off-by: dependabot-preview[bot] <support@dependabot.com> * fix: typo * update suggestion * feat: Added missing redirect function (#718) * added missing redirect function * chore: Add types Co-authored-by: Richie Bendall <richiebendall@gmail.com> * fix: Use req.setTimeout for timeout (#719) * chore: Update typings comment Signed-off-by: Richie Bendall <richiebendall@gmail.com> * chore: Update deps Signed-off-by: Richie Bendall <richiebendall@gmail.com> * docs: center badges & Open Collective button * docs: add missing comma * Remove current stable & LTS node version numbers from the comments I don't think we really want to update them * Bump xo from 0.25.4 to 0.26.1 (#730) Bumps [xo](https://github.com/xojs/xo) from 0.25.4 to 0.26.1. - [Release notes](https://github.com/xojs/xo/releases) - [Commits](xojs/xo@v0.25.4...v0.26.1) Signed-off-by: dependabot-preview[bot] <support@dependabot.com> * Bump @pika/plugin-build-types from 0.8.3 to 0.9.2 (#729) Bumps [@pika/plugin-build-types](https://github.com/pikapkg/builders) from 0.8.3 to 0.9.2. - [Release notes](https://github.com/pikapkg/builders/releases) - [Commits](FredKSchott/pika-pack-builders@v0.8.3...v0.9.2) Signed-off-by: dependabot-preview[bot] <support@dependabot.com> * Bump @pika/plugin-standard-pkg from 0.8.3 to 0.9.2 (#726) Bumps [@pika/plugin-standard-pkg](https://github.com/pikapkg/builders) from 0.8.3 to 0.9.2. - [Release notes](https://github.com/pikapkg/builders/releases) - [Commits](FredKSchott/pika-pack-builders@v0.8.3...v0.9.2) Signed-off-by: dependabot-preview[bot] <support@dependabot.com> * docs: Update information about `req.body` type in v3.x release * Add information about removed body type to the v3 upgrade guide * add awesome badge * Show 2 ways of importing node-fetch (CommonJS & ES module) * update dependencies * lint * refactor: Replace `url.parse` with `new URL()` (#701) * chore: replace `url.parse` with `new URL()` * lint * handle relative URLs * Change error message * detect whether the url is absolute or not * update tests * drop relative url support * lint * fix tests * typo * Add information about dropped arbitrary URL support in v3.x upgrade guide * set xo linting rule (node/no-deprecated-api) to on * remove the `utf8` dependency * fix * refactor: split tests into several files, create the `utils` directory * Update package.json scripts & remove unnecessary xo linting rules * refactor: turn on some xo linting rules to improve code quality * fix tests * Remove invalid urls * fix merge conflict * update the upgrade guide * test if URLs are encoded as UTF-8 * update xo to 0.28.0 * chore: Build before publishing * v3.0.0-beta.1 * fix lint on test/main.js Co-authored-by: Richie Bendall <richiebendall@gmail.com> Co-authored-by: Antoni Kepinski <xxczaki@pm.me> Co-authored-by: aeb-sia <50743092+aeb-sia@users.noreply.github.com> Co-authored-by: Nazar Mokrynskyi <nazar@mokrynskyi.com> Co-authored-by: Steve Moser <contact@stevemoser.org> Co-authored-by: Erick Calder <e@arix.com> Co-authored-by: Yaacov Rydzinski <yaacovCR@gmail.com> Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com> Co-authored-by: Jimmy Wärting <jimmy@warting.se>
1 task
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #309
This PR replaces all
stream.pipe()withpump.