Skip to content

Fix binary file size calculation (readFile without encoding)#94

Merged
iamogbz merged 2 commits into
bundlewatch:masterfrom
cropio:fix/readFile-raw-buffer
Apr 26, 2020
Merged

Fix binary file size calculation (readFile without encoding)#94
iamogbz merged 2 commits into
bundlewatch:masterfrom
cropio:fix/readFile-raw-buffer

Conversation

@oleksii-leonov

@oleksii-leonov oleksii-leonov commented Mar 22, 2020

Copy link
Copy Markdown
Contributor

What kind of change does this PR introduce?

Fix binary file size calculation.

Did you add tests for your changes?

Test snapshots were updated.
Previous snapshots were wrong; you could compare reported with real file size.

If relevant, link to documentation update:

N/A

Summary

We using standard https://github.com/webpack-contrib/compression-webpack-plugin for our builds.

CompressionPlugin generates gzip and brotli compressed files. So, we want to use these files and avoid compression of JS files once again in bundlewatch (via gzip-size or brotli-size).

A simplified version of our config:

{
  "files": [
    {
      "name": "JS vendors chunk",
      "maxSize": "1.0 MB",
      "path": "./packs/js/vendors.chunk.js.br",
      "compression": "none"
    }
  ]
}

But, using "compression": "none" setting we found bundlewatch reports incorrect file size (twice lager). This issue relevant to any binary file.

Example from bundlewatch test data:

test-file-1.jpg file size is 38962 bytes:

__testdata__# ls -la test-file-1.jpg

-rw-r--r--  1 root root  38962 Mar 22 14:42 test-file-1.jpg

Reading this file as fs.readFile(path, 'utf8') leads to incorrect result:

const data = fs.readFileSync('./__testdata__/test-file-1.jpg', 'utf8')
Buffer.byteLength(data)
// 70469

Right way to read as raw buffer fs.readFile(path):

const data = fs.readFileSync('./__testdata__/test-file-1.jpg')
Buffer.byteLength(data)
// 38962

The same issue exists in https://github.com/siddharthkp/bundlesize (siddharthkp/bundlesize#281) and similar fix in their PRs:
siddharthkp/bundlesize#267.

Does this PR introduce a breaking change?

No. But users will receive proper file size (that would differ from past results for binary files).

Other information

`test-file-1.jpg` file size is 38962  bytes:
```
__testdata__# ls -la test-file-1.jpg

-rw-r--r--  1 root root  38962 Mar 22 14:42 test-file-1.jpg
```

Reading this file as `fs.readFile(path, 'utf8')` leads to incorrect result:
```js
const data = fs.readFileSync('./__testdata__/test-file-1.jpg', 'utf8')
Buffer.byteLength(data)
// 70469
```

Right way to read as raw buffer `fs.readFile(path)`:
```js
const data = fs.readFileSync('./__testdata__/test-file-1.jpg')
Buffer.byteLength(data)
// 38962
```

Test snapshots was wrong, you could compare real file size to reported.
@oleksii-leonov oleksii-leonov changed the title Fix file size calculation (readFile without encoding) Fix binary file size calculation (readFile without encoding) Mar 22, 2020
@oleksii-leonov

Copy link
Copy Markdown
Contributor Author

CircleCI passed, however, bundlewatch report can't be sent:

[WARNING] Some CI configuration options were not found (githubAccessToken):
    bundlewatch will be unable to report build status, or save comparison data
    Learn more at: https://bundlewatch.io/
        

Result breakdown at: https://ja2r7.app.goo.gl/WWh9Rtat3aKXhi6f6

PASS ./__testdata__/test-file-1.jpg: 38.05KB < 500KB (no compression)
PASS ./__testdata__/test-file-2.jpg: 248.79KB < 500KB (no compression)
PASS ./lib/app/analyze/analyze.test.mockdata.js: 229B < 2KB (gzip)
PASS ./lib/app/analyze/analyzeFiles/index.js: 938B < 2KB (gzip)
PASS ./lib/app/analyze/index.js: 976B < 2KB (gzip)
PASS ./lib/app/config/ensureValid.js: 924B < 2KB (gzip)
PASS ./lib/app/config/getCIVars.js: 837B < 2KB (gzip)
PASS ./lib/app/config/getConfig.js: 558B < 2KB (gzip)
PASS ./lib/app/errors/ValidationError.js: 175B < 2KB (gzip)
PASS ./lib/app/getLocalFileDetails/getSize.js: 598B < 2KB (gzip)
PASS ./lib/app/getLocalFileDetails/index.js: 599B < 2KB (gzip)
PASS ./lib/app/index.js: 1.35KB < 2KB (gzip)
PASS ./lib/app/reporting/BundleWatchService/index.js: 887B < 2KB (gzip)
PASS ./lib/app/reporting/GitHubService/index.js: 1017B < 2KB (gzip)
PASS ./lib/app/resultsPage/createURL.js: 786B < 2KB (gzip)
PASS ./lib/app/resultsPage/shortenURL.js: 578B < 2KB (gzip)
PASS ./lib/bin/determineConfig.js: 985B < 2KB (gzip)
PASS ./lib/bin/index.js: 1.37KB < 2KB (gzip)
PASS ./lib/logger/index.js: 561B < 2KB (gzip)
PASS ./artifacts/bundlewatch-0.0.0.tgz: 13.26KB < 15KB (gzip)

bundlewatch PASS
Everything is in check (+313.22KB, -0B)

@oleksii-leonov

oleksii-leonov commented Mar 24, 2020

Copy link
Copy Markdown
Contributor Author

Any thoughts on this?

The same issue exists in https://github.com/siddharthkp/bundlesize (siddharthkp/bundlesize#281). PR with fix
siddharthkp/bundlesize#267 created a year ago, still no movement.

I believe this issue quite important. Many projects use webpack CompressionPlugin or similar tools to create compressed assets. Now reported size of any binary assets broken :)

@XhmikosR

XhmikosR commented Apr 8, 2020

Copy link
Copy Markdown
Contributor

@jakebolam this seems like a proper bugfix 🙂

@iamogbz iamogbz merged commit e4fafa6 into bundlewatch:master Apr 26, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants