Use an array for building up mappings in the sourcemap generator.#43785
Use an array for building up mappings in the sourcemap generator.#43785DanielRosenwasser wants to merge 3 commits intomainfrom
Conversation
|
@typescript-bot perf test this faster |
|
Heya @DanielRosenwasser, I've started to run the perf test suite on this PR at 4c86404. You can monitor the build here. |
|
Heya @DanielRosenwasser, I've started to run the abridged perf test suite on this PR at 4c86404. You can monitor the build here. Update: The results are in! |
|
@DanielRosenwasser Here they are:Comparison Report - master..43785
System
Hosts
Scenarios
Developer Information: |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
@typescript-bot perf test this |
|
Heya @DanielRosenwasser, I've started to run the perf test suite on this PR at 312ce16. You can monitor the build here. Update: The results are in! |
|
Heya @DanielRosenwasser, I've started to run the tarball bundle task on this PR at 312ce16. You can monitor the build here. |
|
@DanielRosenwasser Here they are:Comparison Report - master..43785
System
Hosts
Scenarios
Developer Information: |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
@typescript-bot perf test this |
|
Heya @DanielRosenwasser, I've started to run the perf test suite on this PR at 799bbbe. You can monitor the build here. |
|
Heya @DanielRosenwasser, I've started to run the tarball bundle task on this PR at 799bbbe. You can monitor the build here. |
| } | ||
| } | ||
|
|
||
| mappings = mappingSegmentsBuffer.join(""); |
There was a problem hiding this comment.
In local experimentation, I've found that I can get about a 40% speed up by batching the char code conversion:
function base64VLQFormatEncode2(value) {
if (value < 0) {
value = ((-value) << 1) + 1;
}
else {
value = value << 1;
}
// Encode 5 bits at a time starting from least significant bits
const result = [];
do {
let currentDigit = value & 31; // 11111
value = value >> 5;
if (value > 0) {
// There are still more digits to decode, set the msb (6th bit)
currentDigit = currentDigit | 32;
}
result.push(base64FormatEncode(currentDigit));
} while (value > 0);
return result;
}
function buildMappingStr() {
const charCodes = [];
// Mock representation of adding a segment, for benchmarking
for (let i = 0; i < target; i ++) {
charCodes.push(...base64VLQFormatEncode2(i));
}
let str = '';
const segmentLength = 4096;
for (let i = 0, len = charCodes.length; i < len; i += segmentLength) {
const segment = String.fromCharCode.apply(null, charCodes.slice(i, i + segmentLength));
str += segment;
}
return str;
}
There was a problem hiding this comment.
If you are encoding a very large number of mappings (>100000), it gets faster to use an encoder with its own buffer:
class SourceMapEncoder {
constructor() {
this._encoded = '';
this._buffer = new Uint8Array(1024);
this._bufferPos = 0;
}
get encoded() {
if (this._bufferPos > 0) {
this._commit();
}
return this._encoded;
}
/**
* @param {number} code
*/
appendCharCode(code) {
this._buffer[this._bufferPos] = code;
this._bufferPos++;
if (this._bufferPos === this._buffer.length) {
this._commit();
}
}
/**
* @param {number} value
*/
appendBase64VLQ(value) {
if (value < 0) {
value = ((-value) << 1) + 1;
}
else {
value = value << 1;
}
// Encode 5 bits at a time starting from least significant bits
do {
let currentDigit = value & 31; // 11111
value = value >> 5;
if (value > 0) {
// There are still more digits to decode, set the msb (6th bit)
currentDigit = currentDigit | 32;
}
this.appendCharCode(formatArr[currentDigit]);
} while (value > 0);
}
reset() {
this._encoded = '';
this._bufferPos = 0;
}
_commit() {
this._encoded += String.fromCharCode.apply(null, this._buffer.slice(0, this._bufferPos));
this._bufferPos = 0;
}
}
Call .appendBase64VLQ(value) for each mapping to append, then retrieve .encoded at the end.
Still trying to figure out why the overhead is so much higher for small counts
There was a problem hiding this comment.
I actually tried something in the same spirit with a Uint8Array - check out #43987
|
To help with PR housekeeping, I'm going to close this draft PR. It is pretty old now. |
No description provided.