Just a note for anyone looking into this: for text grammars, it seems that adding an extra font-style will not play nicely with the current binary format we use for tokens
Such a change needs to begin in the project vscode-textmate. The bit encoding format for tokens is defined in that project at:
|
/** |
|
* Helpers to manage the "collapsed" metadata of an entire StackElement stack. |
|
* The following assumptions have been made: |
|
* - languageId < 256 => needs 8 bits |
|
* - unique color count < 512 => needs 9 bits |
|
* |
|
* The binary format is: |
|
* - ------------------------------------------- |
|
* 3322 2222 2222 1111 1111 1100 0000 0000 |
|
* 1098 7654 3210 9876 5432 1098 7654 3210 |
|
* - ------------------------------------------- |
|
* xxxx xxxx xxxx xxxx xxxx xxxx xxxx xxxx |
|
* bbbb bbbb bfff ffff ffFF FTTT LLLL LLLL |
|
* - ------------------------------------------- |
|
* - L = LanguageId (8 bits) |
|
* - T = StandardTokenType (3 bits) |
|
* - F = FontStyle (3 bits) |
|
* - f = foreground color (9 bits) |
|
* - b = background color (9 bits) |
|
*/ |
|
export const enum MetadataConsts { |
|
LANGUAGEID_MASK = 0b00000000000000000000000011111111, |
|
TOKEN_TYPE_MASK = 0b00000000000000000000011100000000, |
|
FONT_STYLE_MASK = 0b00000000000000000011100000000000, |
|
FOREGROUND_MASK = 0b00000000011111111100000000000000, |
|
BACKGROUND_MASK = 0b11111111100000000000000000000000, |
|
|
|
LANGUAGEID_OFFSET = 0, |
|
TOKEN_TYPE_OFFSET = 8, |
|
FONT_STYLE_OFFSET = 11, |
|
FOREGROUND_OFFSET = 14, |
|
BACKGROUND_OFFSET = 23 |
|
} |
Adding the strikethrough font style would require four bits instead of three
Once that project has a new token bit encoding format, it needs to be published with a new major version (since this is a breaking change) and then adopted in VS Code.
Also, the enums are powers of 2 because they represent bits, so FontStyle.Strikethrough would be 8. But for it to be 8, then all the other information bits need to be shifted by 1.
