Skip to content

Document how StandardTokenType is selected #196725

@DanTup

Description

@DanTup

I'm trying to track down a bug with code completion not automatically triggering in a location where it's expected. The issue turns out to be that the "Standard Token Type" at this location is "String" but we're actually in an interpolated expression (and want "Other"):

image

I checked the equivalent in TypeScript and the standard token type is "Other":

image

Looking through the code, I found this code that appears to regex on the scope name:

const STANDARD_TOKEN_TYPE_REGEXP = /\b(comment|string|regex|regexp)\b/;
export function toStandardTokenType(tokenType: string): StandardTokenType {
const m = tokenType.match(STANDARD_TOKEN_TYPE_REGEXP);
if (!m) {
return StandardTokenType.Other;
}
switch (m[1]) {
case 'comment':
return StandardTokenType.Comment;
case 'string':
return StandardTokenType.String;
case 'regex':
return StandardTokenType.RegEx;
case 'regexp':
return StandardTokenType.RegEx;
}
throw new Error('Unexpected match for standard token type!');
}

So I change the scope to no longer start with string (some of the parent scopes do, but that's also the case for TypeScript), using the same scope name TS did. However, this did not change the standard token type:

image

I can't find any documentation about this, but since it impacts things like code completion I think it should be clear what a language/grammar should do to ensure these things are categorised correctly.

Thanks!

Metadata

Metadata

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions