Skip to content

Expose TokenKind to the linter#11237

Closed
dhruvmanila wants to merge 1 commit intomainfrom
dhruv/token-kinds
Closed

Expose TokenKind to the linter#11237
dhruvmanila wants to merge 1 commit intomainfrom
dhruv/token-kinds

Conversation

@dhruvmanila
Copy link
Member

Summary

Test Plan

@dhruvmanila dhruvmanila added internal An internal refactor or improvement parser Related to the parser and removed parser Related to the parser labels May 2, 2024
@dhruvmanila dhruvmanila force-pushed the dhruv/token-kinds branch from 7e344c4 to f6fbdb7 Compare May 2, 2024 05:16
@dhruvmanila dhruvmanila force-pushed the dhruv/token-kinds branch from f6fbdb7 to 6b0d7cf Compare May 2, 2024 05:30
@codspeed-hq
Copy link

codspeed-hq bot commented May 2, 2024

CodSpeed Performance Report

Merging #11237 will improve performances by 4.64%

Comparing dhruv/token-kinds (6b0d7cf) with main (4aac1d1)

Summary

⚡ 1 improvements
✅ 29 untouched benchmarks

Benchmarks breakdown

Benchmark main dhruv/token-kinds Change
linter/default-rules[large/dataset.py] 16 ms 15.3 ms +4.64%

@github-actions
Copy link
Contributor

github-actions bot commented May 2, 2024

ruff-ecosystem results

Linter (stable)

✅ ecosystem check detected no linter changes.

Linter (preview)

✅ ecosystem check detected no linter changes.

Formatter (stable)

✅ ecosystem check detected no format changes.

Formatter (preview)

✅ ecosystem check detected no format changes.

dhruvmanila added a commit that referenced this pull request May 14, 2024
## Summary

Alternative to #11237 

This PR adds a new `Tokens` struct which is a newtype wrapper around a
vector of lexer output. This allows us to add a `kinds` method which
returns an iterator over the corresponding `TokenKind`. This iterator is
implemented as a separate `TokenKindIter` struct to allow using the type
and provide additional methods like `peek` directly on the iterator.

This exposes the linter to access the stream of `TokenKind` instead of
`Tok`.

Edit: I've made the necessary downstream changes and plan to merge the
entire stack at once.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

internal An internal refactor or improvement

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant