Configurable lexer with no dependencies in Rust.
The entire implementation lives in a single .rs file. This means that
you can copy-paste it in your project and modify it as you like,
keeping your dependency tree clean. You can easily add custom
Punctuation or Keywords by modifying the file directly.
Check out the examples in the examples directory.
- Author: Giovanni Santini
- Mail: giovanni.santini@proton.me
- License: MIT
// Some input string
let example_input = String::from(
"#include <stdio.h>
if (a == 17*2 + 35) { // single line comment
/* multi
* line
* comment
*/
return b;
}\n");
// Setup the lexer
let mut lexer = JustLex::default();
lexer.set_input(example_input);
// Iterate over the tokens
for token in lexer {
let token = token;
println!("Received token {:?}", token);
if token == Token::None {
break;
}
}Will output:
Received token Punct(Hash)
Received token Symbol("include")
Received token Punct(Less)
Received token Symbol("stdio")
Received token Punct(Dot)
Received token Symbol("h")
Received token Punct(Greater)
...
Run an example:
cargo run --example basic_usage
Run the tests:
cargo test
Run lint:
cargo clippy