Previously, there was two lexer, 1) frontend and 2) emmy. The frontend parser was responsible for parse the file, take out the nodes that makes sense and convert it into emmy format as a string. In next step that emmy format is given to emmy parser which finally converts that to actual tokens. The main downside is that there are two steps to tokenize the file.

Now, that frontend parser is removed. The file is now tokenized in one step making it 1.5x faster than before.