Skip to content

Improve parsing for imports and exports#176

Merged
johno merged 4 commits intomasterfrom
jsx-parsing
Jul 23, 2018
Merged

Improve parsing for imports and exports#176
johno merged 4 commits intomasterfrom
jsx-parsing

Conversation

@johno
Copy link
Member

@johno johno commented Jun 29, 2018

Before we relied 100% on remark's default parser which got us to an MVP but doesn't quite cut it due to the added syntax that imports, exports, and JSX. In many areas that syntax directly conflicts with MD syntax.

For example, consider the following export:

export const frontMatter = {
  heroImg: `${process.env.ASSET_PATH}/space-cats.png`,
  editUrl: 'https://github.com/mdx-js/mdx/edit/master/readme.md',
  authors: [
    'Wilma Flintstone',
    'Fred Flintstone'
  ]
}

The template string, url, and array conflict with Markdown parsing. This expected/intended behavior from an MD perspective, but not from an ES export. This occurs because the paragraph tokenizer then passes the parsed paragraph text on to inline tokenizers in order to parse links, inline code, emphasis, etc.

So we're now introducing our own block parsers that inject themselves before the HTML/paragraph block tokenizers that skip inline tokenization and pass along the raw values for imports/exports. Note that this is still a WIP

Also, I'd like to explicitly thank @wooorm for providing a wonderfully extensible library that allows us to do whacky things 💟.

@ChristopherBiscardi
Copy link
Member

It looks like this is something we'd need to support page queries in gatsby-mdx since right now the graphql tagged template literal is getting mangled as an inline-code element.

@johno johno changed the title [WIP] Improve ES/JSX parsing Improve ES/JSX parsing Jul 23, 2018
@johno johno changed the title Improve ES/JSX parsing Improve parsing for imports and exports Jul 23, 2018
Copy link
Member

@timneutkens timneutkens left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💯

@johno johno merged commit 583b3e5 into master Jul 23, 2018
@johno johno deleted the jsx-parsing branch July 23, 2018 14:54
@ChristopherBiscardi
Copy link
Member

This totally worked for us :D thanks!

johno added a commit that referenced this pull request Jul 25, 2018
* Spike out using custom tokenizer in remark parser

* Make import/export parsing more robust

* Revert unused change in mdxast test

* Move empty newline to utils
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

3 participants