Add style preprocessing by natemoo-re · Pull Request #69 · withastro/compiler

natemoo-re · 2021-10-19T16:36:58Z

Changes

This exposes a preprocessStyle function to the compiler to preprocess styles.
~~Currently, this function has to be sync because async in Go turns out to be pretty weird. I tried.~~

Here are some handy Go + WASM resources specifically around Promises (that may or may not be useful)

Testing

We don't have automatic WASM tests yet, but we should think about adding them!

Docs

N/A

changeset-bot · 2021-10-19T16:37:00Z

🦋 Changeset detected

Latest commit: c6e6781

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package

Name	Type
@astrojs/compiler	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

internal/transform/preprocess.go

lib/compiler/node/index.ts

lib/compiler/test.js

internal/transform/preprocess.go

natemoo-re · 2021-10-20T03:49:43Z

cmd/astro-wasm/astro-wasm.go

+		// Create and return the Promise object
+		promiseConstructor := js.Global().Get("Promise")
+		return promiseConstructor.New(handler)


Small refactor to return a Promise from transform.

natemoo-re · 2021-10-20T03:50:44Z

cmd/astro-wasm/astro-wasm.go

+				// Hoist styles and scripts to the top-level
+				transform.ExtractScriptsAndStyles(doc)


Main difference here is that we're splitting up the transform step into preprocessing and postprocessing.

natemoo-re · 2021-10-20T03:51:37Z

cmd/astro-wasm/astro-wasm.go

+				// Pre-process styles
+				// Important! These goroutines need to be spawned from this file or they don't work
+				var wg sync.WaitGroup
+				if len(doc.Styles) > 0 {
+					if transformOptions.PreprocessStyle.IsUndefined() != true {
+						for i, style := range doc.Styles {
+							wg.Add(1)
+							i := i
+							go preprocessStyle(i, style, transformOptions, wg.Done)
+						}
+					}
+				}
+				// Wait for all the style goroutines to finish
+				wg.Wait()


This spawns workers that run the async preprocessStyle functions and waits until they are all complete to continue.

Note that this is the preprocessStyle function in Go which has a different signature. It safely invokes the JS preprocessStyle.

natemoo-re · 2021-10-20T03:51:59Z

cmd/astro-wasm/astro-wasm.go

+				// Perform CSS and element scoping as needed
+				transform.Transform(doc, transformOptions)


At this point, our CSS is (presumably) actually CSS and we can scope it!

natemoo-re · 2021-10-20T03:53:05Z

internal/transform/utils.go

+func GetAttrs(n *astro.Node) js.Value {
+	attrs := js.Global().Get("Object").New()
+	for _, attr := range n.Attr {
+		switch attr.Type {
+		case astro.QuotedAttribute:
+			attrs.Set(attr.Key, attr.Val)
+		case astro.EmptyAttribute:
+			attrs.Set(attr.Key, true)
+		}
+	}
+	return attrs
+}


This function returns compile-time known attributes as a JS object, which can then be passed to the preprocessors.

natemoo-re · 2021-10-20T03:54:12Z

internal/wasm_utils/wasm_utils.go

+// See https://stackoverflow.com/questions/68426700/how-to-wait-a-js-async-function-from-golang-wasm
+func Await(awaitable js.Value) ([]js.Value, []js.Value) {
+	then := make(chan []js.Value)
+	defer close(then)
+	thenFunc := js.FuncOf(func(this js.Value, args []js.Value) interface{} {
+		then <- args
+		return nil
+	})
+	defer thenFunc.Release()
+
+	catch := make(chan []js.Value)
+	defer close(catch)
+	catchFunc := js.FuncOf(func(this js.Value, args []js.Value) interface{} {
+		catch <- args
+		return nil
+	})
+	defer catchFunc.Release()
+
+	awaitable.Call("then", thenFunc).Call("catch", catchFunc)
+
+	select {
+	case result := <-then:
+		return result, nil
+	case err := <-catch:
+		return nil, err
+	}
+}


This is some magic I don't fully understand, borrowed from StackOverflow. But it works the same as await in JS!

lol I'm both scared and thrilled!

natemoo-re · 2021-10-20T03:54:41Z

lib/compiler/shared/types.ts

  sourcefile?: string;
  sourcemap?: boolean | 'inline' | 'external' | 'both';
  as?: 'document'|'fragment';
+  preprocessStyle: (content: string, attrs: Record<string, string>) => Promise<string>


That's it, that's the feature!

natemoo-re · 2021-10-20T03:56:29Z

lib/compiler/test.js

+        preprocessStyle: async (value, attrs) => {
+          let x = i++;
+          if (!attrs.lang) {
+            return null;
+          }
+          console.log(`Starting to preprocess style ${x} as ${attrs.lang}`);
+          await sleep(3000);
+          console.log(`Finished preprocessing ${x}`);
+          return value.replace('color', 'background');


Should be very straightforward.

preprocessStyle is an async function that gets the contents of the style element and any compile-time known attrs (we only care about lang now, but maybe others in the future!)

Simulating some expensive work with sleep here.

FredKSchott · 2021-10-20T04:46:53Z

I'll leave for others to review, but if this works and we understand how it works, then this sounds super cool!

cmd/astro-wasm/astro-wasm.go

matthewp · 2021-10-20T12:36:00Z

The CI error makes me worried that this is using something not supported by tinygo but I presume this was tested against using tinygo, right? I which case I don't know what that error is about...

Anyways, clever approach, I hope it works out!

natemoo-re · 2021-10-20T15:34:48Z

.vscode/settings.json

+    "GOOS": "js",
+    "GOARCH": "wasm",


Ayyyy no more VS Code warnings for using WASM

natemoo-re · 2021-10-20T15:36:19Z

cmd/astro-wasm/astro-wasm.go

+func preprocessStyle(i int, style *astro.Node, transformOptions transform.TransformOptions, cb func()) {
+	defer cb()
+	attrs := wasm_utils.GetAttrs(style)
+	data, _ := wasm_utils.Await(transformOptions.PreprocessStyle.(js.Value).Invoke(style.FirstChild.Data, attrs))


This took me a bit to figure out. PreprocessStyle is typed as interface{} which is very similar to TypeScript's any (but a little more strict). The transformOptions.PreprocessStyle.(js.Value) part casts the type as js.Value so we can access methods on that type.

natemoo-re · 2021-10-20T15:37:04Z

internal_wasm/utils/utils.go

@@ -0,0 +1,48 @@
+package wasm_utils


Moved all of the wasm_utils to their own internal_wasm directory because we can't test them directly (easily, at least)

matthewp · 2021-10-20T16:18:37Z

lgtm

natemoo-re commented Oct 19, 2021

View reviewed changes

natemoo-re commented Oct 20, 2021

View reviewed changes

matthewp reviewed Oct 20, 2021

View reviewed changes

cmd/astro-wasm/astro-wasm.go Show resolved Hide resolved

natemoo-re force-pushed the feat/preprocess branch from bfcedb0 to c87e08a Compare October 20, 2021 15:23

natemoo-re commented Oct 20, 2021

View reviewed changes

Nate Moore added 12 commits October 20, 2021 11:48

feat: add style preprocessing

1477c6c

refactor: pass preprocessStyle directly to transform

7d294e7

chore: update settings to target js/wasm

e128eea

chore: move wasm-specific utils to own directory

47ee11f

fix: access (js.Value) methods on PreprocessStyle

2c2d857

chore: add comment about chan

89383e3

chore(ci): fix workflow

16e1c7a

fix: update tests to manually extract scripts and styles

1fa733a

chore: fix lint errors

0eb7c1b

feat: update preprocessStyle to return { code, map }

89ffd2d

chore: add changeset

b26a926

chore: update deno wasm

c6e6781

natemoo-re force-pushed the feat/preprocess branch from 09d5897 to c6e6781 Compare October 20, 2021 16:55

natemoo-re merged commit 4410c5a into main Oct 20, 2021

natemoo-re deleted the feat/preprocess branch October 20, 2021 16:59

		// Hoist styles and scripts to the top-level
		transform.ExtractScriptsAndStyles(doc)

		// Perform CSS and element scoping as needed
		transform.Transform(doc, transformOptions)

Conversation

natemoo-re commented Oct 19, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Testing

Docs

Uh oh!

changeset-bot bot commented Oct 19, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🦋 Changeset detected

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

FredKSchott commented Oct 20, 2021

Uh oh!

Uh oh!

matthewp commented Oct 20, 2021

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

matthewp commented Oct 20, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

natemoo-re commented Oct 19, 2021 •

edited

Loading

changeset-bot bot commented Oct 19, 2021 •

edited

Loading