Skip to content

Expose HTMLFormatter.tokenize/1 as a public function#3778

Closed
aptinio wants to merge 1 commit intophoenixframework:mainfrom
aptinio:expose-html-formatter-tokenize
Closed

Expose HTMLFormatter.tokenize/1 as a public function#3778
aptinio wants to merge 1 commit intophoenixframework:mainfrom
aptinio:expose-html-formatter-tokenize

Conversation

@aptinio
Copy link
Copy Markdown
Contributor

@aptinio aptinio commented Apr 29, 2025

Currently, TailwindFormatter vendors HTMLFormatter, Tokenizer and HTMLEngine.

It would be convenient for TailwindFormatter to use HTMLFormatter.tokenize/1 directly and stay in sync with PhoenixLiveView.

@SteffenDE
Copy link
Copy Markdown
Collaborator

Hey @aptinio,

I'm afraid, but I don't think we can expose this publicly at this point (see also #3231 (comment)). We're working on colocated hooks at the moment, which may require some changes to the tokenizer, so I don't want to make its API public right now and instead give this more thought afterwards.

We could maybe make it @doc false and have the function public, but then you would rely on a private API that could change in a future release, so vendoring is probably still the better approach for now...

@aptinio
Copy link
Copy Markdown
Contributor Author

aptinio commented Apr 29, 2025

@SteffenDE, TailwindFormatter is meant to be used with PhoenixLiveView, so vendoring a particular version of PhoenixLiveView sometimes causes a problem when the user has a different version of PhoenixLiveView.

Making it public, even with @doc false would be appreciated. I closely follow PhoenixLiveView development so relying on a private API wouldn't be a problem.

Currently, [TailwindFormatter vendors HTMLFormatter, Tokenizer and HTMLEngine](100phlecs/tailwind_formatter#35).

It would be convenient for TailwindFormatter to use `HTMLFormatter.tokenize/1` directly and stay in sync with PhoenixLiveView.
@aptinio aptinio force-pushed the expose-html-formatter-tokenize branch from ec3d529 to e972f69 Compare April 29, 2025 08:59
@SteffenDE
Copy link
Copy Markdown
Collaborator

Fine with me. cc @josevalim

@josevalim
Copy link
Copy Markdown
Member

My suggestion would be to add a specific attributes_formatter: %{class: TailwindFormatter} configuration to the formatter and have it call the TailwindFormatter instead. WDYT?

@aptinio
Copy link
Copy Markdown
Contributor Author

aptinio commented Apr 30, 2025

That would be great @josevalim! Currently, the source code is tokenized twice, once by TailwindFormatter, which is only concerned with class attributes, then again by Phoenix.LiveView.HTMLFormatter.

@josevalim
Copy link
Copy Markdown
Member

@aptinio would you like to try a PR?

@aptinio
Copy link
Copy Markdown
Contributor Author

aptinio commented May 1, 2025

Yes. I'll give it a shot. Thanks!

@aptinio
Copy link
Copy Markdown
Contributor Author

aptinio commented May 3, 2025

@josevalim, I created #3781. It includes a test but no docs yet.

@josevalim
Copy link
Copy Markdown
Member

Closing in favor of the PR. :)

@josevalim josevalim closed this May 3, 2025
@aptinio aptinio deleted the expose-html-formatter-tokenize branch May 4, 2025 15:50
aptinio added a commit to aptinio/tailwind_formatter that referenced this pull request Jan 25, 2026
…tter

TailwindFormatter currently vendors Phoenix.LiveView.HTMLFormatter
code so that it can use the private `tokenize/1` function. However,
problems arise when the vendored code and the user's Phoenix.LiveView
version gets out of sync. A PR has been submitted to Phoenix.LiveView
team in the hopes of having this function publicly exposed:

phoenixframework/phoenix_live_view#3778

Instead, a suggestion was made to add an `:attribute_formatters` config
option to Phoenix.LiveView.HTMLFormatter. This has been available since
Phoenix.LiveView v1.1

phoenixframework/phoenix_live_view#3781

This PR removes the vendored code and converts TailwindFormatter from
a mix formatter to an attribute formatter of
Phoenix.LiveView.HTMLFormatter.
aptinio added a commit to aptinio/tailwind_formatter that referenced this pull request Jan 25, 2026
TailwindFormatter currently vendors Phoenix.LiveView.HTMLFormatter
code so that it can use the private `tokenize/1` function. However,
problems arise when the vendored code and the user's LiveView version
gets out of sync. A PR has been submitted to the Phoenix.LiveView team
in the hopes of having this function publicly exposed:

phoenixframework/phoenix_live_view#3778

Instead, a suggestion was made to add an `:attribute_formatters` config
option to Phoenix.LiveView.HTMLFormatter. This has been available since
Phoenix.LiveView v1.1

phoenixframework/phoenix_live_view#3781

This PR removes the vendored code and converts TailwindFormatter from
a mix formatter to an attribute formatter of
Phoenix.LiveView.HTMLFormatter.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants