Skip to content

web: Replace lingui.js with lit-localize#5761

Merged
kensternberg-authentik merged 20 commits intomainfrom
lit-localization
Jun 2, 2023
Merged

web: Replace lingui.js with lit-localize#5761
kensternberg-authentik merged 20 commits intomainfrom
lit-localization

Conversation

@kensternberg-authentik
Copy link
Copy Markdown
Contributor

web: replace lingui with lit/localize

## Changes

This rather massive shift replaces the lingui and t() syntax with lit-localize, XLIFF, and the msg() syntax used by lit-localize. 90% of this work was mechanized; simple perl scripts found and replaced all uses of t() with the appropriate corresponding syntax for msg() and msg(str()).

The XLIFF files were auto-generated from the PO files. They have not been audited, and they should be checked over by professional translators. The actual strings have not been changed, but as this was a mechanized change there is always the possibility of mis-translation-- not by the translator, but by the script.

web: replace lingui with lit/localize

\#\# Changes

This rather massive shift replaces the lingui and `t()` syntax with lit-localize, XLIFF, and the `msg()`
syntax used by lit-localize.  90% of this work was mechanized; simple perl scripts found and replaced
all uses of `t()` with the appropriate corresponding syntax for `msg()` and `msg(str())`.

The XLIFF files were auto-generated from the PO files.  They have not been audited, and they should be
checked over by professional translators.  The actual _strings_ have not been changed, but as this was
a mechanized change there is always the possibility of mis-translation-- not by the translator, but by
the script.
@netlify
Copy link
Copy Markdown

netlify bot commented May 25, 2023

Deploy Preview for authentik ready!

Name Link
🔨 Latest commit d365954
🔍 Latest deploy log https://app.netlify.com/sites/authentik/deploys/647901e344e6a50008798983
😎 Deploy Preview https://deploy-preview-5761--authentik.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site settings.

TL;DR:

- Replaced all of Lingui's `t()` syntax with `msg()` syntax.
- Mechanically (i.e with a script) converted all of the PO files to XLIFF files
- Refactored the localization code to be a bit smarter:
  - the function `getBestMatchLocale` takes the locale lists and a requested locale, and returns the
    first match of:
    - The locale's code exactly matches the requested locale
    - The locale code exactly matches the prefix of the requested locale (i.e the "en" part of "en-US")
    - the locale code's prefix exactly matches the prefix of the requested locale
    This function is passed to lit-locate's `loadLocale()`.
  - `activateLocale()` just calls `loadLocale()` now.
  - `autodetectLanguage` searches the following, and picks the first that returns a valid locale
    object, before passing it to `loadLocale()`:
    - The User's settings
    - A `?locale=` component found in `window.location.search`
    - The `window.navigator.language` field
    - English

The `msg()` only runs when it's run.  This seems obvious, but it means that you cannot cache
strings at load time; they must be kept inside functions that are re-run so that the `msg()` engine
can look up the strings in the preferred language of the user at that moment.

You can use thunks-of-strings if you really need them that way.
deleted.

\#\# Details

-   Resolves #5171

\#\# Changes

\#\#\# New Features

-   Adds a "Add an Application" to the LibraryView if there are no applications and the user is an administrator.

\#\#\# Breaking Changes

-   Adds breaking change which causes \<issue\>.

\#\# Checklist

-   [ ] Local tests pass (`ak test authentik/`)
-   [ ] The code has been formatted (`make lint-fix`)

If an API change has been made

-   [ ] The API schema has been updated (`make gen-build`)

If changes to the frontend have been made

-   [ ] The code has been formatted (`make web`)
-   [ ] The translation files have been updated (`make i18n-extract`)

If applicable

-   [ ] The documentation has been updated
-   [ ] The documentation has been formatted (`make website`)
* main: (58 commits)
  web: bump @babel/preset-env from 7.22.2 to 7.22.4 in /web (#5808)
  web: bump @typescript-eslint/eslint-plugin from 5.59.7 to 5.59.8 in /web (#5805)
  web: bump @babel/plugin-transform-runtime from 7.22.2 to 7.22.4 in /web (#5806)
  web: bump @typescript-eslint/parser from 5.59.7 to 5.59.8 in /web (#5807)
  stages/deny: fix typos (#5800)
  core: bump coverage from 7.2.6 to 7.2.7 (#5809)
  blueprints: allow setting user's passwords from blueprints (#5797)
  providers/proxy: add support for traefik.io API and CRD (#5801)
  web: fix build (#5798)
  web/flows: fix RedirectStage not detecting absolute URLs correctly (#5781)
  lifecycle: Add depends_on for worker and server container (#5634)
  web: bump @babel/preset-env from 7.21.5 to 7.22.2 in /web (#5791)
  web: bump @babel/plugin-proposal-decorators from 7.21.0 to 7.22.3 in /web (#5787)
  web: bump @lingui/core from 4.1.2 to 4.2.0 in /web (#5782)
  website: bump postcss from 8.4.23 to 8.4.24 in /website (#5783)
  web: bump @babel/plugin-transform-runtime from 7.21.4 to 7.22.2 in /web (#5789)
  core: bump goauthentik.io/api/v3 from 3.2023051.3 to 3.2023052.1 (#5788)
  web: bump @babel/core from 7.21.8 to 7.22.1 in /web (#5790)
  web: bump yaml from 2.3.0 to 2.3.1 in /web (#5792)
  core: bump django-otp from 1.2.0 to 1.2.1 (#5794)
  ...
@github-actions
Copy link
Copy Markdown
Contributor

authentik translations instructions

Thanks for your pull request!

authentik translations are handled using Transifex. Please edit translations over there and they'll be included automatically.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented May 30, 2023

authentik PR Installation instructions

Instructions for docker-compose

Add the following block to your .env file:

AUTHENTIK_IMAGE=ghcr.io/goauthentik/dev-server
AUTHENTIK_TAG=gh-lit-localization-1685653093-9ef71ec
AUTHENTIK_OUTPOSTS__CONTAINER_IMAGE_BASE=ghcr.io/goauthentik/dev-%(type)s:gh-%(build_hash)s

For arm64, use these values:

AUTHENTIK_IMAGE=ghcr.io/goauthentik/dev-server
AUTHENTIK_TAG=gh-lit-localization-1685653093-9ef71ec-arm64
AUTHENTIK_OUTPOSTS__CONTAINER_IMAGE_BASE=ghcr.io/goauthentik/dev-%(type)s:gh-%(build_hash)s

Afterwards, run the upgrade commands from the latest release notes.

Instructions for Kubernetes

Add the following block to your values.yml file:

authentik:
    outposts:
        container_image_base: ghcr.io/goauthentik/dev-%(type)s:gh-%(build_hash)s
image:
    repository: ghcr.io/goauthentik/dev-server
    tag: gh-lit-localization-1685653093-9ef71ec

For arm64, use these values:

authentik:
    outposts:
        container_image_base: ghcr.io/goauthentik/dev-%(type)s:gh-%(build_hash)s
image:
    repository: ghcr.io/goauthentik/dev-server
    tag: gh-lit-localization-1685653093-9ef71ec-arm64

Afterwards, run the upgrade commands from the latest release notes.

@codecov
Copy link
Copy Markdown

codecov bot commented May 30, 2023

Codecov Report

Patch coverage has no change and project coverage change: -0.13 ⚠️

Comparison is base (aca105b) 92.62% compared to head (9ef71ec) 92.49%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #5761      +/-   ##
==========================================
- Coverage   92.62%   92.49%   -0.13%     
==========================================
  Files         549      549              
  Lines       26326    26326              
==========================================
- Hits        24383    24347      -36     
- Misses       1943     1979      +36     
Flag Coverage Δ
e2e 51.13% <ø> (-0.73%) ⬇️
integration 26.48% <ø> (ø)
unit 89.47% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

see 10 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@kensternberg-authentik kensternberg-authentik changed the title Replace Lingui with Lit/Localize web: Replace lingui.js with lit-localize May 31, 2023
Copy link
Copy Markdown
Contributor Author

@kensternberg-authentik kensternberg-authentik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a few markers.

<ak-sidebar-item ?highlight=${true}>
<span slot="label"
>${t`A newer version of the frontend is available.`}</span
>${msg("A newer version of the frontend is available.")}</span
Copy link
Copy Markdown
Contributor Author

@kensternberg-authentik kensternberg-authentik May 31, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

99% of the changes look like this: a mechanical shift from a lingui t() template to the localize msg() template. Localize also wants you to be explicit about your need for in-line variables, so those have msg(str'') syntax. (Those single quotes are supposed to be backticks, but I don't know how to make Markdown show backticks.)

<dd class="pf-c-description-list__description">
<div class="pf-c-description-list__text">
${this.event.tenant?.name || t`-`}
${this.event.tenant?.name || msg("-")}
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does a single hyphen really need translation? I can't find an example in our code of anyone using anything but a hyphen when this source string is referenced.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I don't think we need to bother translating a single hyphen

${locale.code.toUpperCase()} - ${locale.label}
</option>`;
})}
${options}
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a little more surgery than usual because it displays the locales to the customer, so I needed to replace a bit of the logic. I also separated out the business logic (I see I missed one) from the component itself, which is important to me as it leads to independent testing.

Changed the names of the lit-localize commands to make it clear they're
part of the localization effort, and not just "build" and "extract".
* main:
  ci: build outpost binaries statically linked (#5823)
  blueprints: fix API validation with OCI blueprint path (#5822)
  ci: replace github bot account with github app (#5819)
  providers/ldap: fix LDAP Outpost application selection (#5812)
  website: handle go-get requests statically (#5821)
  web: bump @rollup/plugin-node-resolve from 15.0.2 to 15.1.0 in /web (#5816)
  core: bump github.com/stretchr/testify from 1.8.3 to 1.8.4 (#5818)
  web: bump pyright from 1.1.310 to 1.1.311 in /web (#5817)
  website: switch to selfhosted plausible
@kensternberg-authentik kensternberg-authentik marked this pull request as ready for review May 31, 2023 19:27
@kensternberg-authentik kensternberg-authentik requested a review from a team as a code owner May 31, 2023 19:27
BeryJu and others added 10 commits June 1, 2023 12:57
Signed-off-by: Jens Langhammer <jens@goauthentik.io>
Signed-off-by: Jens Langhammer <jens@goauthentik.io>
Signed-off-by: Jens Langhammer <jens@goauthentik.io>
The CI produced a list of problems that I hadn't caught earlier,
due to a typo ("localize build" is correct, "localize compile" is
not) I had left in package.json.  They were minor and linty, but
it was still wise to fix them.
…nto lit-localization

* refs/remotes/origin/lit-localization:
  use build not compile
  fix package lock?
  update transifex config
This commit fixes some minor linting issues that were hidden by a typo in package.json.  The
issues were not apparently problematic from a Javascript point of view, but they pointed
to sloppy thinking in the progression of types through the system, so I cleaned them
up and formalized the types from LocaleModule to AkLocale.
One problem that has repeatedly come up is that localize's templates do not produce
JavaScript that conforms with our shop style.  I've replaced `build-locale` with
a two-step that builds the locale *and* ensures that it conforms to the shop style
via `prettier` every time.
This commit applies the most recent bundle of translations to the
new lit-locale aspect component.  It also revises the algorithm
for *finding* the correct locale, replacing the complex fall-back
with some rather straightforward regular expressions.

In the case of Chinese, the fallback comes at the end of the
selection list, which may not be, er, politically valuable
(since Taiwan and Hong Kong come before, being exceptions that
need to be tested).  If we need a different order for presentation,
that'll be a future feature.
* main:
  web: bump API Client version (#5826)
  release: 2023.5.3
  website/docs: prepare 2023.5.3 (#5824)
  ci: build outpost binaries statically linked (#5823)
  blueprints: fix API validation with OCI blueprint path (#5822)
  ci: replace github bot account with github app (#5819)
  providers/ldap: fix LDAP Outpost application selection (#5812)
  stages/deny: fix typos (#5800)
  web/flows: fix RedirectStage not detecting absolute URLs correctly (#5781)
Well, that was embarassing.
@BeryJu BeryJu added this to the Release 2023.6 milestone Jun 2, 2023
@rissson
Copy link
Copy Markdown
Member

rissson commented Jun 4, 2023

I think this doc page needs an update with this PR

kensternberg-authentik added a commit that referenced this pull request Jun 5, 2023
* main:
  website/docs: Update troubleshooting login.md (#5814)
  web: bump @codemirror/lang-javascript from 6.1.8 to 6.1.9 in /web (#5853)
  web: bump tslib from 2.5.2 to 2.5.3 in /web (#5854)
  web: bump eslint from 8.41.0 to 8.42.0 in /web (#5852)
  core: bump sentry-sdk from 1.24.0 to 1.25.0 (#5856)
  core: bump github.com/sirupsen/logrus from 1.9.2 to 1.9.3 (#5855)
  translate: Updates for web/xliff/en.xlf in zh-Hans (#5841)
  translate: Updates for web/xliff/en.xlf in zh_CN (#5842)
  web: bump @sentry/browser from 7.53.1 to 7.54.0 in /web (#5830)
  web: bump lit from 2.7.4 to 2.7.5 in /web (#5833)
  core: bump cryptography from 39.0.1 to 41.0.0 (#5840)
  web: bump @sentry/tracing from 7.53.1 to 7.54.0 in /web (#5831)
  web: bump typescript from 5.0.4 to 5.1.3 in /web (#5832)
  web: bump mermaid from 10.2.0 to 10.2.2 in /web (#5839)
  web: Replace lingui.js with lit-localize (#5761)
  ci: outpost: use golangci-lint v1.52.2
  outposts/ldap: fix race condition when refreshing the provider
  website/integrations: fix snipe-it typo (#5836)
  core: bump docker from 6.1.2 to 6.1.3 (#5834)
  core: bump twilio from 8.2.1 to 8.2.2 (#5835)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants