Skip to content

feat(translate): migrate to oneshot endpoint#217

Merged
missuo merged 4 commits into
mainfrom
feat/oneshot-endpoint
May 22, 2026
Merged

feat(translate): migrate to oneshot endpoint#217
missuo merged 4 commits into
mainfrom
feat/oneshot-endpoint

Conversation

@missuo

@missuo missuo commented May 22, 2026

Copy link
Copy Markdown
Member

Summary

Revives the free translation path by switching from the legacy www2.deepl.com/jsonrpc (LMT_handle_texts / LMT_handle_jobs) backend — now blocked by per-IP WAF returning HTTP 429 code 1042911 \"Too many requests\" within a handful of calls — to the oneshot endpoint that DeepL's own Chrome extension and iOS app already use for stateless single-shot translation:

POST https://oneshot-free.www.deepl.com/v1/translate
Authorization: None        # literal string — anonymous magic
Content-Type: application/json
{
  \"text\": [\"...\"],
  \"target_lang\": \"de\",     # ITA-style lowercase (de, en-US, zh-Hans)
  \"source_lang\": \"en\"      # optional
}

→ {\"translations\":[{\"detected_source_language\":\"en\",\"text\":\"...\"}]}

Discovered by reverse-engineering DeepL's official Chrome extension cofdbpoegempjloogbagkncekinflcnj — the JO() fetch wrapper unconditionally sets Authorization: \"Bearer \" + token | \"None\", and the body schema comes straight out of the mN() URL builder + bN() body builder.

Why oneshot

www2.deepl.com/jsonrpc (old) oneshot-free.www.deepl.com/v1 (new)
Anti-bot Aggressive — IP 429s within ~10 calls Independent rate-limit pool, holds at 10x burst
Protocol JSON-RPC + body-spacing fingerprint trick Plain JSON POST
Auth (free) none / dl_session cookie for Pro Literal Authorization: None
Status Web frontend migrated off to SignalR/WSS Actively used by extension & iOS app

What changes

  • Replace makeRequestWithBody + TranslateByDeepLX with a oneshot client (translate/translate.go)
  • Add langCodeToOneshot map (DE → de, EN → en-US, ZH → zh-Hans, etc.)
  • Drop the JSON-RPC machinery: PostData / Params / Lang / Job / TextItem / TranslationResponse / LegacyParams / ... — these were only consumed inside this package
  • Drop translate/utils.go (handlerBodyMethod / getICount / getTimeStamp / getRandomNumber / formatPostString) — none of those tricks apply to oneshot
  • Drop the whatlanggo dependency — oneshot does language detection server-side
  • Pin TLS fingerprint via req.ImpersonateChrome() (was SetTLSFingerprintRandomized(), which occasionally landed on profiles the WAF rejects)
  • dlSession parameter semantics change for Pro callers: it is now sent verbatim as a Bearer token to oneshot-pro.www.deepl.com/v1/translate. Existing Pro users using the legacy dl_session cookie will need to switch to an OAuth access token — documented in PR but breaking for that subset

Public API impact

  • TranslateByDeepLX(...) signature unchanged
  • DeepLXTranslationResult shape unchanged, but:
    • Alternatives is always nil (oneshot does not return alternatives)
    • ID is now a time.UnixMilli() value instead of the JSON-RPC request id
  • /translate, /v1/translate, /v2/translate all keep their wire contracts:

Test plan

Manual verification on an IP being actively 429'd by www2.deepl.com:

  • /translate EN→DE: 200 "Der flinke braune Fuchs springt über den faulen Hund."
  • /translate EN→ZH: 200 "开源软件正在改变世界。"
  • /translate ZH→EN with source_lang: \"auto\": 200 (correctly detected ZH)
  • /translate multi-sentence input: 200 (all sentences translated, order preserved)
  • /v1/translate (Pro path) still gated by -s flag (existing behavior)
  • /v2/translate text-array input → official-API-shaped response
  • 10x burst on /translate with no delay: 10/10 HTTP 200 (concurrent www2 calls would 429 by request 3-4)
  • go build ./... clean
  • go mod tidy removes the now-unused whatlanggo dep

Known follow-ups (not in this PR)

  • Pro Bearer flow could optionally auto-exchange a dl_session cookie via https://w.deepl.com/oidc/cookie_bearer_exchange to keep the old -s semantics
  • Optional support for the oneshot formality / glossary_id parameters (extension passes them)
  • Oneshot accepts a text array — exposing batch translation through a new endpoint would be a free wins on top of this

missuo added 4 commits May 22, 2026 11:34
The www2.deepl.com/jsonrpc backends behind LMT_handle_texts /
LMT_handle_jobs now sit behind aggressive WAF + per-IP throttling that
returns HTTP 429 (code 1042911 "Too many requests") within a handful of
calls from any single host — making the free path effectively unusable.

The official DeepL browser extension and iOS app skip that backend
entirely for stateless single-shot translation and POST to a separate
"oneshot" endpoint on a different host pool with its own (much looser)
rate limit. It accepts anonymous traffic with a literal
`Authorization: None` header, returns plain JSON, and supports the same
language pairs.

Switch the free path to:

  POST https://oneshot-free.www.deepl.com/v1/translate
  Authorization: None
  {"text": ["..."], "target_lang": "de", "source_lang": "en"}

Pro users continue to hit oneshot-pro.www.deepl.com with their bearer
token (the `-s` flag now carries an OAuth access token rather than the
legacy dl_session cookie).

This removes:
  - the JSON-RPC envelope (jsonrpc/method/id/params/timestamp wrapper)
  - the `i`-count timestamp trick (getICount + getTimeStamp)
  - the random-id body-spacing trick (handlerBodyMethod)
  - the whatlanggo client-side detection (oneshot detects server-side)

The DeepLXTranslationResult contract is unchanged for service handlers;
Alternatives is now always nil because the oneshot endpoint does not
return alternative translations.

Verified against /translate, /v1/translate and /v2/translate routes
end-to-end (EN/DE/ZH/JA/FR pairs, multi-sentence input, autodetect, 10x
burst) — all 200 OK on an IP that was concurrently being 429'd by www2.
After capturing the exact bytes the Chrome extension's service-worker
fetch() emits (via an offline echo server pointed at deeplx in place of
oneshot-free.www.deepl.com) and diffing them against what we were
sending, several distinguishable signals remained. Close them all.

Headers
-------
- Origin: chrome-extension://cofdbpoegempjloogbagkncekinflcnj
  (was https://www.deepl.com — a request from www.deepl.com itself
   never lands on the oneshot endpoint, so that origin is unusual.
   The extension ID is the canonical sender.)
- Sec-Fetch-Site: cross-site
  (was same-site — wrong; chrome-extension -> www.deepl.com IS cross-site)
- Drop Referer entirely (extension SW fetch sends none)
- Drop Pragma / Cache-Control / Upgrade-Insecure-Requests / Sec-Fetch-User
  (req.ImpersonateChrome() sets these for top-level navigation; a
   fetch() never sends them — leaving them in is a strong nav-vs-XHR tell)
- Accept-Encoding: gzip, deflate, br
  (was just gzip, Go stdlib default — Chrome 120's fetch() sends all
   three; zstd only landed as a default in Chrome 123+ so leave it off)

Body
----
- Add usage_type: "Translate" and the full app_information object
  (os/os_version/app_version/app_build/instance_id) so the JSON the
  server sees is structurally identical to what background.js IN()
  assembles. Field order in oneshotRequest matches the extension's
  object-literal order so encoding/json produces byte-identical output.
- instance_id is a v4 UUID generated once at process start and reused,
  mirroring the extension's chrome.storage-pinned ID rather than
  rotating per-request (rotation would be a far stronger signal).
- All version strings (TLS handshake, User-Agent, sec-ch-ua,
  app_information.os_version) are pinned to Chrome 120 so they tell
  one consistent story.

Transport
---------
- SetBodyBytes instead of bytes.NewReader so Content-Length is set
  (an io.Reader body forces Transfer-Encoding: chunked, which a
   fetch() with JSON.stringify body never emits)
- Once we set Accept-Encoding manually, the Go stdlib disables its
  transparent decompression and req hands us raw compressed bytes.
  Handle gzip / deflate / br by hand from Content-Encoding.
- DisableAutoReadResponse so we own the body stream end-to-end.

The Chrome 120 TLS ClientHello, HTTP/2 SETTINGS frame, pseudo-header
order and sec-ch-ua claim continue to come from ImpersonateChrome()
unchanged.

Verified end-to-end:
- Outbound bytes (against a local echo server) diff-match the
  extension's observed profile on every header and on body JSON order.
- Live oneshot-free.www.deepl.com calls: 4 language pairs OK,
  /v2/translate official-API compat OK, 10x burst 10/10 200.
Run `go get -u ./...` + `go mod tidy`. Direct upgrades:

- github.com/andybalholm/brotli   1.2.0 → 1.2.1
- github.com/tidwall/gjson         1.18.0 → 1.19.0

Indirect (notable):

- github.com/bytedance/sonic       1.15.0 → 1.15.1
- github.com/bytedance/sonic/loader 0.5.0 → 0.5.1
- github.com/bytedance/gopkg       0.1.3 → 0.1.4
- github.com/cloudwego/base64x     0.1.6 → 0.1.7
- github.com/gin-contrib/sse       1.1.0 → 1.1.1
- github.com/go-playground/validator/v10 10.30.1 → 10.30.2
- github.com/goccy/go-json         0.10.5 → 0.10.6
- github.com/klauspost/compress    1.18.4 → 1.18.6
- github.com/mattn/go-isatty       0.0.20 → 0.0.22
- github.com/pelletier/go-toml/v2  2.2.4  → 2.3.1
- golang.org/x/arch                0.24.0 → 0.27.0
- golang.org/x/crypto              0.48.0 → 0.52.0
- golang.org/x/net                 0.51.0 → 0.55.0
- golang.org/x/sys                 0.41.0 → 0.45.0
- golang.org/x/text                0.34.0 → 0.37.0

github.com/imroc/req/v3 (the HTTP client we depend on for Chrome
impersonation) is already on its latest tag v3.57.0 and pins
github.com/quic-go/quic-go to <= v0.57.x — newer quic-go removed
ConnectionTracingID/ConnectionTracingKey, which req's internal/http3
still references. That constraint also holds gin-gonic/gin at v1.11.0
and gin-contrib/cors at v1.7.6 (their later versions pull quic-go
≥ 0.58 transitively). Pin quic-go to v0.57.1 to keep the build green;
revisit when req publishes a release compatible with quic-go ≥ 0.58.

Build + live oneshot end-to-end: 4 language pairs OK, /v2/translate
official-API compat OK, 8x burst 8/8 200.
A real chrome-extension fetch() to oneshot-free.www.deepl.com inherits
whatever cookies the browser has on .deepl.com — at minimum
`userCountry=<iso2>` and `verifiedBot=false`, both of which the
deepl.com server sets on any page load. Our outbound bytes were
otherwise extension-identical but went out cookieless, which is a
distinguishable signal.

Wire a process-wide net/http/cookiejar onto the req.Client and trigger
a single warmup GET to https://www.deepl.com/translator on the first
translate call (sync.Once). The Set-Cookie response (userCountry,
verifiedBot) lands on .deepl.com, which the jar then automatically
echoes back on every subsequent POST to oneshot-free.www.deepl.com
(cookies set on .deepl.com match any *.deepl.com subdomain).

Verified outbound:
  Cookie: userCountry=JP; verifiedBot=false

Latency cost: first call after process start pays one extra HTTP GET
(~1s warmup); subsequent calls are unaffected (sync.Once + connection
keep-alive).

Note: we cannot replicate the _ga / _ga_<id> cookies a real user
would also carry — those are set client-side by GA's JS, which a
non-browser HTTP client can't execute. The userCountry+verifiedBot
pair already matches the "first-time visitor with JS disabled" profile,
which is the closest plausible non-browser approximation.
@missuo missuo merged commit 98918dd into main May 22, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant