Skip to content

Migrate Babel from Flow to TypeScript (except Babel parser)#11578

Merged
nicolo-ribaudo merged 69 commits intobabel:mainfrom
zxbodya:ts
Nov 25, 2021
Merged

Migrate Babel from Flow to TypeScript (except Babel parser)#11578
nicolo-ribaudo merged 69 commits intobabel:mainfrom
zxbodya:ts

Conversation

@zxbodya
Copy link
Copy Markdown
Contributor

@zxbodya zxbodya commented May 18, 2020

Edit by @nicolo-ribaudo
When migrating @babel/core we shouldn't forget about

"TODO": "The @babel/traverse dependency is only needed for the NodePath TS type. After converting @babel/core to TS we can import NodePath from there.",
"dependencies": {
"@babel/traverse": "workspace:^7.12.17",

Still work in progress, but most of the migration already done:

  1. updated lint/build configuration
  2. migrated everything to typescript using flowts(see commits with messages starting with flowts:)
  3. configured typescript project references so it is nice and fast
  4. did some manual fixes:
    • updated generated code to be in TypeScript
    • did manual fixes where types happened to be incorrect
    • backported some types/comments from DefinitelyTyped

Currently, the whole project passes the type check, also existing tests should be working.

Commits are grouped by package and sorted in topological order (before fixing some package - all of its dependencies should be already fixed)

I tried to avoid unnecessary changes, and so - for example, did not use strict mode anywhere(I think it would be good to add in some places, but - later).

Remaining things, I am planing before marking this as ready for review:

  1. to merge some dependency updates and related improvements (separate PRs)
  2. to review the changes I did and to update the comments (some comments might be no longer relevant, and some fixes might be not needed anymore)

Some background:

I working on a tool automating the Flow to TypeScript migration, which is already in a quite working state.

It is built using Babel, and partially because of that - I want to improve TypeScript typings for Babel.

I think, the best way to so is to convert the whole project to TypeScript, which I believe also has other benefits:

  • all the benefits of TypeScript in comparison to Flow (much bigger community, better IDE support, etc…)
  • finding and fixing bugs…
  • improving type coverage - currently, it is very sparsely typed, and I believe a bit more types would be helpful

Related meeting notes: https://github.com/babel/notes/blob/master/2020/05-07.md#investigate-using-typescript-in-our-code


Migration summary:

The most complicated/problematic was babel/parser - there are a lot of issues in flow types that were highlighted after the migration(basically, as often happens with flow… - some issues were ignored) in particular, there are a lot of issues in node type definitions:

  • some of the types there were impossible(types intersections that should result in empty property types, where it clearly should not be)
  • some node types were missing
  • also, it is out of sync with babel/types (which I think it should use instead of redeclaring all the node types)

weird changes (EDIT: we don't import package.json anymore)

There is a tricky problem about importing ../package.json in TypeScript.

Generally TypeScript can support importing json files, but it also wants all imported files to be included as sources.

So if to include only src - ../package.json would be allowed.
If to include package.json into sources - when building, structure of all the sources (not only src as needed) would be replicated.

To solve this, I did a bit weird trick:

Where needed - I added src/package.js, with following content:

import { version } from "../package.json";
export { version };

Which is not type-checked, and can be imported instead of package.json.

Suggestions for better options are welcome.

common fixes:

  • remove unnecessary typecasts(most common as Array<any>)

  • add missing optional argument annotations (most often boolean arguments which are optional, are not marked as such)

  • add as const in some places to get more specific types (typically for "enum-like" constants)

    • Object.freeze({}) -> Object.freeze({} as const)
    • {…} -> {…} as const
  • remove some anys(btw, in flow Function and Object - are aliases to any), in many cases typescript can infer better type

  • remove some typecasts to unknown (mixed in flow) or replace it with any where appropriate

  • where needed replace $FlowIgnore with @ts-ignore

  • fixing usages of not existing types (some types used in flow code not were imported, or imported from somewhere they do not exist)

  • replace void with undefinded in some places (in ts void mean no value and should be used only as return type)

  • fixing optional argument annotations (in typescript undefned type, does not mean that argument is optional, and so it would require argument which however might be undefined)

  • change type annotation for some functions to be type guards, for example:

    - function isThenable(val: unknown): boolean {
    + function isThenable<T extends PromiseLike<any>>(val: unknown): val is T {
  • in some cases - added type refinement, before the condition, for example: if (t.isStringLiteral(node) && node.value === "use strict") instead of just node.value === "use strict" (which technically will work, but would make TS not happy)

  • add missing type parameter(most common Array to Array<any>, but also the same for Set, Map and WeekMap types)

  • fix incorrect DOM type usages:

    • replace Location with t.SourceLocation
    • replace Node with t.Node
  • add missing fields in some types(for example recordAndTupleSyntaxType, jsescOption and jsonCompatibleStrings in Format in babel-generator)

  • declared instance fields in class declarations (often they were assigned somewhere in class methods or constructor, but not declared)

  • replace some assertions like path.type === "JSXFragment" with appropriate utility calls path.isJSXFragment()

  • remove unused call arguments or properties on options object where in some places, where function does not have that argument or does not use the property

  • @ts-expect-error when doing something not type-safe, like assigning .type on AST node

  • @ts-expect-error when getting tagExpr.value from Literal (because it is not present on NullLiteral)

  • import and use types from babel-types and babel-travese in some plugins, helpers (not everywhere, I would suggest this can be one of the next steps)

  • replace template -> template.statement where appropriate to have simpler type-casts

  • use !!value in some places where value should be converted to boolean

  • additional type annotations in some places(tried to minimize this, also to consider as one of the next steps after landing this)

  • typecasts like path.get("tag.object") as NodePath where correct type cannot be inferred

  • type declaration for visitor state in some helpers/plugins (as possible next step - to do more of this)

  • use Visitor type from babel-traverse for visitor declaration in plugins (allows improving type coverage for individual visitors without manually typing each property in declaration)

  • in some places, t.Node validators were used on NodePath(which evetually worked, but is not correct in general), example fix: t.isImportDeclaration(stmt) -> stmt.isImportDeclaration()

notable changes:

  • updated build config to support TypeScript instead of Flow
    • use babel preset typescript instead of flow plugin
    • config updates to allow using .ts files
    • update code generating scripts to generate typescript code
  • babel-types
    • add more generated types to src (adopting existing logic from scripts/typescript)
    • update generateBuilders to specify exact argument types
    • improve deprecated types generation
    • backport types from generate index.d.ts to be included in sources
    • avoid export of builders starting with upper case in ts sources (generate separate js file for deprecated exports, to avoid exporting types for them)
  • babel-parser
    • use abstact class (instead of hack with flow type comments)

    • refactor parser plugins to be typed

    • specify exact argument list in methods overridden in parser plugins, and in calls to original implementation - to have exactly the same method signature(which is now type-checked). for example:

      - return super.parseMaybeAssign(...args);
      + return super.parseMaybeAssign(
      +  noIn,
      +  refExpressionErrors,
      +  afterLeftParse,
      +  refNeedsArrowPos,
      + );
    • refactoring in node type definitions

      • use interfaces instead of type aliases for node type definitions
      • extend node types (instead of & in flow types, which often looked not correct)
    • babel-parser move included type definitions to source code

    • add/fix some node types

  • babel-generator
    • babel-generator type this for generators assigned to Printer.prototype
  • babel-traverse
    • this type annotation for mixins in NodePath, export types for fields added by mixins
    • validators, asserts and virtual types code generation scripts
  • babel-core
    • remove not existing type imports, like replacing type SourceMap = typeof import("convert-source-map").SourceMap; with any in packages/babel-core/src/transformation/file/generate.ts
    • add types for some dependencies
  • babel-helper-plugin-utils
    • generic type for declare utility

Going to periodically rebase this on current master, while finishing work on.

However, there is not likely to be a big change here, so comments and suggestions are welcome.


Q                       A
Fixed Issues?
Patch: Bug Fix?
Major: Breaking Change?
Minor: New Feature?
Tests Added + Pass? Yes (almost, to be updated)
Documentation PR Link
Any Dependency Changes?
License MIT

@babel-bot
Copy link
Copy Markdown
Collaborator

babel-bot commented May 18, 2020

Build successful! You can test your changes in the REPL here: https://babeljs.io/repl/build/49954/

@codesandbox-ci
Copy link
Copy Markdown

codesandbox-ci bot commented May 18, 2020

This pull request is automatically built and testable in CodeSandbox.

To see build info of the built libraries, click here or the icon next to each commit SHA.

Latest deployment of this branch, based on commit bc07a4d:

Sandbox Source
babel-repl-custom-plugin Configuration
babel-plugin-multi-config Configuration

@zxbodya zxbodya changed the base branch from master to main July 2, 2020 21:23
@nicolo-ribaudo nicolo-ribaudo added the Flow -> TS Tracking repository migration from Flow to TS label Nov 6, 2020
@zxbodya zxbodya force-pushed the ts branch 2 times, most recently from 8d7f7be to af41cdc Compare November 10, 2020 21:53
@zxbodya zxbodya force-pushed the ts branch 3 times, most recently from 9e2a988 to 36da578 Compare November 22, 2020 19:13
Object.assign(exports, node);

// make TypeScript happy, by making this file a "module"
export default register;
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an observable change, because it changes the source type (and thus the compiled output) from "script" to "module".
I'll revert this file to .js; it will go away in Babel 8 anyway.

Copy link
Copy Markdown
Member

@nicolo-ribaudo nicolo-ribaudo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry that this took so long! There are still many places where we need to improve our types, bug this is a good step.

@nicolo-ribaudo nicolo-ribaudo merged commit 0058b7f into babel:main Nov 25, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

Flow -> TS Tracking repository migration from Flow to TS outdated A closed issue/PR that is archived due to age. Recommended to make a new issue

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants