Revision 0.31.0 by sinclairzx81 · Pull Request #525 · sinclairzx81/typebox

sinclairzx81 · 2023-08-08T22:12:36Z

0.31.0 Overview

Revision 0.31.0 is a subsequent milestone revision for the TypeBox library and a direct continuation of the work carried out in 0.30.0 to optimize and prepare TypeBox for a 1.0 release candidate. This revision implements a new codec system with Transform types, provides configurable error message generation for i18n support, adds a library wide exception type named TypeBoxError and generalizes the Rest type to enable richer composition. This revision also finalizes optimization work to reduce the TypeBox package size.

This revision contains relatively minor breaking changes due to internal type renaming. A minor semver revision is required.

Transform Types

Revision 0.31.0 includes a new codec system referred to as Transform types. A Transform type is used to augment a regular TypeBox type with Encode and Decode functions. These functions are invoked via the new Encode and Decode functions available on both Value and TypeCompiler modules.

The following shows a Transform type which increments and decrements a number.

import { Value } from '@sinclair/typebox/value'

const T = Type.Transform(Type.Number())             // const T = {
  .Decode(value => value + 1)                       //   type: 'number',
  .Encode(value => value - 1)                       //   [Symbol(TypeBox.Kind)]: 'Number',
                                                    //   [Symbol(TypeBox.Transform)]: { 
                                                    //     Decode: [Function: Decode], 
                                                    //     Encode: [Function: Encode] 
                                                    //   }
                                                    // }

const A = Value.Decode(T, 0)                        // const A: number = 1

const B = Value.Encode(T, 1)                        // const B: number = 0

Encode and Decode

Revision 0.31.0 includes new functions to Decode and Encode values. These functions are written in service to Transform types, but can be used equally well without them. These functions return a typed value that matches the type being transformed. TypeBox will infer decode and encode differently, yielding the correct type as derived from the codec implementation.

The following shows decoding and encoding between number to Date. Note these functions will throw if the value is invalid.

const T = Type.Transform(Type.Number())
  .Decode(value => new Date(value))               // number to Date
  .Encode(value => value.getTime())               // Date to number

// Ok
//
const A = Value.Decode(T, 42)                     // const A = new Date(42)

const B = Value.Encode(T, new Date(42))           // const B = 42

// Error
//
const C = Value.Decode(T, true)                   // Error: Expected number

const D = Value.Encode(T, 'not a date')           // Error: getTime is not a function

The Decode function is extremely fast when decoding regular TypeBox types; and TypeBox will by pass codec execution if the type being decoded contains no interior Transforms (and will only use Check). When using Transforms however, these functions may incur a performance penelty due to codecs operating structurally on values using dynamic techniques (as would be the case for applications manually decoding values). As such the Decode design is built to be general and opt in, but not necessarily high performance.

StaticEncode and StaticDecode

Revision 0.31.0 includes new inference types StaticEncode and StaticDecode. These types can be used to infer the encoded and decoded states of a Transform as well as regular TypeBox types. These types can be used to replace Static for Request and Response inference pipelines.

The following shows an example Route function that uses Transform inference via StaticDecode.

// Route
// 
export type RouteCallback<TRequest extends TSchema, TResponse extends TSchema> = 
  (request: StaticDecode<TRequest>) => StaticDecode<TResponse> // replace Static with StaticDecode

export function Route<TPath extends string, TRequest extends TSchema, TResponse extends TSchema>(
  path: TPath,
  requestType: TRequest,
  responseType: TResponse,
  callback: RouteCallback<TRequest, TResponse>
) {
  // route handling here ...

  const input = null // receive input
  const request = Value.Decode(requestType, input)
  const response = callback(request)
  const output = Value.Encode(responseType, response)
  // send output
}

// Route: Without Transform
//
const Timestamp = Type.Number()

Route('/exampleA', Timestamp, Timestamp, (value) => {
  return value // value observed as number
})

// Route: With Transform
// 
const Timestamp = Type.Transform(Type.Number())
  .Decode(value => new Date(value))
  .Encode(value => value.getTime())

Route('/exampleB', Timestamp, Timestamp, (value) => {
  return value // value observed as Date
})

Rest Types

Revision 0.31.0 updates the Rest type to support variadic tuple extraction from Union, Intersection and Tuple types. Previously the Rest type was limited to Tuple types only, but has been extended to other types to allow uniform remapping without having to extract types from specific schema representations.

The following remaps a Tuple into a Union.

const T = Type.Tuple([                              // const T = {
  Type.String(),                                    //   type: 'array',
  Type.Number()                                     //   items: [ 
])                                                  //     { type: 'string' },
                                                    //     { type: 'number' }
                                                    //   ],
                                                    //   additionalItems: false,
                                                    //   minItems: 2,
                                                    //   maxItems: 2,
                                                    // }

const R = Type.Rest(T)                              // const R = [
                                                    //   { type: 'string' },
                                                    //   { type: 'number' }
                                                    // ]

const U = Type.Union(R)                             // const U = {
                                                    //   anyOf: [
                                                    //     { type: 'string' },
                                                    //     { type: 'number' }
                                                    //   ]
                                                    // }

This type can be used to remap Intersect a Composite

const I = Type.Intersect([                          // const I = { 
  Type.Object({ x: Type.Number() }),                //   allOf: [{
  Type.Object({ y: Type.Number() })                 //     type: 'object',
])                                                  //     required: ['x'],
                                                    //     properties: {
                                                    //       x: { type: 'number' }
                                                    //     }
                                                    //   }, {
                                                    //     type: 'object',
                                                    //     required: ['y'],
                                                    //     properties: {
                                                    //       y: { type: 'number' }
                                                    //     }
                                                    //   }]
                                                    // }

const C = Type.Composite(Type.Rest(I))              // const C = {
                                                    //   type: 'object',
                                                    //   required: ['x', 'y'],
                                                    //   properties: {
                                                    //     'x': { type: 'number' },
                                                    //     'y': { type: 'number' }
                                                    //   }
                                                    // }

Record Key

Revision 0.31.0 updates the inference strategy for Record types and generalizes RecordKey to TSchema. This update aims to help Record types compose better when used with generic functions. The update also removes the overloaded Record factory methods, opting for a full conditional inference path. It also removes the RecordKey type which would type error when used with Record overloads. The return type of Record will be TNever if passing an invalid key. Valid Record key types include TNumber, TString, TInteger, TTemplateLiteral, TLiteralString, TLiteralNumber and TUnion.

// 0.30.0
//
import { RecordKey, TSchema } from '@sinclair/typebox'

function StrictRecord<K extends RecordKey, T extends TSchema>(K: K, T: T) {
  return Type.Record(K, T, { additionalProperties: false })    // Error: RecordKey unresolvable to overload
}
// 0.31.0
//
import { TSchema } from '@sinclair/typebox'

function StrictRecord<K extends TSchema, T extends TSchema>(K: K, T: T) {
  return Type.Record(K, T, { additionalProperties: false })    // Ok: dynamically mapped
}

const A = StrictRecord(Type.String(), Type.Null())             // const A: TRecord<TString, TNull>

const B = StrictRecord(Type.Literal('A'), Type.Null())         // const B: TObject<{ A: TNull }>

const C = StrictRecord(Type.BigInt(), Type.Null())             // const C: TNever

TypeBoxError

Revision 0.31.0 updates all errors thrown by TypeBox to extend the sub type TypeBoxError. This can be used to help narrow down the source of errors in try/catch blocks.

import { Type, TypeBoxError } from '@sinclair/typebox'
import { Value } from '@sinclair/typebox/value'

try {
  const A = Value.Decode(Type.Number(), 'hello')
} catch(error) {
  if(error instanceof TypeBoxError) {
    // typebox threw this error
  }
}

TypeSystemErrorFunction

Revision 0.31.0 adds functionality to remap error messages with the TypeSystemErrorFunction. This function is invoked whenever a validation error is generated in TypeBox. The following is an example of a custom TypeSystemErrorFunction using some of the messages TypeBox generates by default. TypeBox also provides the DefaultErrorFunction which can be used for fallthrough cases.

import { TypeSystemErrorFunction, DefaultErrorFunction } from '@sinclair/typebox/system'

// Example CustomErrorFunction
export function CustomErrorFunction(schema: Types.TSchema, errorType: ValueErrorType) {
  switch (errorType) {
    case ValueErrorType.ArrayContains:
      return 'Expected array to contain at least one matching value'
    case ValueErrorType.ArrayMaxContains:
      return `Expected array to contain no more than ${schema.maxContains} matching values`
    case ValueErrorType.ArrayMinContains:
      return `Expected array to contain at least ${schema.minContains} matching values`
    ...
    default: return DefaultErrorFunction(schema, errorType)
  }
}
// Sets the CustomErrorFunction
TypeSystemErrorFunction.Set(CustomErrorFunction)

It is possible to call .Set() on the TypeSystemErrorFunction module prior to each call to .Errors(). This can be useful for applications that require i18n support in their validation pipelines.

Reduce Package Size

Revision 0.31.0 completes a full sweep of code optimizations and modularization to reduce package bundle size. The following table shows the bundle sizes inclusive of the new 0.31.0 functionality against 0.30.0.

// Revision 0.30.0
//
┌──────────────────────┬────────────┬────────────┬─────────────┐
│       (index)        │  Compiled  │  Minified  │ Compression │
├──────────────────────┼────────────┼────────────┼─────────────┤
│ typebox/compiler     │ '131.4 kb' │ ' 59.4 kb' │  '2.21 x'   │
│ typebox/errors       │ '113.6 kb' │ ' 50.9 kb' │  '2.23 x'   │
│ typebox/system       │ ' 78.5 kb' │ ' 32.5 kb' │  '2.42 x'   │
│ typebox/value        │ '182.8 kb' │ ' 80.0 kb' │  '2.28 x'   │
│ typebox              │ ' 77.4 kb' │ ' 32.0 kb' │  '2.42 x'   │
└──────────────────────┴────────────┴────────────┴─────────────┘

// Revision 0.31.0
//
┌──────────────────────┬────────────┬────────────┬─────────────┐
│       (index)        │  Compiled  │  Minified  │ Compression │
├──────────────────────┼────────────┼────────────┼─────────────┤
│ typebox/compiler     │ '149.5 kb' │ ' 66.1 kb' │  '2.26 x'   │
│ typebox/errors       │ '112.1 kb' │ ' 49.4 kb' │  '2.27 x'   │
│ typebox/system       │ ' 83.2 kb' │ ' 37.1 kb' │  '2.24 x'   │
│ typebox/value        │ '191.1 kb' │ ' 82.7 kb' │  '2.31 x'   │
│ typebox              │ ' 73.0 kb' │ ' 31.9 kb' │  '2.29 x'   │
└──────────────────────┴────────────┴────────────┴─────────────┘

Additional code reductions may not be possible without implicating code maintainability. The typebox module may however be broken down into sub modules in later revisions to further bolster modularity, but is retained as a single file on this revision for historical reasons (not necessarily technical ones).

JsonTypeBuilder and JavaScriptTypeBuilder

Revision 0.31.0 renames the StandardTypeBuilder and ExtendedTypeBuilder to JsonTypeBuilder and JavaScriptTypeBuilder respectively. Applications that extend TypeBox's TypeBuilders will need to update to these names.

// 0.30.0
//
export class ApplicationTypeBuilder extends ExtendedTypeBuilder {}

// 0.31.0
//
export class ApplicationTypeBuilder extends JavaScriptTypeBuilder {}

These builders also update the jsdoc comment to [Json] and [JavaScript] inline with this new naming convention.

TypeSystemPolicy

Revision 0.31.0 moves the TypeSystem.Policy configurations into a new type named TypeSystemPolicy. This change was done to unify internal policy checks used by the Value and Error modules during bundle size optimization; as well as to keep policy configurations contextually separate from the Type and Format API on the TypeSystem module.

// Revision 0.30.0
//
import { TypeSystem } from '@sinclair/typebox/system'

TypeSystem.AllowNaN = true

// Revision 0.31.0
//
import { TypeSystemPolicy } from '@sinclair/typebox/system'

TypeSystemPolicy.AllowNaN = true

TypeSystemPolicy.IsNumberLike(NaN) // true

jtlapp · 2023-08-11T13:40:27Z

Please also strongly consider using JavaScript naming conventions for 1.0, with a possible exception for the schema definition functions. Specifically, non-types/non-classes would start with a lowercase letter. In ten years of using JavaScript, your library is the only one I've used that violates these conventions. While I absolutely love TypeBox and am integrating it throughout my app, I have to say that this is and continues to be seriously irritating.

If you want broader adoption of TypeBox, this may be the most important thing you can do.

I experience a huge amount of friction using TypeBox, as my brain is wired to think of uppercase-first identifiers indicate namespaces and types. One of my objectives creating libraries based on TypeBox is to hide this friction and try to make TypeBox more pleasant to use.

I have several times debated whether I should be using TypeBox because of this friction. I can't be the only one.

Something to consider.

sinclairzx81 · 2023-08-11T16:04:40Z

@jtlapp Hey, thanks for the feedback.

as my brain is wired to think of uppercase-first identifiers indicate namespaces and "types"

It's very unlikely TypeBox would be able to change the Pascal casing used in the library at this point (it's been using this casing for 6 years and I think most developers who use TB seem to be ok with it). It's somewhat of a defining thing about the library at this point, but there are quite a few rationales for opting for Pascal over the more traditional camel casing.

The main reason TypeBox opts for it is that types are typically written with Pascal casing (across all languages). The other main reason is that Pascal provides strong visual distinction between variables representing types, and variables representing values (where in JavaScript, everything is a value). JavaScript doesn't really have a convention for expressing these two concepts through casing rules (on account of it not having types), so TypeBox uses Pascal to draw the distinction out.

So, these are the main reasons TB uses Pascal. As for JavaScript conventions, I lean more towards the view that conventions should be broken where appropriate, and TypeBox is generally a reflection of this view as the library is addressing domains quite distinct to what most JavaScript libraries would addressing....

Thanks again for feedback!
All the best
S

jtlapp · 2023-08-11T17:08:14Z

I did say, "with a possible exception for the schema definition functions," which seems to eliminate your first two points.

I'd be happy with initial-lowercase names for non-type functions:

Value.check()
Value.create()
Value.clone()
Value.check()
Value.cast()
Value.diff()
etc...

I'll wrap any functions that look like types so I'm not dealing with this friction. I bet if you were to poll people, the vast majority would agree with using JavaScript conventions, except for JSON Schema type functions. I'd also bet that your audience is smaller that it could be for creating this friction.

Anyway, I'll not argue further, as it's clear that you aren't open to changing naming conventions.

sinclairzx81 · 2023-08-11T17:12:28Z

@jtlapp Heya, all good.

I'll wrap any functions that look like types so I'm not dealing with this friction. I bet if you were to poll people, the vast majority would agree with using JavaScript conventions, except for JSON Schema type functions. I'd also bet that your audience is smaller that it could be for creating this friction.

Yeah, I have considered possibly using camel casing rules for these, but for the sake of consistency throughout, I've kept them Pascal as well. However it is possible to import each Value API separately as of the last release, so if you are lowercasing them in your code base, you might be able to handle that via import aliases

import { Cast as cast } from '@sinclair/typebox/value/cast'
import { Create as create } from '@sinclair/typebox/value/create'
import { Clone as clone } from '@sinclair/typebox/value/clone'
import { Check as check } from '@sinclair/typebox/value/check'

Hope that helps, I'll give the casing some more thought as things progress. There has been some consideration moving the Value and Compiler API's to separate packages, so doing so might make using a different casing style more feasible, but while they're in the TB codebase, the casing is used for consistency.

Cheers!
S

jtlapp · 2023-08-11T17:23:18Z

That's a bit tedious, but it does help. Thanks for keeping an open mind, after all!

sinclairzx81 mentioned this pull request Aug 8, 2023

Can Date be cast from string/number? #343

Closed

sinclairzx81 force-pushed the revision branch 6 times, most recently from d844c2b to 1e489ec Compare August 11, 2023 11:46

sinclairzx81 mentioned this pull request Aug 11, 2023

Unable to assign a default Date #531

Closed

sinclairzx81 force-pushed the revision branch from 6ed228f to 630b5e7 Compare August 11, 2023 13:38

sinclairzx81 force-pushed the revision branch 6 times, most recently from 30d0e28 to 60f4cee Compare August 11, 2023 21:18

Revision 0.31.0

d304047

sinclairzx81 force-pushed the revision branch from dbea43a to d304047 Compare August 11, 2023 21:26

sinclairzx81 merged commit 18d1cf7 into master Aug 11, 2023

sinclairzx81 deleted the revision branch August 11, 2023 21:31

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Revision 0.31.0#525

Revision 0.31.0#525
sinclairzx81 merged 1 commit intomasterfrom
revision

sinclairzx81 commented Aug 8, 2023 •

edited

Loading

Uh oh!

jtlapp commented Aug 11, 2023 •

edited

Loading

Uh oh!

sinclairzx81 commented Aug 11, 2023 •

edited

Loading

Uh oh!

jtlapp commented Aug 11, 2023

Uh oh!

sinclairzx81 commented Aug 11, 2023 •

edited

Loading

Uh oh!

jtlapp commented Aug 11, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

sinclairzx81 commented Aug 8, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

0.31.0

Overview

Contents

Transform Types

Encode and Decode

StaticEncode and StaticDecode

Rest Types

Record Key

TypeBoxError

TypeSystemErrorFunction

Reduce Package Size

JsonTypeBuilder and JavaScriptTypeBuilder

TypeSystemPolicy

Uh oh!

jtlapp commented Aug 11, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sinclairzx81 commented Aug 11, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jtlapp commented Aug 11, 2023

Uh oh!

sinclairzx81 commented Aug 11, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jtlapp commented Aug 11, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

sinclairzx81 commented Aug 8, 2023 •

edited

Loading

jtlapp commented Aug 11, 2023 •

edited

Loading

sinclairzx81 commented Aug 11, 2023 •

edited

Loading

sinclairzx81 commented Aug 11, 2023 •

edited

Loading