The following post is about creating verifiable generic typeguards in Typescript. I don’t actually recommend using this as it slows compilation times and is a bit of a hack.
Typescript is a language that prides itself on allowing developers to write type safe code. One of the ways that Typescript allows developers to write type safe code is through the use of typeguards. Typeguards are functions that return a boolean value that indicates whether a value is of a certain type. For example, the following function is a typeguard that checks if a value is a string:
function isString(value: unknown): value is string {
return typeof value === 'string';
}
This can then be used to check if a value is a string:
const value: unknown = getSomeUntrustedValue();
if (isString(value)) {
// typescript knows the value is a string so we can access the length property
console.log(value.length);
}
The problem with this approach is that it requires a new typeguard function to be written for each type that needs to be checked, and each written typeguard function can be wrong and will need to be updated every time the type is updated, but won’t warn the developer when this happens. I wanted to create something that would both make writing typeguards easier and would warn the developer when the typeguard is out of date.
Note there are existing alternatives to my approach that are probably better:
Typescript has a powerful type system that allows for pretty complex behavior such as building arithmetic operations. However it is also an erasable type system, meaning you can’t use the type system to do anything at runtime. This means that you can’t use the type system to create typeguards. However, you can use the type system to create verified typeguards.
Below is a simple example of this concept. The function isArrayMatchingTypeguard takes a typeguard function and returns a typeguard function that checks if an array is of the type that the original typeguard function checks for.
export function isArrayMatchingTypeguard<T>(typeguard: (x: unknown) => x is T) {
return (data: unknown): data is T[] => {
if (!Array.isArray(data)) return false
return data.every(typeguard)
}
}
const value: unknown = getSomeUntrustedValue();
if (isArrayMatchingTypeguard(isString)(value)) {
// typescript knows the value is an array of strings so we can access the length property
console.log(value[0].length);
}
The only thing left is to create typeguards for every basic type, and every way of combining types. The basic types are easy:
export function isNull(value: unknown): value is null {
return value === null;
}
export function isUndefined(value: unknown): value is undefined {
return value === undefined;
}
export function isBoolean(value: unknown): value is boolean {
return typeof value === "boolean"
}
export function isNumber(value: unknown): value is number {
return typeof value === "number"
}
export function isString(value: unknown): value is string {
return typeof value === "string"
}
export function isObject(value: unknown): value is { [key: string | number | symbol]: unknown } {
return typeof value === "object" && value !== null && !Array.isArray(value)
}
Although this is technically all the basic types, there is one more case that is important to add:
export function isLiteral<T extends any>(literal: T): (value: unknown) => value is T {
return (value: unknown): value is T => value === literal
}
This allows for checking for literal types, which is important for checking for enums.
The simple case of a map type object({[key: string]: A}) is relatively simple:
export function isObjectWithValues<T>(typeguard: (x: unknown) => x is T) {
return (data: unknown): data is { [key: string]: T } => {
if (!isObject(data)) return false
return Object.values(data).every(typeguard)
}
}
The more common object type you want to cover has set keys however, for example:
type Person = {
name: string,
age: number,
description?: string
favoriteNumbers: number[]
}
It should be clear that this is not a trivial case to cover. You have to handle required and optional properties, have different typeguards for each property. Therefore I will have to show a could of utility types first. First is a way to filter which properties are optional and which are required:
type ForcedOptionalKey<T extends {}> = keyof { [K in keyof T as undefined extends T[K] ? K : never]: K } & keyof T
type ForcedRequiredKey<T extends {}> = keyof { [K in keyof T as undefined extends T[K] ? never : K]: K } & keyof T
This works by mapping the keys of an object to never to remove them iff the value is optional. We check if the value is optional by checking if undefined is a valid value via undefined extends T[K]. I then use the keyof operator to get the keys of the object. Because Typescript can no longer tell that these are keys of T after this operation I then use the & operator to get the intersection with the keys of the object. These are then used in the following utility functions:
type PropertyKey = string | number | symbol;
export function hasOwnProperty<X extends {}, Y extends PropertyKey>(obj: X, prop: Y): obj is X & Record<Y, unknown>;
export function hasOwnProperty<V, X extends { [key: PropertyKey]: V }, Y extends PropertyKey>(obj: X, prop: Y): obj is X & Record<Y, V>;
export function hasOwnProperty<X extends {}, Y extends PropertyKey>(obj: X, prop: Y): obj is X & Record<Y, unknown> {
return obj.hasOwnProperty(prop)
}
export function checkRequiredProperty<T extends {}>() {
return function <X extends {}, Y extends ForcedRequiredKey<T>, TG extends undefined | ((x: unknown) => x is T[Y])>(obj: X, prop: Y, typeGuard?: TG): obj is X & Record<Y, TG extends undefined ? unknown : T[Y]> {
if (!hasOwnProperty(obj, prop)) return false
return typeGuard ? typeGuard(obj[prop]) : true
}
}
export function checkOptionalProperty<T extends {}>() {
return function <X extends {}, Y extends ForcedOptionalKey<T>, TG extends undefined | ((x: unknown) => x is T[Y])>(obj: X, prop: Y, typeGuard?: TG): obj is X & Record<Y, TG extends undefined ? unknown : (T[Y] | undefined)> {
if (!hasOwnProperty(obj, prop)) return true
return typeGuard ? typeGuard(obj[prop]) : true
}
}
hasOwnProperty is just a version of obj.hasOwnProperty that typeguards the object so the key can be used for the function implementation. These functions seem unneccessarily complex as they immediately return another function, but this is used to reduce the amount of generic parameters you need to pass in. Using these you can get the following typeguard:
export function autoTypeguard<T extends {}>(typeGuardRequiredKeys: { [key in ForcedRequiredKey<T>]-?: (d: unknown) => d is T[key] }, typeGuardOptionalKeys: { [key in ForcedOptionalKey<T>]-?: (d: unknown) => d is T[key] }): (data: unknown) => data is CheckNotUnion<T> {
return function (data: unknown): data is CheckNotUnion<T> {
if (data === null || data === undefined || typeof data !== "object" || Array.isArray(data)) return false
for (const key in typeGuardRequiredKeys) {
const requiredKey = key as ForcedRequiredKey<T>
if (!checkRequiredProperty<T>()(data, requiredKey, typeGuardRequiredKeys[requiredKey])) return false
}
for (const key in typeGuardOptionalKeys) {
const optionalKey = key as ForcedOptionalKey<T>
if (!checkOptionalProperty<T>()(data, optionalKey, typeGuardOptionalKeys[optionalKey])) return false
}
return true
}
}
// Using typeguards directly can start to become unreadable so bringing them into their own function can help
const personTypeguard: (data: unknown) => data is Person = autoTypeguard<Person>({
name: isString,
age: isNumber,
favoriteNumbers: isArrayMatchingTypeguard(isNumber)
}, {
description: isString
})
Here you can already start to see how you can chain these typeguards together to create more complex typeguards. You can also notice how this is automatically checked by the compiler. If you were to add a new required property to Person and forget to add it to the typeguard, or if you were to change the type of a property, the compiler would warn you that the typeguard is out of date. You might notice the CheckNotUnion type and it may seem unnecessary, but it is used to prevent the developer from accidentally using a union type in the typeguard. The typeguard would not be able to check for the union type correctly, and the entire point of this is to ensure you don’t accidentally mess up your typeguard. CheckNotUnion is defined as follows:
type CheckNotUnion<T> = TuplifyUnion<T> extends [infer _] ? T : unknown
This is relatively simple, it simply checks if there is only one type in the union, and if there is it returns that type, otherwise it returns unknown. But what is TuplifyUnion?
type TuplifyUnion<T, L = LastOf<T>, N = [T] extends [never] ? true : false> = true extends N ? [] : Push<TuplifyUnion<Exclude<T, L>>, L>
What?
This is where the hacky part starts. It uses type parameter default values as pseudo-variables since the type system doesn’t have typical variable definition. Then it uses a recursive type to take out the last case of the union and push it to a tuple. This type of type transformation is not intended by typescript developers and union cases are supposed to be unordered and therefore their order could technically change at any time, although they haven’t while I’ve been using these functions. I’ll show the types it uses:
// This is a built in type
type Exclude<T, U> = T extends U ? never : T;
type Push<T extends unknown[], V> = [...T, V];
type UnionToIntersection<U> = (U extends unknown ? (k: U) => void : never) extends ((k: infer I) => void) ? I : never
type LastOf<T> = UnionToIntersection<T extends unknown ? () => T : never> extends () => (infer R) ? R : never
What?
Exclude and Push are pretty simple, they are just removing a type from a union and adding a type to a tuple respectively.
UnionToIntersection is a bit more complex. It converts a union type to an intersection by putting it as a function parameter and then inferring that same function parameter. This works because the union gets broken up in the first step(A|B becomes ((k: A) => void) | ((k: B) => void)) which is interpretted specially as a function with multiple type definitions. Then the infer tries to get a single type due to it seeing a single function with different definitions and ends up making an intersection. The extends unknown is used to split up the union when mapping to a function so it doesn’t become ((k: A | B) => void).
Finally LastOf first maps the union to a function and then creates an intersection. The function mapping is important because UnionToIntersection<string | number> is never but UnionToIntersection<() => string | () => number> is () => string & () => number. This doesn’t make sense, but it works ¯_(ツ)_/¯. Then it infers the type parameter of the intersection function and this pulls out the last type.
The simplest way to create a union typeguard is to just split the union into two parts and check each part. This can be done with the following function:
export function eitherType<A, B>(aTypeGuard: (x: unknown) => x is A, bTypeGuard: (x: unknown) => x is B): (x: unknown) => x is A | B {
return (x: unknown): x is A | B => {
return aTypeGuard(x) || bTypeGuard(x)
}
}
This doesn’t work so well for a union with more than two types. For that you need a more complex following function:
type TupleToTypeguards<T extends unknown[]> = { [K in keyof T]: (data: unknown) => data is T[K] }
type NoArrayUnionTypeguards<T> = TupleToTypeguards<TuplifyUnion<T>>
export function unionTypeguard<T>(typeguards: NoArrayUnionTypeguards<T>): (data: unknown) => data is T
export function unionTypeguard<T, U extends (NoArrayUnionTypeguards<T>[number][])>(typeguards: U): (data: NoArrayUnionTypeguards<T>[number][] extends U ? unknown : T) => data is T
export function unionTypeguard<T, U extends (NoArrayUnionTypeguards<T>[number][])>(typeguards: U): (data: NoArrayUnionTypeguards<T>[number][] extends U ? unknown : T) => data is T {
return function (data: unknown): data is T {
for (const typeguard of typeguards) {
if (typeguard(data)) return true
}
return false
}
}
This uses the previously explained type transformation to convert a union to a tuple of typeguards. This is then used to create a typeguard that checks if the data is of any of the types in the union. The multiple function signitures are used to make completions work if you define the typeguards in order, while still allowing you to pass in the typeguards out of order with some amount of checking.
Tuple typeguards are relatively simple at this point. You can just use the following function:
export function isTupleMatchingTypeguards<T extends any[]>(...typeguards: TupleToTypeguards<T>): (x: unknown) => x is T {
return (data: unknown): data is T => {
if (!Array.isArray(data)) return false
if (data.length !== typeguards.length) return false
for (let i = 0; i < typeguards.length; i++) {
const typeguard = typeguards[i]
if (typeguard === undefined) throw new Error("typeguard is undefined")
if (!typeguard(data[i])) return false
}
return true
}
}
This function does not however finish covering all type cases as it misses the case of a tuple with a rest element such as [A, ...B].
export function isSplitTupleMatchingTypeguards<T extends any[]>(startTypeguard: (d: unknown) => d is T[0], restTypeguard: (d: unknown) => d is Tail<T>): (x: unknown) => x is T {
return (data: unknown): data is T => {
if (!Array.isArray(data)) return false
if (data.length === 0) return false
if (!startTypeguard(data[0])) return false
if (!restTypeguard(data.slice(1))) return false
return true
}
}
Another case this type of function can handle is for typeguarding a shared key from a union. Using these can reduce the duplicate code in some cases:
type TupleOmit<T extends unknown[], Key extends string> = { [K in keyof T]: Omit<T[K], Key> }
type BetterOmit<T extends {}, K extends keyof T> = TupleOmit<TuplifyUnion<T>, K & string>[number]
export function partialTypeguard<T extends {}, K extends ForcedRequiredKey<T>>(key: K, partialTypeguard: (data: unknown) => data is T[K], restTypeguard: (data: unknown) => data is BetterOmit<T, K>): (data: unknown) => data is T {
return function (data: unknown): data is T {
if (data === null || typeof data !== "object" || Array.isArray(data)) return false
if (!hasOwnProperty(data, key)) return false
if (!partialTypeguard(data[key])) return false
let datacopy = structuredClone(data)
delete datacopy[key]
return restTypeguard(datacopy)
}
}
export function partialTypeguardOptional<T extends {}, K extends ForcedOptionalKey<T>>(key: K, partialTypeguard: (data: unknown) => data is T[K], restTypeguard: (data: unknown) => data is BetterOmit<T, K>): (data: unknown) => data is T {
return function (data: unknown): data is T {
if (data === null || typeof data !== "object" || Array.isArray(data)) return false
if (!hasOwnProperty(data, key)) return true
if (!partialTypeguard(data[key])) return false
let datacopy = structuredClone(data)
delete datacopy[key]
return restTypeguard(datacopy)
}
}
An example of when these would be useful is when you have a type like this:
type Request = {
type: "GET",
url: string,
token?: string
} | {
type: "POST",
url: string,
token?: string,
body: string
} | {
type: "DELETE",
url: string,
token?: string
}
By using the above functions you can avoid duplicating the url and token typeguards like so:
const longRequestTypeguard: (data: unknown) => data is Request = unionTypeguard<Request>([
autoTypeguard<TuplifyUnion<Request>[0]>({
type: isLiteral("GET"),
url: isString,
}, {
token: isString,
}),
autoTypeguard<TuplifyUnion<Request>[1]>({
type: isLiteral("POST"),
url: isString,
body: isString,
}, {
token: isString,
}),
autoTypeguard<TuplifyUnion<Request>[2]>({
type: isLiteral("DELETE"),
url: isString,
}, {
token: isString,
}),
])
type PartialRequest = BetterOmit<BetterOmit<Request, "url">, "token">
const shortRequestTypeguard: (data: unknown) => data is Request = partialTypeguard("url", isString,
partialTypeguardOptional("token", isString, unionTypeguard([
autoTypeguard<TuplifyUnion<PartialRequest>[0]>({
type: isLiteral("GET"),
}, {}),
autoTypeguard<TuplifyUnion<PartialRequest>[1]>({
type: isLiteral("POST"),
body: isString,
}, {}),
autoTypeguard<TuplifyUnion<PartialRequest>[2]>({
type: isLiteral("DELETE"),
}, {}),
])))
As you can see, the shortRequestTypeguard is much more readable and shorter than the longRequestTypeguard.
This shows it is possible to generate verified typeguards in vanilla typescript. However, not only does it use abuse niche features of the type system that could change, but it also still requires a lot of boilerplate to be written. It has been somewhat useful for me, but I wouldn’t neccessarily recommend it for others. If you want to see a project that uses this, you can check out my Overwatch patch compare tool.
]]>Conventional rating systems such as Elo, Glicko, and Trueskill are fundementally designed for games with low input randomness. By that I mean that they decrease in accuracy for games in which the game starts out differently each time it is played. However, many modern video games using these types of systems have very high input randomness due to their inclusion of factors such as random gamemodes, random maps, and chosen characters for themselves, their teammates or their enemies. Although the impact of these factors can be somewhat mitigated by factors such as a pick/ban system, or allowing swapping characters, these have their own downsides such as lower match variety and counterswapping.
Neural networks have increasingly been shown to be capable of solving almost arbitrarily complex problems given sufficent data. The fundemental idea proposed here is to use a neural network to predict match outcomes based on all data possibly available at matchmaking time, such as player skill and character picks, map, and side that goes first in the case of an asymmetrical gamemode. This predictor can then be used as a heuristic to find well balanced matchups.
I will not go too deep into the design of how the neural network would work because:
However the layers I would propose are:
When using this system you probably want to use character specific mmr instead of a role based or global mmr. The neural network may interpret character mmrs differently, and using character specific mmr should lead to better matchmaking. This also has other potential positive impacts on player experience (see Opening up character design).
One benefit of this system is that via backpropagation the impact of changing inputs on the matchup can be measured. This has an additional cost of approximately double that of the evaluation function but has a number of potential benefits.
One way backpropagation can be used is to measure the impact of changing a player’s input mmr on the expected match result. This can be used as a heuristic for measuring a player’s impact.
It could be used for matchmaking to prevent matches where a player would have a very small impact, as these matches can be frustrating for the player. Similarly it could also be used to prevent matches where a single player has too high of an impact, as these matches can be both frustrating for the high impact player as it becomes carry or lose, as well as frustrating for other players as their impact can only be seen if the high impact player neither carries nor underperforms.
It could also be used to update player mmr after the match. If a player had a small impact on a match it makes sense that that game is less good data for updating mmr compared to a game where a player had a large impact on a match. This should allow players to find their correct mmr faster.
This also has the benefit of being able to correctly separate the mmr of players who play together due to a grouping system. Currently, as far as I know the only system that helps with this is the rating deviation system, which can help adjust stabalized mmr less when they are grouped with players with unstabalized mmr. It does not however account for new players of different skill playing together, which can lead to unbalanced matches if these players ever play without eachother.
Backpropagation can also be used to evaluate the impact of character pick or map on the matchup. This can be used in the case where a guess occurs for a possible match that is not balanced. By reading the gradients of the one-hot vectors, maps or character picks that would lead to a better matchup can be determined. In the case of maps, the matchup can be immediately reevaluated with the more fair map. In the case of character picks, the player can be swapped out for another player of the changed character pick. Alternatively, if the player is allowed to queue for multiple characters at a time, you can swap only their character, which may be nessecary in the case of groups when you can’t just swap a player out.
Some character based games allow the player to swap characters mid game, whereas others don’t. They both have upsides and downsides I’ll list here for reference.
I would argue that neural rating fixes most of the downsides of changing to a no swapping system. In regards to the above:
Another advantage is that because you can determine character picks at matchmaking time, you can prevent putting players who want to play the same character on the same team, thereby allowing players to play the characters they like more often.
One problem of adapting no character swapping neural matchmaking to character swapping games is that the dataset of games available include swapping, and are therefore not ideal data for no swapping matchmaking. You can take data from those chance few games that did not have swaps, but depending on the complexity of the game and number of games available to train on, this data alone may not be enough.
One source of data you may be able to use to enhance your dataset is individual fights. By training on simplified game states such as only including score, time, and ultimate economy data to predict the overall outcome of the game, including only gamestates where there were no swaps for the rest of the game, you can train on valid data, and then evaluate at the game’s start state when matchmaking. As a bonus, you can give players this tool to help identify their mistakes based on significant decreases to their probability of winning after losing a fight.
Another way you can increase your dataset is by altering your current data in ways that should not affect outcome. For example you can change the order players are inputted, swap both the teams and the results, and increase the mmrs of the winning team, as this should not affect the result.
Using character mmrs and neural ratings should decrease player’s variance in performance compared to expected performance, thereby decreasing cases where a player lashes out due to feeling like a game is unwinnable due to a teammate’s underperformance.
One response I’ve gotten when suggesting this idea is that because you’re increasing the burden of quality on the matchmaker, the amount of time it takes a player to find a game must go up. In general I don’t entirely disagree with this sentiment, although I believe that between players queueing for multiple characters to get shorter queue times, and using map picks to compensate for bad matchups the impact should be minimal. I also think that avoiding bad matchups also helps even out other factors that have previously been a problem for matchmaking. For example, normally the impact of high rating disparities causes the result of a match to be unpredictable at matchmaking time, leading to lower match quality. However, if such impacts could be predicted by the matchmaker, you could include much wider rating disparities without a guaranteed decrease in match fairness. Another factor is that a wider variety of fair matches are possible, for example when overwatch implemented role queue queue times went up significantly, but many of the player’s complaints with open queue had to do with the increased impact of matchups, that could potentially be accounted for by a neural rating system.
Although character swapping allows for more unique character design than normal no swapping, it also limits character design as unique characters leading to unfair matchups continue to cause issues through counterswapping and characters being oppressively powerful in certain situations but unplayable in others. Accounting for matchups properly through matchmaking could allow designers to create more unique and fun characters. In addition, effects such as character overall power can be accounted for via the matchmaker, either by putting them in a mirror, against better opponents or against counters. This allows designers to make characters that are unbalanced in either direction without messing up fair matches independent of character pick. With character specific mmr, all that matters is how good you are relative to other players of your character, meaning players don’t have to play meta to climb leaderboards, and variety of play can come from player choice rather than balance.
Having this system also allows you to get better data about the balance of a character. Raw winrates can paint a misleading picture of overall character power as players only use characters in the situations they are powerful. This can either lead to an unwarranted belief of high power, as players who use them in their niches outperform those who don’t. Or the opposite, where if a character is, for example, used to get out of spawn faster to defend in a losing fight, they might have a lower winrate statistic than what they would have if played more universally as they are only played in losing situations. By evaluating the rating system on hypothetical examples, designers can get a better sense of when a character is strong.
In the general sense, changes to a game affect matchmaking quality. If a character is made more powerful, the players of that character will win an unexpected amount of games. Meanwhile other players playing against them will lose an unexpected amnount of games. Still others will swap to playing the character because they are powerful, lose games because they don’t know how to play them, and then start winning as they learn the character’s basics enough to get up to their normal skill. These factors decrease the fairness of matches and therefore enjoyment of players, and are difficult to mitigate with a traditional rating system. With a neural rating system, designers can predict the impact of a change on winrates and attempt to adjust the system to account for this. This should lead to improved early matchmaking, and as the system learns on new data, it will continue to improve, such that players who join late after these changes are not affected by a decrease in match quality.
Note this is optional extension to the idea, and not a critical part of it.
Just because a match is fair, doesn’t mean it is guaranteed to be fun for players. I have previously talked about accounting for impact in matchmaking to mitigate this, but there are more examples. In some matchups characters might have abilities with very little power which often decreases their fun, even if the match is fair.
If you ask players after each match to rate how much fun they had, you could not only get good data on situations where the game isn’t fun for designers, but also train your neural rating system to output a player expected enjoyment value as well as its match prediction result. This can then be used in matchmaking to reduce the prevalence of matchups that aren’t fun. There are some interesting improvements you would likely want to make for this idea, like normalizing the player fun ratings for different characters and match results. Ultimately the impact of this would have to be tested to find problems with it, and whether it would work overall.
Note this is optional extension to the idea, and not a critical part of it.
Character picks and overall skill level are not the only thing that can affect a player’s performance in a game. Other factors include playstyle and situational strengths. For example a player might have very good mechanics, but relativly bad game sense, which would impact their performance in matchups that require different amounts of the different skills. By using a neural rating system, multiple ratings for different skills can be put in to evaluate a matchup, and they can be adjusted differently based on the differing result of backpropagation between them. This could lead to better matchmaking accuracy.
One difficulty with this strategy is getting the neural network to learn these seperate ratings in the first place. To do this you would initialize the training data with either random mmrs, or a random variation on mmr, and apply the gradients of the mmr to the training data. This has its own costs however, as it would likely make it more prone to overfitting, and might make applying old mmr to the new system difficult.
By intentionally losing certain matchups on alt accounts, players could provide invalid training data that would then allow them to come back and win those matchups on their main, thereby abusing the system to gain an unfairly high rating. There are ways to detect this, and it’s unclear why a player would do this as opposed to other cheating methods such as hacking, win trading, or smurfing.
If this system causes designers to come up with more unique characters, and make changes for fun rather than balance, then the pro scene could be incredibly unbalanced and therefore less fun to watch and play in. You could try doing tournaments based on a swiss system, with varying points depending on matchup, but this would still likely be worse.
This does not account for player mental state such as tilted players, tired players, players being toxic or distracting to make their teammates play worse, optimistic players, energized players, or players that give good communication to make their team play better. These will still have a cost to match fairness and are hard to account for.
As this system is more complex than current matchup evaluation systems, it neccessarily comes with an increased compute cost in both training and matchmaking, this may have an affect to increase queue times, or increase matchmaking expenses to make it not worth it economically.
]]>