https://github.com/Zogalicious/DecisionMultiverse https://www.figma.com/proto/mW7fREoJwINmzstbwSXRB3/Prototyping-in-Figma?node-id=0-1&t=QFcZFsRZHC2T6wog-1

DecisionMultiverse

Change a video game character's past actions in order to see alternate futures.

Overview

Problem: Games like Disco Elysium or Hades have hugely complex dialogue systems, where thousands of lines and hours of speech need to be written. For main characters, this makes sense. But if you just want to test out dialogue pathways of a main character, or want to create many different NPCs, this quickly becomes impossible, especially for smaller game studios. This relates to GreatUniHack 2024’s main theme of time travel, because it is all about changing a character’s past actions in order to see alternate futures.

Solution: Using AI to fill out these dialogue trees is a good fit. It is infeasible to manually write thousands of lines of dialogue for thousands of characters, so this could be useful, say, in an open-world game. By using agentic AI, characters can actually act out actions in this tree, rather than just saying stuff. And, since any event in the tree can be modified, characters can instantly react to changing user actions (future project: treating actions of player as extra context to inform decision tree).

App workflow

Character contextual information
- Questionnaire upon first registration, to gain basic info about the NPC which contextualises tree generation. (User can also add more context in the future.)
Tree creation
- User creates a tree by adding nodes. Each node represents an event, where higher-up nodes are further in the past, and lower-down nodes are further in the future.
- They can draw links between parent and child nodes: a link specifies that the parent node has contributed towards causing the child node. If multiple parent nodes have caused a child node, the links can be specified with different probabilities to show relative contributions (eg 0.6, 0.4 for 2 parent nodes causing a child node).
- AI model then builds out the tree. It could do so infinitely, so user needs to specify how many levels down it should go (due to limited compute power).
Tree modification
- User can change any node in the tree.
- When user changes a node in the tree, all nodes underneath it update, to show what could have happened if that event had been different.
- For each updated node, a list of events with different probabilities is calculated. By default, the one with the highest probability is selected from the list. But if user clicks on a node, this list is shown, and they can select a different event in it.
- If user selects a different event, all the events underneath it are updated.
- Once again, the user can change any node in the tree, and, when changed, select between a list of different events per node.

Technical details

We use JSON to store details of each event, and their links, in the file event_tree.json. We also store a text file with information about the character. We read and then pass this information to the AI model in the prompt. The model we're using is GPT 4o-mini.

Each event has the format { "position": [x,y] "parents": [[parent1Position,parent1Contribution],[parent2Position,parent2Contribution],[parent3Position,parent3Contribution]] "content": "Event content" } where "position" gives the event's x and y coordinates on the tree, where "parents" gives the event's parents' positions, along with the contribution each parent has towards causing the event, where "content" is a text description of the event's content.

Each prompt consists of a query, event_tree.json, and character_info.txt, separated by 3 double-quote marks (""") as a delimiter.

The model should output the updated event_tree.json file.

All queries are either to delete events from the event_tree.json file, calculate new events based on the current ones in the event_tree.json file and add these new events to the file, input new events to the event_tree.json file, or modify existing events in the event_tree.json file. Queries to delete events (delete queries) will start with the text "delete: ", queries to calculate new events (calculate queries) will start with the text "calculate: ", queries to input new events (input queries) will start with the text "input: ", and queries to modify existing events (modify queries) will start with the text "modify: ".

A delete query gives the position of the event to delete. For example, the query "delete: [1,1]" would delete the event at position [1,1]. Then, the model may need to update other event positions. For example, if events [3,1] and [2,1] exist, and [1,1] is deleted, [3,1] will have to become [2,1] and [2,1] will have to become [1,1].

A calculate query only gives a number, which is the number of levels (below the current lowest event) the AI model should continue the current tree for. Then, the model will add the events to the event_tree.json file. For example, if the current lowest event has depth 4, a query of "calculate: 5" will cause the model to calculate events all the way down to level 9. For each new event, the model can easily fill out "position", but it should be creative with "content", even if this strays somewhat from the sort of event contents that are already in the event_tree.json file. The model will also use character_info.json to inform "content" and "position" for each new event. To fill out "parents", it should use reasoning similar to how the current event_tree.json file does this.

An input query specifies an event in the same way as the event_tree.json file, with the exception that instead of "position", "y" will be specified, and the model will have to calculate the required x value. To calculate this x value, it will need to find the highest x value at the specified y position, and increment it by 1. For example, if the highest x value at the specified y position is 3, the x value of the new event is 4, and so the position is [4,3]. Then, the model will add the event to the event_tree.json file.

A modify query specifies an event in the same way as the event_tree.json file. However, the event position the query gives will be the same as an existing event's position in the event_tree.json file. The model will delete the existing event (and update other event positions, as specified in the "delete query" section). Then it will calculate new events, in the same way as specified in the "calculate query" section), with the exception that it will only calculate new child events of the modified event, and then recursively calculate child events of the child events, all the way down to the current deepest level in event_tree.json. It will recursively delete the current child events underneath the modified event, and then add these new events.

Along with each query, the event_tree.json and character_info.txt files will be passed in the prompt to the model. 3 double-quote marks (""") will separate the query, event_tree.json, and character_info.txt.

For example, a valid prompt would be 'input: { "position": [0,0] "parents": [] "content": "Character bought 4 apples from the supermarket" } """ { "position": [1,0] "parents": [] "content": "Apples cost $1.50 each" }, { "position": [2,0] "parents": [] "content": "Character has $10" } """ Character is a university student living in Manchester. Character is attending GreatUniHack 2024 and has enjoyed it so far, because they like building projects.'

The valid output from this prompt would be '{ "position": [0,0] "parents": [] "content": "Character bought 4 apples from the supermarket" }, { "position": [1,0] "parents": [] "content": "Apples cost $1.50 each" }, { "position": [2,0] "parents": [] "content": "Character has $10" }'

Building/running app

Need to use a v20 version of Node like v20.18.0 because OpenAI npm module relies on punycode, which is deprecated in later Node versions. To do so, you can install nvm and then use 'nvm use' in the project directory, which will use the version specified in .nvmrc (v20.18.0).