Skip to content

History storage and playback#41

Merged
creesch merged 23 commits intomainfrom
3-store-chat-messages-per-server-within-the-minecraft-config-directory-instead-of-using-front-end
Dec 24, 2024
Merged

History storage and playback#41
creesch merged 23 commits intomainfrom
3-store-chat-messages-per-server-within-the-minecraft-config-directory-instead-of-using-front-end

Conversation

@creesch
Copy link
Copy Markdown
Collaborator

@creesch creesch commented Dec 22, 2024

This turns out to be a slightly bigger job than I expected because I wanted to make sure the pre-conditions are right.

@creesch creesch marked this pull request as draft December 22, 2024 12:51
@danthedaniel
Copy link
Copy Markdown
Member

What do you think about adding a schema migration system? Perhaps just initialize the schema version table and write in a value of version 1 for now?

@creesch
Copy link
Copy Markdown
Collaborator Author

creesch commented Dec 22, 2024

What do you think about adding a schema migration system? Perhaps just initialize the schema version table and write in a value of version 1 for now?

I suppose I can add a table for schema versioning.
Though, assuming you mean for the database schema. It is a really simple database with well-defined fields. What we are storing in there isn't anything we control the schema over, in fact we already store the minecraft version next to the json payload which is effectively the schema there.

But it is simple enough to add it. So, why not?

@creesch
Copy link
Copy Markdown
Collaborator Author

creesch commented Dec 22, 2024

Database schema added. For now it throws an error and crashes the entire mod, which is good enough for now. In the future we might want to let it fail gracefully and put a red message in the chat warning about this.

But that is not within the scope of this PR.

@creesch
Copy link
Copy Markdown
Collaborator Author

creesch commented Dec 22, 2024

Alright, history playback is working. Sort of, just hacked it in there when a client freshly connects and the server is connected. But the concept works absolutely fine.

Still need to implement it properly based on a client request. Also need to add the method to fetch messages older than a certain timestamp for use of scrolling up.

@creesch
Copy link
Copy Markdown
Collaborator Author

creesch commented Dec 23, 2024

Alright, moved some stuff around and am no longer using the init event (though it is still in the code) to trigger cleaning of messages.

Now I just need to implement a scroll back mechanism. Websockets don't do request reply traffic without keeping track of correlation IDs I am not too keen about. I am thinking of doing something simpler.

  1. Something triggers a requestHistory function in the client. Either a join event or assuming it has simple "load more" button that was clicked.
  2. If visible the button is hidden.
  3. The client sends a history request and sets a boolean isLoadingHistory to true. Blocking new requests until it is set to false again.
  4. The backend fetches the messages and somehow figures out if there are more. Probably just an extra count query using the oldest message timestamp to determine if there is more to fetch. Alternative do limit +1 see if it returns that number (meaning there is at least 1 more message to fetch) discard the +1 message.
  5. The backend then first sends a historyMetada message back to the client containing the results of that count and the timestamp of the oldest message it retrieved.
  6. The client receives the metadata. If there is more to load it will set isLoadingHistory to false and after a short delay it will show the "load more" button. The last bit to prevent someone spamming that button and causing issues.
  7. Meanwhile, after it has sent the metadata the backend sends the actual messages one by one as it already works at the moment.
  8. The client processes these messages as they arrive.

TODO:

  • historyMetaData type definition
  • The message repository still needs to be updated to support the "before" parameter. Simple change though.
  • Sane implementation of fetching the history metadata
  • The load more button probably should be added to the page dynamically the first time historyMetaData is received. In my mind it also makes sense to store the metadata on there. Though we can probably also just use a global object.

@Tildes-MC Tildes-MC deleted a comment from sonarqubecloud bot Dec 24, 2024
@creesch creesch changed the title WIP: History storage and playback History storage and playback Dec 24, 2024
@creesch creesch marked this pull request as ready for review December 24, 2024 13:39
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry @danthedaniel at some point I had gotten it in such a state that I lost overview. So I started structuring things which ended up being somewhat of a refactoring of the entire thing.

The good news is that it now is much better structured (I think)

@creesch
Copy link
Copy Markdown
Collaborator Author

creesch commented Dec 24, 2024

@danthedaniel, I think it is done. It ended up being a lot more work than I initially bargained for as I wanted to do things right.

I am going to do a bunch more testing, but feel confident enough that I marked the PR as ready.

Edit: If you don't feel like doing a proper review I am okay with that. Just wanted to give you the chance to have a look at it before it is merged.

Copy link
Copy Markdown
Member

@danthedaniel danthedaniel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't have time to do a detailed review. But it seems good from my cursory look.

@creesch creesch merged commit 5782e67 into main Dec 24, 2024
@creesch creesch deleted the 3-store-chat-messages-per-server-within-the-minecraft-config-directory-instead-of-using-front-end branch December 24, 2024 20:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants