Skip to content

Improve load times by off loading property insertion to the background thread#657

Merged
KirillOsenkov merged 1 commit intoKirillOsenkov:mainfrom
yuehuang010:dev/yuehuang/main_pl_load
Feb 24, 2023
Merged

Improve load times by off loading property insertion to the background thread#657
KirillOsenkov merged 1 commit intoKirillOsenkov:mainfrom
yuehuang010:dev/yuehuang/main_pl_load

Conversation

@yuehuang010
Copy link
Copy Markdown
Contributor

Improve load times by off loading property insertion to a background thread

  • In my sample binlog of 40mb. I observed up to 2x speed up. Before 15s. After 7s.
  • A 3s is spent in the Analyzing step which remains unchanged. Split the load message this timing messages.
  • Parsing the binlog alone takes 5s. The queue backs up when processing ProjectStarted Event, as it parses the Properties and Items.

@KirillOsenkov KirillOsenkov merged commit 7298a8d into KirillOsenkov:main Feb 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants