Skip to content

perf(model): add caching for data keys#271

Merged
uiolee merged 1 commit intohexojs:masterfrom
D-Sketon:perf
Jan 7, 2025
Merged

perf(model): add caching for data keys#271
uiolee merged 1 commit intohexojs:masterfrom
D-Sketon:perf

Conversation

@D-Sketon
Copy link
Member

@D-Sketon D-Sketon commented Dec 29, 2024

check list

  • Add test cases for the changes.
  • Passed the CI test.

Description

I've noticed that ObjectKeys for warehouses/models have a large overhead

{A23BB88A-D214-4BF7-AE8C-C9614AD39BF3}

So I tried to add caching to the dataKeys, and this is what it looks like after I added it:

1x hexo-many-posts
before

Cold processing
┌──────────────────────────────┬───────────────┐
│ (index)                      │ Cost time (s) │
├──────────────────────────────┼───────────────┤
│ Load Plugin/Scripts/Database │ '0.24s'       │
│ Process Source               │ '1.45s'       │
│ Render Posts                 │ '1.37s'       │
│ Render Files                 │ '2.10s'       │
│ Save Database                │ '0.15s'       │
│ Total time                   │ '5.30s'       │
└──────────────────────────────┴───────────────┘
Hot processing
┌──────────────────────────────┬───────────────┐
│ (index)                      │ Cost time (s) │
├──────────────────────────────┼───────────────┤
│ Load Plugin/Scripts/Database │ '0.61s'       │
│ Process Source               │ '0.54s'       │
│ Render Files                 │ '1.76s'       │
│ Save Database                │ '0.15s'       │
│ Total time                   │ '3.06s'       │
└──────────────────────────────┴───────────────┘
Another Cold processing
┌──────────────────────────────┬───────────────┐
│ (index)                      │ Cost time (s) │
├──────────────────────────────┼───────────────┤
│ Load Plugin/Scripts/Database │ '0.24s'       │
│ Process Source               │ '1.51s'       │
│ Render Posts                 │ '1.42s'       │
│ Render Files                 │ '2.05s'       │
│ Save Database                │ '0.14s'       │
│ Total time                   │ '5.36s'       │
└──────────────────────────────┴───────────────┘

after

Cold processing
┌──────────────────────────────┬───────────────┐
│ (index)                      │ Cost time (s) │
├──────────────────────────────┼───────────────┤
│ Load Plugin/Scripts/Database │ '0.26s'       │
│ Process Source               │ '1.21s'       │
│ Render Posts                 │ '1.24s'       │
│ Render Files                 │ '2.06s'       │
│ Save Database                │ '0.17s'       │
│ Total time                   │ '4.94s'       │
└──────────────────────────────┴───────────────┘
Hot processing
┌──────────────────────────────┬───────────────┐
│ (index)                      │ Cost time (s) │
├──────────────────────────────┼───────────────┤
│ Load Plugin/Scripts/Database │ '0.58s'       │
│ Process Source               │ '0.36s'       │
│ Render Files                 │ '1.77s'       │
│ Save Database                │ '0.14s'       │
│ Total time                   │ '2.85s'       │
└──────────────────────────────┴───────────────┘
Another Cold processing
┌──────────────────────────────┬───────────────┐
│ (index)                      │ Cost time (s) │
├──────────────────────────────┼───────────────┤
│ Load Plugin/Scripts/Database │ '0.24s'       │
│ Process Source               │ '1.22s'       │
│ Render Posts                 │ '1.25s'       │
│ Render Files                 │ '2.08s'       │
│ Save Database                │ '0.14s'       │
│ Total time                   │ '4.93s'       │
└──────────────────────────────┴───────────────┘

4x hexo-many-posts
before

Cold processing
┌──────────────────────────────┬───────────────┐
│ (index)                      │ Cost time (s) │
├──────────────────────────────┼───────────────┤
│ Load Plugin/Scripts/Database │ '0.25s'       │
│ Process Source               │ '5.37s'       │
│ Render Posts                 │ '7.99s'       │
│ Render Files                 │ '8.57s'       │
│ Save Database                │ '0.76s'       │
│ Total time                   │ '22.94s'      │
└──────────────────────────────┴───────────────┘
Hot processing
┌──────────────────────────────┬───────────────┐
│ (index)                      │ Cost time (s) │
├──────────────────────────────┼───────────────┤
│ Load Plugin/Scripts/Database │ '1.37s'       │
│ Process Source               │ '4.73s'       │
│ Render Files                 │ '7.08s'       │
│ Save Database                │ '0.57s'       │
│ Total time                   │ '13.75s'      │
└──────────────────────────────┴───────────────┘
Another Cold processing
┌──────────────────────────────┬───────────────┐
│ (index)                      │ Cost time (s) │
├──────────────────────────────┼───────────────┤
│ Load Plugin/Scripts/Database │ '0.25s'       │
│ Process Source               │ '5.87s'       │
│ Render Posts                 │ '7.81s'       │
│ Render Files                 │ '7.34s'       │
│ Save Database                │ '0.61s'       │
│ Total time                   │ '21.88s'      │
└──────────────────────────────┴───────────────┘

after

Cold processing
┌──────────────────────────────┬───────────────┐
│ (index)                      │ Cost time (s) │
├──────────────────────────────┼───────────────┤
│ Load Plugin/Scripts/Database │ '0.25s'       │
│ Process Source               │ '4.50s'       │
│ Render Posts                 │ '5.27s'       │
│ Render Files                 │ '7.49s'       │
│ Save Database                │ '0.72s'       │
│ Total time                   │ '18.24s'      │
└──────────────────────────────┴───────────────┘
Hot processing
┌──────────────────────────────┬───────────────┐
│ (index)                      │ Cost time (s) │
├──────────────────────────────┼───────────────┤
│ Load Plugin/Scripts/Database │ '1.33s'       │
│ Process Source               │ '2.11s'       │
│ Render Files                 │ '6.01s'       │
│ Save Database                │ '0.56s'       │
│ Total time                   │ '10.01s'      │
└──────────────────────────────┴───────────────┘
Another Cold processing
┌──────────────────────────────┬───────────────┐
│ (index)                      │ Cost time (s) │
├──────────────────────────────┼───────────────┤
│ Load Plugin/Scripts/Database │ '0.24s'       │
│ Process Source               │ '4.20s'       │
│ Render Posts                 │ '5.09s'       │
│ Render Files                 │ '6.86s'       │
│ Save Database                │ '0.65s'       │
│ Total time                   │ '17.04s'      │
└──────────────────────────────┴───────────────┘

I'm not sure that such a modification would cause any other problems

Additional information

@coveralls
Copy link

coveralls commented Dec 29, 2024

Pull Request Test Coverage Report for Build 12534648234

Details

  • 27 of 27 (100.0%) changed or added relevant lines in 1 file are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage increased (+0.008%) to 98.283%
Totals Coverage Status
Change from base Build 12162326009: 0.008%
Covered Lines: 4407
Relevant Lines: 4484
💛 - Coveralls

@uiolee uiolee merged commit ca7f851 into hexojs:master Jan 7, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@yoshinorin yoshinorin yoshinorin approved these changes

@uiolee uiolee uiolee approved these changes

@SukkaW SukkaW SukkaW approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@D-Sketon @coveralls @yoshinorin @uiolee @SukkaW