Skip to content

[Dubbo-Performance] Reduce memory allocation for URL parse#5880

Merged
chickenlj merged 8 commits intoapache:masterfrom
LinShunKang:performance/reduce_object_allocation_v5
Apr 14, 2020
Merged

[Dubbo-Performance] Reduce memory allocation for URL parse#5880
chickenlj merged 8 commits intoapache:masterfrom
LinShunKang:performance/reduce_object_allocation_v5

Conversation

@LinShunKang
Copy link
Copy Markdown
Contributor

@LinShunKang LinShunKang commented Mar 17, 2020 •
edited
Loading

What is the purpose of the change

Reduce memory allocation for URL parse.

Brief changelog

  • Add URLStrParser to parse encodedURLString and decodedURLString.
  • Use URLStrParser#parseEncodedStr(String) to replace URL#valueOf(URL#decode(String)).
  • Use URLStrParser#parseDecodedStr(String) to replace URL#valueOf(String).
  • Optimize URL#toMethodParameters.

Verifying this change

Detail information can see: URLParseOptimize

JMH result:

Benchmark                                Mode  Cnt    Score    Error   Units
URLDecodeBenchmark.testDecodedOptimize  thrpt    5  652.419 ± 14.828  ops/ms
URLDecodeBenchmark.testDecodedOriginal  thrpt    5  517.530 ±  5.144  ops/ms
URLDecodeBenchmark.testEncodedOptimize  thrpt    5  401.168 ±  5.407  ops/ms
URLDecodeBenchmark.testEncodedOriginal  thrpt    5  163.323 ±  0.640  ops/ms

Aprof result:

Benchmark                                Allocated
MeasureURLDecodeGC.testDecodedOptimize   95,171MB
MeasureURLDecodeGC.testDecodedOriginal   273,073MB  
MeasureURLDecodeGC.testEncodedOptimize   95,171MB
MeasureURLDecodeGC.testEncodedOriginal   382,050MB

Follow this checklist to help us incorporate your contribution quickly and easily:

  • [] Make sure there is a GITHUB_issue field for the change (usually before you start working on it). Trivial changes like typos do not require a GITHUB issue. Your pull request should address just this issue, without pulling in other changes - one PR resolves one issue.
  • Format the pull request title like [Dubbo-XXX] Fix UnknownException when host config not exist #XXX. Each commit in the pull request should have a meaningful subject line and body.
  • Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
  • Write necessary unit-test to verify your logic correction, more mock a little better when cross module dependency exist. If the new feature or significant change is committed, please remember to add sample in dubbo samples project.
  • Run mvn clean install -DskipTests=false & mvn clean test-compile failsafe:integration-test to make sure unit-test and integration-test pass.
  • If this contribution is large, please follow the Software Donation Guide.

@LinShunKang
Copy link
Copy Markdown
Contributor Author

LinShunKang commented Mar 17, 2020

@chickenlj @beiwei30 PTAL

@codecov-io
Copy link
Copy Markdown

codecov-io commented Mar 18, 2020

Codecov Report

Merging #5880 into master will decrease coverage by 0.12%.
The diff coverage is 61.87%.

Impacted file tree graph

@@             Coverage Diff              @@
##             master    #5880      +/-   ##
============================================
- Coverage     61.24%   61.12%   -0.13%     
+ Complexity      499      495       -4     
============================================
  Files           981      983       +2     
  Lines         39142    39375     +233     
  Branches       5643     5675      +32     
============================================
+ Hits          23973    24067      +94     
- Misses        12542    12651     +109     
- Partials       2627     2657      +30
Impacted Files Coverage Δ Complexity Δ
...mon/src/main/java/org/apache/dubbo/common/URL.java 55.05% <ø> (+0.07%) 0.00 <0.00> (ø)
.../java/org/apache/dubbo/common/utils/Utf8Utils.java 14.66% <14.66%> (ø) 0.00 <0.00> (?)
...he/dubbo/registry/zookeeper/ZookeeperRegistry.java 57.82% <50.00%> (-0.29%) 0.00 <0.00> (ø)
...ain/java/org/apache/dubbo/common/URLStrParser.java 77.84% <77.84%> (ø) 0.00 <0.00> (?)
...ava/org/apache/dubbo/common/utils/StringUtils.java 76.50% <87.87%> (+0.90%) 0.00 <0.00> (ø)
...apache/dubbo/common/constants/CommonConstants.java 100.00% <100.00%> (ø) 0.00 <0.00> (ø)
...ache/dubbo/remoting/transport/AbstractChannel.java 37.50% <0.00%> (-50.00%) 0.00% <0.00%> (ø%)
...che/dubbo/remoting/transport/mina/MinaChannel.java 39.47% <0.00%> (-11.85%) 15.00% <0.00%> (-2.00%)
.../apache/dubbo/rpc/protocol/AsyncToSyncInvoker.java 62.06% <0.00%> (-10.35%) 0.00% <0.00%> (ø%)
...ubbo/rpc/protocol/dubbo/ChannelWrappedInvoker.java 39.13% <0.00%> (-8.70%) 0.00% <0.00%> (ø%)
... and 39 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 61d1ba7...48b0c9d. Read the comment docs.

@beiwei30
Copy link
Copy Markdown
Member

beiwei30 commented Mar 20, 2020

Great job. @guohao will review this change.

guohao
guohao reviewed Mar 20, 2020
View reviewed changes
Copy link
Copy Markdown
Contributor

@guohao guohao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

主要目标是减少 URL decode 过程中的 char[]/ String / byte[] 内存申请。
代码很工整，TempBuf的思路很好，可以减少 url decode 时的临时 char[] 和 byte[] 不断申请新内存。
基本做到了不申请多余的字符串，思路很清晰。

guohao
guohao approved these changes Mar 21, 2020
View reviewed changes
Copy link
Copy Markdown
Contributor

@guohao guohao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+2

@chickenlj chickenlj merged commit afa2dc9 into apache:master Apr 14, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@chickenlj chickenlj chickenlj requested changes

@guohao guohao guohao approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@LinShunKang @codecov-io @beiwei30 @guohao @chickenlj