Skip to content
This repository was archived by the owner on Dec 22, 2021. It is now read-only.

perf: Increase write performance #25

Merged
James-Frowen merged 3 commits into from
Nov 17, 2020
Merged

perf: Increase write performance #25

James-Frowen merged 3 commits into from
Nov 17, 2020

Conversation

@James-Frowen
Copy link
Contributor

@James-Frowen James-Frowen commented Nov 12, 2020

@James-Frowen James-Frowen mentioned this pull request Nov 16, 2020
Closed
@James-Frowen James-Frowen changed the title perf: send messages immediately when using sslstream perf: Increase write performance Nov 16, 2020
@James-Frowen
Copy link
Contributor Author

James-Frowen commented Nov 16, 2020

Results with 100 NetworkTransforms to 4 clients on localhost

SendLoop n is the thread number. some runs dont start at 1, this is from how the test was run but shouldn't effect the results

times is time spend in the Write function
bytes is how many bytes were sent via the write function

ssl using sslstream
batch message were grouped before sent via stream.write
sleep Thread.Sleep(1) after send before writer (to give time for multiple sends to be added to queue before write)
flush stream.flush called after all writes

--------------------
[ssl:False  batch:False  flush:False  sleep:False  ]
--------------------
SendLoop 1           Calls: 7821 times[avg:   0.024 max:   4.397 total: 190.812] bytes[avg:      57 max:     135 total:  453118]
SendLoop 2           Calls: 7821 times[avg:   0.024 max:   4.437 total: 184.024] bytes[avg:      57 max:     135 total:  453118]
SendLoop 3           Calls: 7821 times[avg:   0.027 max:   6.208 total: 212.157] bytes[avg:      57 max:     135 total:  453118]
SendLoop 4           Calls: 7821 times[avg:   0.022 max:   5.411 total: 172.442] bytes[avg:      57 max:     135 total:  453118]

--------------------
[ssl:False  batch:True   flush:False  sleep:False  ]
--------------------
SendLoop 5           Calls: 1039 times[avg:   0.042 max:   0.538 total:  43.561] bytes[avg:     429 max:    4959 total:  446449]
SendLoop 6           Calls:  950 times[avg:   0.049 max:   2.261 total:  46.145] bytes[avg:     469 max:    4959 total:  446449]
SendLoop 7           Calls:  927 times[avg:   0.042 max:   1.987 total:  38.554] bytes[avg:     481 max:    5850 total:  446449]
SendLoop 8           Calls: 1057 times[avg:   0.044 max:   0.598 total:  46.349] bytes[avg:     422 max:    5073 total:  446449]

--------------------
[ssl:False  batch:True   flush:False  sleep:True   ]
--------------------
SendLoop 1           Calls:  233 times[avg:   0.118 max:   0.857 total:  27.490] bytes[avg:    1940 max:   13413 total:  452186]
SendLoop 3           Calls:  233 times[avg:   0.136 max:   0.875 total:  31.716] bytes[avg:    1884 max:    5415 total:  439043]
SendLoop 4           Calls:  233 times[avg:   0.120 max:   0.988 total:  27.980] bytes[avg:    1940 max:   13568 total:  452206]
SendLoop 5           Calls:  232 times[avg:   0.148 max:   1.217 total:  34.339] bytes[avg:    1949 max:   13568 total:  452206]

--------------------
[ssl:False  batch:False  flush:True   sleep:False  ]
--------------------
SendLoop 5           Calls: 8144 times[avg:   0.023 max:   4.921 total: 189.794] bytes[avg:      57 max:     135 total:  471529]
SendLoop 6           Calls: 8144 times[avg:   0.021 max:   3.417 total: 172.712] bytes[avg:      57 max:     135 total:  471529]
SendLoop 7           Calls: 8144 times[avg:   0.023 max:   3.466 total: 190.707] bytes[avg:      57 max:     135 total:  471529]
SendLoop 8           Calls: 8144 times[avg:   0.022 max:   4.316 total: 179.826] bytes[avg:      57 max:     135 total:  471529]

--------------------
[ssl:False  batch:True   flush:True   sleep:False  ]
--------------------
SendLoop 5           Calls: 1089 times[avg:   0.043 max:   1.938 total:  46.922] bytes[avg:     409 max:    6500 total:  445879]
SendLoop 6           Calls: 1062 times[avg:   0.045 max:   2.211 total:  47.970] bytes[avg:     419 max:    4902 total:  445879]
SendLoop 7           Calls: 1091 times[avg:   0.040 max:   2.829 total:  43.563] bytes[avg:     408 max:   13544 total:  445879]
SendLoop 8           Calls:  994 times[avg:   0.051 max:   0.634 total:  50.408] bytes[avg:     448 max:    4731 total:  445879]

--------------------
[ssl:False  batch:True   flush:True   sleep:True   ]
--------------------
SendLoop 1           Calls:  258 times[avg:   0.113 max:   0.831 total:  29.208] bytes[avg:    1798 max:    5358 total:  463987]
SendLoop 2           Calls:  257 times[avg:   0.111 max:   0.808 total:  28.506] bytes[avg:    1805 max:    5358 total:  463987]
SendLoop 3           Calls:  256 times[avg:   0.175 max:   0.819 total:  44.847] bytes[avg:    1812 max:    5358 total:  463987]
SendLoop 4           Calls:  257 times[avg:   0.108 max:   0.822 total:  27.653] bytes[avg:    1856 max:   13548 total:  477229]

--------------------
[ssl:True   batch:False  flush:False  sleep:False  ]
--------------------
SendLoop 1           Calls: 8046 times[avg:   0.298 max:  15.826 total:2397.805] bytes[avg:      56 max:     135 total:  458593]
SendLoop 2           Calls: 8046 times[avg:   0.297 max:  14.304 total:2391.197] bytes[avg:      56 max:     135 total:  458593]
SendLoop 3           Calls: 8130 times[avg:   0.295 max:  16.349 total:2396.536] bytes[avg:      57 max:     135 total:  470731]
SendLoop 4           Calls: 8130 times[avg:   0.296 max:  16.047 total:2406.474] bytes[avg:      57 max:     135 total:  470731]

--------------------
[ssl:True   batch:True   flush:False  sleep:False  ]
--------------------
SendLoop 1           Calls:  572 times[avg:   1.616 max: 620.973 total: 924.200] bytes[avg:     766 max:    8892 total:  438166]
SendLoop 3           Calls:  582 times[avg:   1.633 max: 620.940 total: 950.542] bytes[avg:     752 max:    8892 total:  438166]
SendLoop 4           Calls:  586 times[avg:   1.621 max: 621.020 total: 949.926] bytes[avg:     747 max:    8892 total:  438166]
SendLoop 5           Calls:  571 times[avg:   1.629 max: 621.738 total: 930.361] bytes[avg:     790 max:   16337 total:  451579]

--------------------
[ssl:True   batch:True   flush:False  sleep:True   ]
--------------------
SendLoop 5           Calls:  253 times[avg:   0.511 max:   2.039 total: 129.351] bytes[avg:    1716 max:   13004 total:  434365]
SendLoop 6           Calls:  249 times[avg:   0.572 max:   3.467 total: 142.525] bytes[avg:    1744 max:    7564 total:  434365]
SendLoop 7           Calls:  250 times[avg:   0.585 max:   2.813 total: 146.178] bytes[avg:    1737 max:   13278 total:  434365]
SendLoop 8           Calls:  249 times[avg:   0.584 max:   3.002 total: 145.536] bytes[avg:    1744 max:   13548 total:  434365]

--------------------
[ssl:True   batch:False  flush:True   sleep:False  ]
--------------------
SendLoop 2           Calls: 8121 times[avg:   0.415 max: 797.596 total:3366.563] bytes[avg:      57 max:     135 total:  468519]
SendLoop 5           Calls: 8145 times[avg:   0.412 max: 772.980 total:3357.117] bytes[avg:      57 max:     135 total:  471586]
SendLoop 6           Calls: 8145 times[avg:   0.412 max: 772.155 total:3351.812] bytes[avg:      57 max:     135 total:  471586]
SendLoop 7           Calls: 8145 times[avg:   0.412 max: 770.828 total:3359.085] bytes[avg:      57 max:     135 total:  471586]

--------------------
[ssl:True   batch:True   flush:True   sleep:False  ]
--------------------
SendLoop 1           Calls:  628 times[avg:   1.981 max: 892.039 total:1243.868] bytes[avg:     722 max:   16341 total:  453688]
SendLoop 2           Calls:  632 times[avg:   1.962 max: 891.821 total:1239.789] bytes[avg:     717 max:   16341 total:  453688]
SendLoop 3           Calls:  607 times[avg:   2.002 max: 892.125 total:1215.095] bytes[avg:     747 max:   16341 total:  453688]
SendLoop 4           Calls:  581 times[avg:   2.046 max: 874.996 total:1188.512] bytes[avg:     780 max:   16341 total:  453688]

--------------------
[ssl:True   batch:True   flush:True   sleep:True   ]
--------------------
SendLoop 1           Calls:  228 times[avg:   1.062 max:  25.138 total: 242.215] bytes[avg:    1906 max:   13413 total:  434707]
SendLoop 2           Calls:  228 times[avg:   1.017 max:  25.393 total: 231.821] bytes[avg:    1906 max:   13278 total:  434707]
SendLoop 3           Calls:  229 times[avg:   0.962 max:  25.136 total: 220.194] bytes[avg:    1898 max:   13139 total:  434703]
SendLoop 5           Calls:  227 times[avg:   1.020 max:  25.259 total: 231.569] bytes[avg:    1914 max:   13548 total:  434536]

Conclusions

  • Flush seems to have little different over NetworkStream but makes SslStream slower
  • Batching message before stream.write inproves normal and ssl writes
    • NetworkStream x5 faster
    • SslStream x2.5 faster (results show over 1/2 of the write time was in 1 write call, see max. May need more testing)
  • Batching for ssl seems to result is larger writes than
  • batch+sleep before write seems to have a large effect on ssl stream, should be optional (trading delay vs cpu time)
    • NetworkStream x6 faster than master, x1.2 faster than batching
    • SslStream x18 faster than master, x7 faster than batching
    • might be worth testing with builtin noDelay to see if that effects write times.

@James-Frowen James-Frowen force-pushed the James-Frowen-patch-1 branch 2 times, most recently from 08b10ed to 19327fe Compare November 16, 2020 22:51
@James-Frowen
Copy link
Contributor Author

James-Frowen commented Nov 16, 2020

Some more data about send times https://docs.google.com/spreadsheets/d/1amQQ2yfzKO4vjSeAJquv2GUp8WPNPciobxcITeLsnVE/edit?usp=sharing

Seems that the high max for batched send happens near the start, and not all of the time.

@sonarqubecloud
Copy link

sonarqubecloud bot commented Nov 16, 2020

Kudos, SonarCloud Quality Gate passed!

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities (and Security Hotspot 0 Security Hotspots to review)
Code Smell A 0 Code Smells

No Coverage information No Coverage information
No Duplication information No Duplication information

@James-Frowen James-Frowen merged commit 68762b9 into master Nov 17, 2020
@James-Frowen James-Frowen deleted the James-Frowen-patch-1 branch November 17, 2020 00:02
James-Frowen added a commit to MirrorNetworking/Mirror that referenced this pull request Nov 17, 2020
@James-Frowen
perf: Increase write performance (MirrorNetworking/SimpleWebTransport#25
793d347 
)

* adding options to increase performance of send loop

adding config to Transport

* removing extra comments

* changing default value of sleep to false
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@James-Frowen