New lib for optionally preloading crypto functions#1853
New lib for optionally preloading crypto functions#1853robgjansen merged 12 commits intoshadow:mainfrom
Conversation
|
Adding benchmark runtime graph:
|
|
I verified that the crypto preload lib is correctly appearing in the nodes' Which is why we see no significant change in performance. A grab from
|
|
It appears that tor is being linked against I don't think we should have to build openssl manually like we used to do (and then link tor to it using its |
|
It looks like I can reproduce this in an ubuntu 20.04 container on an ubuntu 20.04 host. I see the "Loading the preloaded crypto interception lib" in the plugin's stderr, but no counters (I removed the |
|
The openssl crypto functions are not intercepted in an ubuntu 20.04 container, but they are intercepted in a debian 11 container. The openssl rand functions are intercepted in both the ubuntu 20.04 and debian 11 containers. |
e4c5916 to
df13476
Compare
Explanation of the problem is at #1370 (comment), work-around is to conditionally intercept Now waiting on a performance benchmark test to see if it works. |
|
Latest findings: doesn't seem to work on Ubuntu 20.04 because there is a fair amount of AES triggered internally in openssl. It does seem to work as expected in Debian 11 though:
|
df13476 to
60741d0
Compare
|
@sporksmith What do you think about just removing the whole cache and |
The current approach might be nice to keep around if we ever want to pursue it further - e.g. we care about getting it to work on other platforms, or it stops working on Debian. I could go either way though. |
4fb7810 to
3202172
Compare
|
Latest results using benchmark runner in Debian 11 container running on machine without AES-NI, and after setting things up for consistent simulation stop times. The PR (with
|
The option is false by default. If set to true, we will add the openssl crypto preload lib to the LD_PRELOAD path for all nodes running in Shadow to skip some encrypt and decrypt operations. This may speed up simulations, particularly on older CPUs that do not support AES-NI optimizations. Turning on crypto preloading should be done with care, since it may have unintended side effects for the applications running in Shadow.
Giving them a common prefix ensures that they appear together in the Shadow options list, making it easier to find them and to see that there are multiple related options available.
We keep a small cache of callers into EVP_EncryptUpdate so we don't have to perform expensive string operations on every call. For now we just use a small cache of pointers and a dead simple caching policy; this works fine in Tor, where testing reveals that there is a small number of callers of EVP_EncryptUpdate from tor/libssl.so.
3202172 to
d0c09aa
Compare
|
Full benchmark results: |
Adds a new lib that preloads some AES functions so we can optionally skip some CPU crypto operations for faster simulation on older hardware (e.g. CPUs that do not support AES-NI).
When enabled (it's off by default), the preload lib will be injected into the LD_PRELOAD environment for all nodes, and we skip some AES symmetric encryption and decryption operations, including the ones used by Tor. (Disabling encryption is safe in private, synthetic Shadow nets.)
refs #1370