Error running Shadow under tornettools after latest Fedora 37 update #2686
Replies: 4 comments 8 replies
-
|
Update: I've found that the "Simple Tor Network" example in the Shadow documentation (https://shadow.github.io/docs/guide/getting_started_tor.html), not using tornettools, is producing the same error. I've tried rebuilding tor from the latest source, and building it from a much older source, and got the same error. All of these tor versions launch fine outside of Shadow. Here's the output from the "Simple Tor Network" example.
|
Beta Was this translation helpful? Give feedback.
-
Are you sure you have enough system memory? How much memory does your system have, and how much is remaining when you start the simulation? Are you running in Docker or any other virtualized/containerized environment? What happens if you run Shadow with the |
Beta Was this translation helpful? Give feedback.
-
|
This machine has 32 GB of memory. Once I startup a 2% scale tornettools simulation, top shows me about 26 GB free and it slowly ticks down from there as the simulation progresses. This is a bare-metal installation of Fedora. I just tried using the What are the consequences of continuing to use |
Beta Was this translation helpful? Give feedback.
-
|
Can you post the output of |
Beta Was this translation helpful? Give feedback.
-
|
Here's that output. It looks like my simulation took almost twice as much time as usual, but worked as advertised. You're awesome, by the way.
|
Beta Was this translation helpful? Give feedback.
-
|
@sporksmith Any idea why it would be returning |
Beta Was this translation helpful? Give feedback.
-
Yeah, that's weird. Seeing if I can reproduce in Docker...
Wow, that's surprising. Lately I've only seen a small % performance difference with it on or off, including local runs of the tor minimal test. Guess I should take a look at a bigger sim in #2581 . (Of course, it's also possible the difference is related to something else in fedora 38 or system configuration etc) |
Beta Was this translation helpful? Give feedback.
-
|
The minimal tor test works for me in a fedora 37 docker container. (I accidentally tried fedora 38, on which it turns out tgen doesn't build, but let's disregard that yak for now shadow/tgen#44) I had to make a small fix to the integration test scripts, which I'll send out shortly, but it's orthogonal to this. Anyway this doesn't necessarily tell us a whole lot other than that we can't easily reproduce the issue. e.g. the problem could be related to the newer kernel, which wouldn't reproduce in Docker since it uses the host kernel. |
Beta Was this translation helpful? Give feedback.
-
|
@sporksmith Yeah, it's the newer kernel. You gave me the idea to try booting with the previous 6.0.16 kernel, and Shadow's included "Simple Tor Network" test completed successfully (without disabling the memory manager). Then I booted back into the latest 6.1.6 kernel and it gave me the ENOMEM error again. Now that I can run with or without the memory manager, I'll try to give you a more precise observation of the slowdown that I'm seeing with the memory manager disabled. I might have been comparing two simulations with different consensus files and a few other differences. |
Beta Was this translation helpful? Give feedback.
-
|
Filed #2692 |
Beta Was this translation helpful? Give feedback.
-
|
I found the problem and a workaround (in case the performance difference of using |
Beta Was this translation helpful? Give feedback.
-
|
Very cool! I'm happy to keep running the 6.0.16 kernel for now to avoid the issue, but it's comforting to know that the issue might be fixed by the time Fedora removes this kernel. |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
I'll preface this by acknowledging that the Shadow documentation only claims support for Fedora 34, 35, and 36. However, since 34 and 35 are now end-of-life, and we're approaching the release of 38, I think this is relevant. My Fedora 37 system received a large number of package updates, including critical security updates, yesterday evening. This morning, my tornettools simulations (including the minimal tornet-0.01 example) are now terminating with the following output.
The basic example Shadow simulations without tornettools still seem to run fine, which is puzzling to me because tornettools appears to use Shadow in a straightforward manner under the hood (removing the few default arguments to Shadow didn't help either). The basic example Shadow simulations that don't use tor still seem to run fine, so it's a tor + Shadow issue instead of a tornettools issue. I've already updated to the latest v2.3.0 of Shadow. Before I do something drastic like wiping this machine and starting over with Fedora 36, since that might also be affected, do we have any ideas?Beta Was this translation helpful? Give feedback.
All reactions