Skip to content

Run small Tor network as part of CI tests#1184

Merged
stevenengler merged 9 commits intoshadow:devfrom
stevenengler:run-tor-in-ci
Mar 22, 2021
Merged

Run small Tor network as part of CI tests#1184
stevenengler merged 9 commits intoshadow:devfrom
stevenengler:run-tor-in-ci

Conversation

@stevenengler
Copy link
Copy Markdown
Contributor

@stevenengler stevenengler commented Mar 15, 2021

This PR adds a small tor experiment to the CMake tests that is not run by default. This test is run in the CI whenever the dev branch is updated, or if run manually using a workflow_dispatch event.

@codecov
Copy link
Copy Markdown

codecov bot commented Mar 15, 2021

Codecov Report

Merging #1184 (751ddf1) into dev (66b9971) will increase coverage by 0.04%.
The diff coverage is 89.28%.

Impacted file tree graph

@@            Coverage Diff             @@
##              dev    #1184      +/-   ##
==========================================
+ Coverage   55.46%   55.50%   +0.04%     
==========================================
  Files         140      140              
  Lines       20239    20267      +28     
  Branches     4895     4897       +2     
==========================================
+ Hits        11225    11250      +25     
- Misses       6007     6008       +1     
- Partials     3007     3009       +2     
Flag Coverage Δ
tests 55.50% <89.28%> (+0.04%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
src/main/core/manager.c 67.32% <0.00%> (-0.45%) ⬇️
src/main/core/support/object_counter.c 92.50% <96.15%> (+0.39%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 66b9971...751ddf1. Read the comment docs.

@stevenengler stevenengler force-pushed the run-tor-in-ci branch 4 times, most recently from f56450b to 306cd63 Compare March 17, 2021 00:07
@robgjansen
Copy link
Copy Markdown
Member

The v3bw file was supposed to be loaded by the authority. The tor.authority.torrc file has the following option:

V3BandwidthsFile shadow.data/hosts/torflowauthority/v3bw

Maybe we can just put the v3bw file in the 4uthority dir instead, and update this path? Or maybe we don't actually need the v3bw file after all? (I'm surprised that Tor does not fail since the above file no longer exists at the specified path.)


In full.config.yaml it does look like the torbridgeclient is using the 4uthority relay as the bridge instead of a separate bridge relay:

...--UseBridges 1 --Bridge 100.0.0.1:9111

So removing the bridge dir seems ok.

@stevenengler stevenengler force-pushed the run-tor-in-ci branch 4 times, most recently from 4e4da0d to c151fa2 Compare March 19, 2021 03:02
@stevenengler
Copy link
Copy Markdown
Contributor Author

The v3bw file was supposed to be loaded by the authority. The tor.authority.torrc file has the following option:

V3BandwidthsFile shadow.data/hosts/torflowauthority/v3bw

Maybe we can just put the v3bw file in the 4uthority dir instead, and update this path? Or maybe we don't actually need the v3bw file after all? (I'm surprised that Tor does not fail since the above file no longer exists at the specified path.)

I added this file back.

In full.config.yaml it does look like the torbridgeclient is using the 4uthority relay as the bridge instead of a separate bridge relay:

...--UseBridges 1 --Bridge 100.0.0.1:9111

So removing the bridge dir seems ok.

Should I re-add the bridge and update the bridge line for torbridgeclient to use it?

@robgjansen
Copy link
Copy Markdown
Member

Should I re-add the bridge and update the bridge line for torbridgeclient to use it?

I don't think we need it because the authority already fills the bridge role. I don't think we need a bridge that does not also act as an authority, but if you are excited to get that working and want to add it, go ahead.

@stevenengler
Copy link
Copy Markdown
Contributor Author

Should I re-add the bridge and update the bridge line for torbridgeclient to use it?

I don't think we need it because the authority already fills the bridge role. I don't think we need a bridge that does not also act as an authority, but if you are excited to get that working and want to add it, go ahead.

I'm fine with leaving it out, just wanted to know if you had a preference.

@stevenengler stevenengler force-pushed the run-tor-in-ci branch 14 times, most recently from 3c75b67 to a6dec41 Compare March 22, 2021 15:29
@stevenengler stevenengler marked this pull request as ready for review March 22, 2021 15:44
@stevenengler stevenengler force-pushed the run-tor-in-ci branch 2 times, most recently from 9360d5d to 31b2bc5 Compare March 22, 2021 20:34
@stevenengler
Copy link
Copy Markdown
Contributor Author

@robgjansen Before merging, I'll disable the on: pull_request trigger for this action. There is also a commit specifically to show changes I made to the config files you sent me, so let me know if you don't agree with any of them.

@stevenengler stevenengler requested a review from robgjansen March 22, 2021 21:07
Comment on lines +26 to +38
add_shadow_tests(BASENAME tor-minimal
METHODS ptrace
LOGLEVEL info
ARGS
--pin-cpus
--workers 2
--data-template "shadow.data.template"
PROPERTIES
TIMEOUT 180
LABELS tor
CONFIGURATIONS tor
RUN_SERIAL TRUE)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should check that Bootstrapped 100 appears 9 times in the Tor log files (once per Tor process) and stream-success 40 times in the tgen log files (10 downloads per tgen client) after the simulation finishes. Are we doing this somewhere that I am overlooking?

Copy link
Copy Markdown
Member

@robgjansen robgjansen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you! Looks good (assuming the tests pass).

It turns out that setting `PASS_REGULAR_EXPRESSION` causes ctest to
ignore the return value of the test, so now we add an explicit
string in Shadow that signals a memory leak and check for it using
`FAIL_REGULAR_EXPRESSION`.
@stevenengler stevenengler merged commit ab58be9 into shadow:dev Mar 22, 2021
@stevenengler stevenengler deleted the run-tor-in-ci branch March 22, 2021 23:50
@stevenengler stevenengler added the Component: Testing Unit and integration tests and frameworks label Jul 2, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Component: Testing Unit and integration tests and frameworks

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants