Skip to content

Some determinism fixes#1460

Merged
robgjansen merged 12 commits intoshadow:mainfrom
robgjansen:determinism
Jun 28, 2021
Merged

Some determinism fixes#1460
robgjansen merged 12 commits intoshadow:mainfrom
robgjansen:determinism

Conversation

@robgjansen
Copy link
Copy Markdown
Member

@robgjansen robgjansen commented Jun 19, 2021

This primarily includes a fix for a confirmed non-deterministic behavior I found in the epoll getEvents function (in c6039d3).

I also updated a couple of other places in the code where we iterate a hash table (which is non-deterministic) - these were unconfirmed issues but I suspected they might cause problems. (Probably best if we stop using glib hash table iterators in general, and instead prefer an iterator that guarantees a total ordering; rust probably makes this easier.)

Closes #1294

@robgjansen robgjansen added Type: Bug Error or flaw producing unexpected results Component: Main Composing the core Shadow executable labels Jun 19, 2021
@robgjansen robgjansen self-assigned this Jun 19, 2021
@robgjansen robgjansen mentioned this pull request Jun 19, 2021
7 tasks
@codecov
Copy link
Copy Markdown

codecov bot commented Jun 19, 2021

Codecov Report

Merging #1460 (cf8d759) into main (6872439) will increase coverage by 0.01%.
The diff coverage is 71.11%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1460      +/-   ##
==========================================
+ Coverage   53.73%   53.74%   +0.01%     
==========================================
  Files         137      137              
  Lines       20578    20614      +36     
  Branches     5210     5223      +13     
==========================================
+ Hits        11058    11080      +22     
- Misses       6605     6611       +6     
- Partials     2915     2923       +8     
Flag Coverage Δ
tests 53.74% <71.11%> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
src/main/host/descriptor/tcp.c 77.85% <61.53%> (-0.24%) ⬇️
src/main/host/descriptor/epoll.c 77.73% <75.00%> (-0.87%) ⬇️
src/main/host/syscall/unistd.c 49.26% <0.00%> (-0.50%) ⬇️
src/main/host/thread_ptrace.c 49.33% <0.00%> (-0.17%) ⬇️
src/main/host/syscall/socket.c 78.30% <0.00%> (-0.15%) ⬇️
src/main/core/worker.c 75.86% <0.00%> (+0.68%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6872439...cf8d759. Read the comment docs.

@robgjansen robgjansen marked this pull request as ready for review June 19, 2021 02:32
@robgjansen robgjansen requested a review from sporksmith June 19, 2021 02:32
Copy link
Copy Markdown
Contributor

@sporksmith sporksmith left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving now since the current state is definitely an improvement, but maybe worth a re-review if you do decide to switch to trees.

@robgjansen
Copy link
Copy Markdown
Member Author

I would like to check the performance implications of the changes in the PR before merging it.

@robgjansen
Copy link
Copy Markdown
Member Author

robgjansen commented Jun 28, 2021

Tested this PR in a 5% Tor network suing shadow/tornettools. Before is at 6872439, after is this branch rebased on top of 6872439.

run_time

This PR appears to have a negligible performance impact.

@robgjansen robgjansen enabled auto-merge June 28, 2021 00:07
@robgjansen robgjansen merged commit 0bac752 into shadow:main Jun 28, 2021
@robgjansen robgjansen deleted the determinism branch June 28, 2021 00:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Component: Main Composing the core Shadow executable Type: Bug Error or flaw producing unexpected results

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Re-enable phold determinism test and get it to pass

2 participants