Skip to content

Migrate most of Process from C to Rust#2631

Merged
sporksmith merged 27 commits intoshadow:mainfrom
sporksmith:process-switch
Jan 6, 2023
Merged

Migrate most of Process from C to Rust#2631
sporksmith merged 27 commits intoshadow:mainfrom
sporksmith:process-switch

Conversation

@sporksmith
Copy link
Copy Markdown
Contributor

@sporksmith sporksmith commented Dec 22, 2022

Best reviewed as individual commits.

This moves each member of the C Process struct into the Rust struct, and adds corresponding Rust methods. Many process_ methods are now thin wrappers around a _process_ method that operates on the Rust process.

I ran out of time to do the thread list, but don't foresee any major complications there. The next steps will be roughly:

  • Move the last member (thread list) of Process to Rust
  • Implement the remaining process_ methods in Rust, such that they're all thin wrappers around _process_ methods taking the Rust Process.
  • The big changeover:
    • Delete process.c and process.h.
    • Rename or alias RustProcess to Process in the C code.
    • Rename the _process_* accessors to process_* - i.e. they'll have almost the same signature as the methods they're replacing, except the first parameter will be a const pointer to the RustProcess.

@github-actions github-actions bot added Component: Libraries Support functions like LD_PRELOAD and logging Component: Main Composing the core Shadow executable labels Dec 22, 2022
@codecov
Copy link
Copy Markdown

codecov bot commented Dec 22, 2022

Codecov Report

Base: 68.15% // Head: 68.01% // Decreases project coverage by -0.13% ⚠️

Coverage data is based on head (d44764f) compared to base (c82a9e4).
Patch coverage: 81.26% of modified lines in pull request are covered.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2631      +/-   ##
==========================================
- Coverage   68.15%   68.01%   -0.14%     
==========================================
  Files         201      201              
  Lines       29502    29852     +350     
  Branches     5794     5814      +20     
==========================================
+ Hits        20107    20304     +197     
- Misses       4784     4930     +146     
- Partials     4611     4618       +7     
Flag Coverage Δ
tests 68.01% <81.26%> (-0.14%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
src/lib/shadow-shim-helper-rs/src/shim_shmem.rs 74.91% <ø> (-2.58%) ⬇️
src/main/core/worker.rs 82.55% <ø> (ø)
src/main/host/memory_manager/mod.rs 67.74% <ø> (-8.71%) ⬇️
src/main/host/syscall/formatter.rs 79.81% <0.00%> (-0.74%) ⬇️
src/main/lib.rs 100.00% <ø> (ø)
src/main/host/host.rs 77.59% <62.16%> (-3.74%) ⬇️
src/main/core/support/configuration.rs 75.33% <66.66%> (-1.67%) ⬇️
src/main/host/descriptor/mod.rs 67.84% <72.72%> (+0.20%) ⬆️
src/main/host/process.rs 84.09% <84.00%> (-1.27%) ⬇️
src/main/core/manager.rs 64.52% <100.00%> (+0.01%) ⬆️
... and 20 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@sporksmith sporksmith changed the title Process switch Migrate most of Process from C to Rust Dec 23, 2022
@sporksmith
Copy link
Copy Markdown
Contributor Author

sporksmith commented Dec 23, 2022

Started a tor benchmark: https://github.com/shadow/benchmark/actions/runs/3762181499

Update:

Results: https://github.com/shadow/benchmark-results/tree/master/tor/2022-12-23-T00-36-14

Looks like no change, as expected.

@sporksmith sporksmith marked this pull request as ready for review December 23, 2022 00:59
@stevenengler
Copy link
Copy Markdown
Contributor

stevenengler commented Jan 5, 2023

@sporksmith Is it straightforward for you to move a6a0045 to a new PR? I need more time to look through this commit, but the rest looks good to me and don't want to hold it up.

Using it directly this way skips the runtime checking that Process does
to ensure there are no incompatible borrows from the MemoryManager.

I think the primary reason it was being used here was so that errors
writing to memory could be handled instead of panicking. Now that
process_flushPtrs supports this, we can use that instead.
The C code was attempting to detect incompatible borrows, but had some
gaps. It detected incompatible borrows by the legacy "caching" memory
access APIs with each-other, but not with the newer APIs.

During this migration I cached the Ref or RefMut of the MemoryManager
used to create the memory reference as well as the memory reference
itself. This requires a self-referential data structure and is a bit
fragile, but should prevent incompatible borrows more reliably than the
old code.

As a result, I had to add some extra flushes to ensure there weren't
outstanding borrows from a syscall handler when running the
strace-logging code, which also tries to access memory.
@sporksmith sporksmith enabled auto-merge January 6, 2023 19:36
@github-actions github-actions bot added the Component: Documentation In-repository documentation, under docs/ label Jan 6, 2023
@sporksmith sporksmith merged commit d30684c into shadow:main Jan 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Component: Documentation In-repository documentation, under docs/ Component: Libraries Support functions like LD_PRELOAD and logging Component: Main Composing the core Shadow executable

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants