Skip to content

Migrate most of Process from C to Rust#2631

Merged
sporksmith merged 27 commits intoshadow:mainfrom
sporksmith:process-switch
Jan 6, 2023
Merged

Migrate most of Process from C to Rust#2631
sporksmith merged 27 commits intoshadow:mainfrom
sporksmith:process-switch

Conversation

@sporksmith
Copy link
Copy Markdown
Contributor

@sporksmith sporksmith commented Dec 22, 2022

Best reviewed as individual commits.

This moves each member of the C Process struct into the Rust struct, and adds corresponding Rust methods. Many process_ methods are now thin wrappers around a _process_ method that operates on the Rust process.

I ran out of time to do the thread list, but don't foresee any major complications there. The next steps will be roughly:

  • Move the last member (thread list) of Process to Rust
  • Implement the remaining process_ methods in Rust, such that they're all thin wrappers around _process_ methods taking the Rust Process.
  • The big changeover:
    • Delete process.c and process.h.
    • Rename or alias RustProcess to Process in the C code.
    • Rename the _process_* accessors to process_* - i.e. they'll have almost the same signature as the methods they're replacing, except the first parameter will be a const pointer to the RustProcess.

@github-actions github-actions Bot added Component: Libraries Support functions like LD_PRELOAD and logging Component: Main Composing the core Shadow executable labels Dec 22, 2022
@codecov
Copy link
Copy Markdown

codecov Bot commented Dec 22, 2022

Codecov Report

Base: 68.15% // Head: 68.01% // Decreases project coverage by -0.13% ⚠️

Coverage data is based on head (d44764f) compared to base (c82a9e4).
Patch coverage: 81.26% of modified lines in pull request are covered.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2631      +/-   ##
==========================================
- Coverage   68.15%   68.01%   -0.14%     
==========================================
  Files         201      201              
  Lines       29502    29852     +350     
  Branches     5794     5814      +20     
==========================================
+ Hits        20107    20304     +197     
- Misses       4784     4930     +146     
- Partials     4611     4618       +7     
Flag Coverage Δ
tests 68.01% <81.26%> (-0.14%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
src/lib/shadow-shim-helper-rs/src/shim_shmem.rs 74.91% <ø> (-2.58%) ⬇️
src/main/core/worker.rs 82.55% <ø> (ø)
src/main/host/memory_manager/mod.rs 67.74% <ø> (-8.71%) ⬇️
src/main/host/syscall/formatter.rs 79.81% <0.00%> (-0.74%) ⬇️
src/main/lib.rs 100.00% <ø> (ø)
src/main/host/host.rs 77.59% <62.16%> (-3.74%) ⬇️
src/main/core/support/configuration.rs 75.33% <66.66%> (-1.67%) ⬇️
src/main/host/descriptor/mod.rs 67.84% <72.72%> (+0.20%) ⬆️
src/main/host/process.rs 84.09% <84.00%> (-1.27%) ⬇️
src/main/core/manager.rs 64.52% <100.00%> (+0.01%) ⬆️
... and 20 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@sporksmith sporksmith changed the title Process switch Migrate most of Process from C to Rust Dec 23, 2022
@sporksmith
Copy link
Copy Markdown
Contributor Author

sporksmith commented Dec 23, 2022

Started a tor benchmark: https://github.com/shadow/benchmark/actions/runs/3762181499

Update:

Results: https://github.com/shadow/benchmark-results/tree/master/tor/2022-12-23-T00-36-14

Looks like no change, as expected.

@sporksmith sporksmith marked this pull request as ready for review December 23, 2022 00:59
@stevenengler
Copy link
Copy Markdown
Contributor

stevenengler commented Jan 5, 2023

@sporksmith Is it straightforward for you to move a6a0045 to a new PR? I need more time to look through this commit, but the rest looks good to me and don't want to hold it up.

Comment thread src/main/host/process.rs Outdated
Comment thread src/main/host/process.rs Outdated
Comment thread src/main/host/process.c
Comment thread src/main/host/process.rs Outdated
Comment thread src/main/host/process.rs
Using it directly this way skips the runtime checking that Process does
to ensure there are no incompatible borrows from the MemoryManager.

I think the primary reason it was being used here was so that errors
writing to memory could be handled instead of panicking. Now that
process_flushPtrs supports this, we can use that instead.
The C code was attempting to detect incompatible borrows, but had some
gaps. It detected incompatible borrows by the legacy "caching" memory
access APIs with each-other, but not with the newer APIs.

During this migration I cached the Ref or RefMut of the MemoryManager
used to create the memory reference as well as the memory reference
itself. This requires a self-referential data structure and is a bit
fragile, but should prevent incompatible borrows more reliably than the
old code.

As a result, I had to add some extra flushes to ensure there weren't
outstanding borrows from a syscall handler when running the
strace-logging code, which also tries to access memory.
@sporksmith sporksmith enabled auto-merge January 6, 2023 19:36
@github-actions github-actions Bot added the Component: Documentation In-repository documentation, under docs/ label Jan 6, 2023
@sporksmith sporksmith merged commit d30684c into shadow:main Jan 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Component: Documentation In-repository documentation, under docs/ Component: Libraries Support functions like LD_PRELOAD and logging Component: Main Composing the core Shadow executable

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants