Skip to content

feat(preview1): implement core I/O functionality#6440

Merged
pchickey merged 3 commits intobytecodealliance:mainfrom
rvolosatovs:feat/preview1-preview2
May 24, 2023
Merged

feat(preview1): implement core I/O functionality#6440
pchickey merged 3 commits intobytecodealliance:mainfrom
rvolosatovs:feat/preview1-preview2

Conversation

@rvolosatovs
Copy link
Copy Markdown
Member

@rvolosatovs rvolosatovs commented May 23, 2023

Moved from bytecodealliance/preview2-prototyping#175

Refs #6370

The implementation tries to follow existing preview1 behavior as much as possible - as a result, there are a few differences in behavior from existing adapter. Please see the original PR for more details.

A follow-up issues/PRs fill be filed for:

  • fd_readdir
  • fixing windows support for a few tests (still not sure what's causing the failures)
  • making all preview1 implementations consistent
  • adding rights handling

Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net>
Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net>
@rvolosatovs rvolosatovs force-pushed the feat/preview1-preview2 branch 2 times, most recently from c6585fd to fad7fa5 Compare May 23, 2023 17:40
@github-actions github-actions bot added the wasi Issues pertaining to WASI label May 23, 2023
@github-actions
Copy link
Copy Markdown

Subscribe to Label Action

cc @kubkon

Details This issue or pull request has been labeled: "wasi"

Thus the following users have been cc'd because of the following labels:

  • kubkon: wasi

To subscribe or unsubscribe from this label, edit the .github/subscribe-to-label.json configuration file.

Learn more.

@rvolosatovs rvolosatovs marked this pull request as ready for review May 23, 2023 18:25
@rvolosatovs rvolosatovs requested a review from a team as a code owner May 23, 2023 18:25
@rvolosatovs rvolosatovs requested review from itsrainy and removed request for a team May 23, 2023 18:25
@rvolosatovs
Copy link
Copy Markdown
Member Author

cc @pchickey

@itsrainy
Copy link
Copy Markdown
Contributor

this looks good to me, but I'll defer to @pchickey as the shepherd of the preview2 -> wasmtime move.

@itsrainy itsrainy requested review from pchickey and removed request for itsrainy May 23, 2023 20:20
Copy link
Copy Markdown
Contributor

@pchickey pchickey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great work! Just a few small issues. Thank you for finding all of these bad behaviors in the legacy implementation!

let descriptors = self.descriptors.take();
self.view.adapter_mut().descriptors = Some(descriptors);
}
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wasn't expecting this sort of machinery to be required - we can't get away with implementing the DescriptorsRef methods on WasiPreview1Ctx? I can't intuit where the problem is in a more straightforward implementation, so if this is required, lets leave a comment on the struct definition explaining why.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll add a comment, but the biggest reasons why this is required are:

  1. By far the biggest issue is bindgen generating host methods with &mut self receivers, preventing e.g. simultaneous lazy initialization of the descriptor table and calling a preview2 method using any data borrowed from that table, even looking up something like table().is_file is problematic
  2. WasiPreview1Adapter (assuming that's what you meant by WasiPreview1Ctx) does not have access to preview2 functionality and/or the underlying table, so it cannot initialize itself using something like https://docs.rs/once_cell/latest/once_cell/sync/struct.Lazy.html . Lazily initializing the descriptor map requires access to both the WasiPreview1Adapter and wasi::preopens::Host, so the only place where this functionality could exist is in an extension trait (WasiPreview1ViewExt). Note that from within this trait, we can never return anything borrowed directly, because we'd first need to call adapter_mut() to get access to the state (which we may need to lazily initialize). And even if we had a way to return a borrowed descriptor table directly, it would not be possible to call any other method on self due to 1., since that would cause a double mutable borrow, which is not allowed.

In short, an alternative solution would be storing Option<Arc<Mutex<Descriptors>>> in the adapter and returning Arc<Mutex<Descriptors>> in the WasiPreview1AdapterExt::descriptors(&mut self) method, which IMO is way more awkward to work with and simply redundant complexity.
Overall, the DescriptorsRef is just a workaround for bindgen design drawbacks. I believe this could be simplified dramatically if host methods would just take &self

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That makes sense. Thank you.

We should look into adding a bindgen option to enable &self receivers in the future, and then we may be able to unwind a bit of this complexity. For now, this is a fine implementation once it has a comment.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've added a few comments and renamed the struct to Transaction so it's a bit more obvious what it actually is

_fs_rights_inheriting: types::Rights,
) -> Result<(), types::Error> {
todo!()
unreachable!()
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is still a todo!() - we can leave it to a follow up PR, but unreachable!() communicates to me that it is a bug in this code if it is reachable, while this is reachable from any guest program.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I took this directly from

/// Adjust the rights associated with a file descriptor.
/// This can only be used to remove rights, and returns `errno::notcapable` if called in a way that would attempt to add rights
#[no_mangle]
pub unsafe extern "C" fn fd_fdstat_set_rights(
fd: Fd,
fs_rights_base: Rights,
fs_rights_inheriting: Rights,
) -> Errno {
unreachable!()
}
and it was surprising to me as well.
It seems to me that the adapter has to implement rights handling, which the existing one does not do. I'll mark this as todo! and file an issue for rights handling follow-up

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, that is indeed a bug in the component adapter as well. unreachabable!() is just how we spell todo!() and panic!() and etc over there since we cant use std's macros.

In the legacy implementation, set_rights is a no-op, it only validates the fd. That is the correct behavior for both of these spots. Would you mind making this fix to the component adapter as well?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's take care of it as part of #6446 ?

prtest:full

Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net>
@pchickey pchickey added this pull request to the merge queue May 24, 2023
Merged via the queue into bytecodealliance:main with commit 6bbfbcc May 24, 2023
@rvolosatovs rvolosatovs deleted the feat/preview1-preview2 branch May 25, 2023 15:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

wasi Issues pertaining to WASI

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants