Node exceptions using Uint8Array in async fn #2245
Description
I ran into what seems like a double-free when using Uint8Array in an async fn.
Unfortunately, I don't have a small reproducible test, but the scenario looked like the following (simplified to highlight the problem):
#[napi]
pub struct Stream {
send: SendStream
}
#[napi]
impl Stream {
#[napi]
pub async unsafe fn write(&mut self, data: Uint8Array) -> Result<()> {
self.send.write_all(&data).await.map_err(to_err)
}
}Later used like so:
async function sink (stream: Stream, source: AsyncIterable<Uint8Array>) {
for await (const chunk of source) {
await stream.write(chunk)
}
}In unit tests, everything works as it should. However, when integrated in a live application, it results in segfaults and other node stack traces:
eg:
#
# Fatal error in , line 0
# Check failed: backing_store->is_wasm_memory().
#FailureMessage Object: 0x7f2323e39950
----- Native stack trace -----
1: 0xffab01 [node]
2: 0x28b3dfb V8_Fatal(char const*, ...) [node]
3: 0x15be061 [node]
4: 0x15be085 v8::internal::BackingStore::~BackingStore() [node]
5: 0x121f42b std::_Sp_counted_deleter<v8::internal::BackingStore*, std::default_delete<v8::internal::BackingStore>, std::allocator<void>, (__gnu_cxx::_Lock_policy)2>::_M_dispose() [node]
6: 0x13d5a82 v8::internal::ArrayBufferSweeper::SweepingJob::SweepListFull(v8::internal::ArrayBufferList*) [node]
7: 0x13d5dc9 v8::internal::ArrayBufferSweeper::SweepingJob::Sweep() [node]
8: 0x13d5fcc v8::internal::ArrayBufferSweeper::DoSweep(v8::internal::ArrayBufferSweeper::SweepingType, v8::internal::ThreadKind, unsigned long) [node]
9: 0x13d66d9 v8::internal::ArrayBufferSweeper::RequestSweep(v8::internal::ArrayBufferSweeper::SweepingType, v8::internal::ArrayBufferSweeper::TreatAllYoungAsPromoted) [node]
10: 0x148b592 v8::internal::MarkCompactCollector::Finish() [node]
11: 0x14691ae v8::internal::Heap::MarkCompact() [node]
12: 0x1469d35 v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::internal::GarbageCollectionReason, char const*) [node]
13: 0x146b089 [node]
14: 0x146e518 [node]
15: 0x1e05891 [node]
/home/devops/beacon/beacon_run.sh: line 17: 2795598 Illegal instruction (core dumped) node /usr/src/lodestar/packages/cli/bin/lodestar beacon --rcConfig /home/devops/beacon/rcconfig.yml
or
#
# Fatal error in , line 0
# Check failed: object_ != kGlobalHandleZapValue.
#FailureMessage Object: 0x7f5d3ecf68b0
----- Native stack trace -----
1: 0xffab01 [node]
2: 0x28b3dfb V8_Fatal(char const*, ...) [node]
3: 0x13ce325 [node]
4: 0xf1246c napi_reference_unref [node]
5: 0x7f6f11bd3449 [/usr/src/lodestar/node_modules/@chainsafe/libp2p-quic/libp2p-quic.linux-x64-gnu.node]
6: 0x7f6f11cc629e [/usr/src/lodestar/node_modules/@chainsafe/libp2p-quic/libp2p-quic.linux-x64-gnu.node]
7: 0x7f6f11c0ec74 [/usr/src/lodestar/node_modules/@chainsafe/libp2p-quic/libp2p-quic.linux-x64-gnu.node]
8: 0x7f6f11c0b8c9 [/usr/src/lodestar/node_modules/@chainsafe/libp2p-quic/libp2p-quic.linux-x64-gnu.node]
9: 0x7f6f11cbde41 [/usr/src/lodestar/node_modules/@chainsafe/libp2p-quic/libp2p-quic.linux-x64-gnu.node]
10: 0x7f6f11c9fed4 [/usr/src/lodestar/node_modules/@chainsafe/libp2p-quic/libp2p-quic.linux-x64-gnu.node]
11: 0x7f6f11cbce5d [/usr/src/lodestar/node_modules/@chainsafe/libp2p-quic/libp2p-quic.linux-x64-gnu.node]
12: 0xf2d3b8 [node]
13: 0x1e6bd43 [node]
14: 0x1e80a74 [node]
15: 0x1e6ca67 uv_run [node]
16: 0xe77d46 node::SpinEventLoopInternal(node::Environment*) [node]
17: 0x109b3a6 node::worker::Worker::Run() [node]
18: 0x109b869 [node]
19: 0x7f6f43915ac3 [/lib/x86_64-linux-gnu/libc.so.6]
20: 0x7f6f439a7850 [/lib/x86_64-linux-gnu/libc.so.6]
/home/devops/beacon/beacon_run.sh: line 17: 2793724 Illegal instruction (core dumped) node /usr/src/lodestar/packages/cli/bin/lodestar beacon --rcConfig /home/devops/beacon/rcconfig.yml
When the function signature is changed to use Vec<u8> and there are no stack traces
pub async unsafe fn write(&mut self, data: Vec<u8>) -> Result<()> {Also, the function can be rewritten as an AsyncTask (being sure to ref/unref the js value) and there are no stack traces
#[napi]
pub struct Stream {
send: Arc<Mutex<SendStream>>
}
#[napi]
impl Stream {
#[napi]
pub fn write(&mut self, #[napi(ts_arg_type = "Uint8Array")] data: JsTypedArray) -> Result<AsyncTask<Write>> {
let data = data.into_value()?;
let byte_offset = data.byte_offset;
let length = data.length;
let data = data.arraybuffer.into_ref()?;
Ok(AsyncTask::new(Write {
data,
byte_offset,
length,
send: self.send.clone(),
}))
}
}
pub struct Write {
data: Ref<JsArrayBufferValue>,
byte_offset: usize,
length: usize,
send: Arc<Mutex<SendStream>>,
}
impl Task for Write {
type Output = ();
type JsValue = JsUndefined;
fn compute(&mut self) -> Result<Self::Output> {
block_on(async move {
let mut send = self.send.lock().await;
send.write_all(&self.data[self.byte_offset..self.byte_offset+self.length]).await.map_err(to_err)
})
}
fn resolve(&mut self, env: Env, _output: Self::Output) -> Result<Self::JsValue> {
env.get_undefined()
}
fn finally(&mut self, env: Env) -> Result<()> {
self.data.unref(env)?;
Ok(())
}
}