• Version: 12.13.0
• Platform: Linux x 5.0.0-36-generic Gitter chat room? #39-Ubuntu SMP Tue Nov 12 09:46:06 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
• Subsystem: net, worker_threads

From forks to threads

I was working on moving from forks to Worker Threads and was experimenting with passing file descriptors between threads. Forks work great with passing file descriptors around due to the extended IPC, making it possible to share open ports.

Sharing file descriptors

Using Worker Threads, this becomes impossible, For that reason, I had to create a master thread handling connection events, and pass those connections to worker threads through postMessage. Since the whole object cannot be passed through message, I thought the lightest and fastest way to do this would be to post the fd as a message, and have the worker thread create a new Socket using the fd.

It works great until it does not. It is really unpredictable, but always seems to crash at some point.

Repro steps

Here is the smallest portion of code I could come up with to reproduce the issue. Those are two files : master.js and worker.js.

Full test HERE.

// master.js
const { Worker } = require('worker_threads');
const net = require('net');

const workers = new Worker("./worker.js");

const server = net.createServer(conn => {
    conn.unref();
    workers.postMessage({ duplex_fd : conn._handle.fd });
});

server.listen(12345);

// worker.js
const { Socket } = require('net');

require('worker_threads').parentPort.on('message', (msg) => {
    const sock = new Socket({ fd : msg.duplex_fd, readable : true, writable : true, allowHalfOpen : true });  
    sock.end("Hello, World", () => {
        sock.destroy();
    });
})

After an unpredictable while, I get either one of those two errors :

node: ../deps/uv/src/unix/core.c:930: uv__io_stop: Assertion `loop->watchers[w->fd] == w' failed.
Aborted (core dumped)

or

events.js:187
      throw er; // Unhandled 'error' event
      ^

Error: read EBADF
    at TCP.onStreamRead (internal/stream_base_commons.js:201:27)
Emitted 'error' event on Socket instance at:
    at emitErrorNT (internal/streams/destroy.js:92:8)
    at emitErrorAndCloseNT (internal/streams/destroy.js:60:3)
    at processTicksAndRejections (internal/process/task_queues.js:80:21) {
  errno: 'EBADF',
  code: 'EBADF',
  syscall: 'read'
}

This is something I used to do in C++ : have a master thread handle incoming connections, and pass the fd integer to whatever thread is available. Maybe I'm misunderstanding how Nodejs handles file descriptors in the background?

The full example I wrote had a worker pool and sometimes was able to handle over 5000 requests before crashing. The crashes are random.

If it can help, here is the stress.js file I used to conduct the tests.

// stress.js
const net = require('net');
const cluster = require('cluster');

if (cluster.isMaster) {
    let reqSent = 0;
    for (let i = 0; i < 10; i++) cluster.fork().on('message', m => m == "+" && console.log(reqSent++));
} else {
    const sendReq = () => {
        const sock = net.connect(12345, 'localhost', () => {
            sock.write("Hello?", () => {
                sock.end();
                sock.destroy();
                process.send("+");
                setImmediate(() => sendReq());
            });
        });
    };
    sendReq();
}

Let me know if you need more info, or if I simply misunderstand how to use this feature.

Notes

This also happens with file streams, and sockets on top of HTTP.