gaio

gaio

Introduction

中文介绍

In a typical Go network program, you accept a connection with conn := lis.Accept(), then spawn a goroutine to handle incoming data using go func(net.Conn). Next, you allocate a buffer with buf := make([]byte, 4096) and wait for data with conn.Read(buf).

For servers managing 10,000+ connections with frequent short messages (e.g., <512 bytes), context switching costs significantly exceed message receiving costs—each context switch requires over 1,000 CPU cycles, or approximately 600 ns on a 2.1 GHz processor.

By eliminating the one-goroutine-per-connection model through Edge-Triggered I/O Multiplexing, you can save the 2KB (R) + 2KB (W) stack space typically allocated per goroutine. Additionally, using an internal swap buffer eliminates the need for buf := make([]byte, 4096), trading some performance for memory efficiency.

The gaio library implements the proactor pattern, effectively balancing memory constraints with performance requirements.

How It Works

The dup function is used to copy the file descriptor from net.Conn:

NAME
       dup, dup2, dup3 - duplicate a file descriptor

LIBRARY
       Standard C library (libc, -lc)

SYNOPSIS
       #include <unistd.h>

       int dup(int oldfd);
       int dup2(int oldfd, int newfd);

       #define _GNU_SOURCE             /* See feature_test_macros(7) */
       #include <fcntl.h>              /* Definition of O_* constants */
       #include <unistd.h>

       int dup3(int oldfd, int newfd, int flags);

DESCRIPTION
       The dup() system call allocates a new file descriptor that refers to the same open file description as the de‐
       scriptor oldfd.  (For an explanation of open file descriptions, see open(2).)  The new file descriptor  number
       is guaranteed to be the lowest-numbered file descriptor that was unused in the calling process.

       After  a  successful return, the old and new file descriptors may be used interchangeably.  Since the two file
       descriptors refer to the same open file description, they share file offset and file status flags;  for  exam‐
       ple,  if  the  file  offset  is  modified by using lseek(2) on one of the file descriptors, the offset is also
       changed for the other file descriptor.

       The two file descriptors do not share file descriptor flags (the close-on-exec flag).  The close-on-exec  flag
       (FD_CLOEXEC; see fcntl(2)) for the duplicate descriptor is off.

Features

• High Performance: Battle-tested in High-Frequency Trading environments, achieving 30K–40K RPS on a single HVM server.
• Scalability: Designed for C10K+ concurrent connections, optimizing both parallelism and per-connection throughput.
• Flexible Buffering: Use Read(ctx, conn, buffer) with a nil buffer to leverage the internal swap buffer.
• Non-Intrusive Integration: Compatible with net.Listener and net.Conn (supports syscall.RawConn), enabling seamless integration into existing applications.
• Efficient Context Switching: Minimizes context switching overhead for small messages, ideal for high-frequency message exchanges.
• Customizable Delegation: Applications can control when to delegate net.Conn to gaio, such as after handshakes or specific net.TCPConn configurations.
• Back-Pressure Handling: Applications can control read/write request submission timing, enabling per-connection back-pressure management to throttle sending when necessary—particularly useful when transferring data from a faster source (A) to a slower destination (B).
• Lightweight and Maintainable: Approximately 1,000 lines of code, facilitating easy debugging and maintenance.
• Cross-Platform Support: Compatible with Linux and BSD.

Conventions

• Connection Delegation: Once you submit an async read/write request for a net.Conn to gaio.Watcher, that connection becomes delegated to the watcher. Subsequent calls to conn.Read or conn.Write will return errors, but TCP properties set via SetReadBuffer(), SetWriteBuffer(), SetLinger(), SetKeepAlive(), and SetNoDelay() will be preserved.

• Resource Management: When you no longer need a connection, call Watcher.Free(net.Conn) to immediately close the socket and release resources. If you forget to call Watcher.Free(), the runtime garbage collector will clean up system resources when net.Conn is no longer referenced elsewhere. Similarly, failing to call Watcher.Close() will allow the garbage collector to clean up all related resources once the watcher is unreferenced.

• Load Balancing: For connection load balancing, create multiple gaio.Watcher instances and distribute net.Conn using your preferred strategy. For acceptor load balancing, use go-reuseport as the listener.

• Safe Read Requests: When submitting read requests with a nil buffer, the []byte slice returned from Watcher.WaitIO() remains valid until the next Watcher.WaitIO() call.

TL;DR

package main

import (
        "log"
        "net"

        "github.com/xtaci/gaio"
)

// this goroutine waits for all I/O events and sends back everything it receives
// in an async manner
func echoServer(w *gaio.Watcher) {
        for {
                // loop wait for any IO events
                results, err := w.WaitIO()
                if err != nil {
                        log.Println(err)
                        return
                }

                for _, res := range results {
                        switch res.Operation {
                        case gaio.OpRead: // read completion event
                                if res.Error == nil {
                                        // send back everything, we won't start to read again until write completes.
                                        // submit an async write request
                                        w.Write(nil, res.Conn, res.Buffer[:res.Size])
                                }
                        case gaio.OpWrite: // write completion event
                                if res.Error == nil {
                                        // since write has completed, let's start read on this conn again
                                        w.Read(nil, res.Conn, res.Buffer[:cap(res.Buffer)])
                                }
                        }
                }
        }
}

func main() {
        w, err := gaio.NewWatcher()
        if err != nil {
              log.Fatal(err)
        }
        defer w.Close()
	
        go echoServer(w)

        ln, err := net.Listen("tcp", "localhost:0")
        if err != nil {
                log.Fatal(err)
        }
        log.Println("echo server listening on", ln.Addr())

        for {
                conn, err := ln.Accept()
                if err != nil {
                        log.Println(err)
                        return
                }
                log.Println("new client", conn.RemoteAddr())

                // submit the first async read IO request
                err = w.Read(nil, conn, make([]byte, 128))
                if err != nil {
                        log.Println(err)
                        return
                }
        }
}

More examples

package main

import (
        "fmt"
        "log"
        "net"
        "time"

        "github.com/xtaci/gaio"
)

func main() {
        // by simply replace net.Listen with reuseport.Listen, everything is the same as in push-server
        // ln, err := reuseport.Listen("tcp", "localhost:0")
        ln, err := net.Listen("tcp", "localhost:0")
        if err != nil {
                log.Fatal(err)
        }

        log.Println("pushing server listening on", ln.Addr(), ", use telnet to receive push")

        // create a watcher
        w, err := gaio.NewWatcher()
        if err != nil {
                log.Fatal(err)
        }

        // channel
        ticker := time.NewTicker(time.Second)
        chConn := make(chan net.Conn)
        chIO := make(chan gaio.OpResult)

        // watcher.WaitIO goroutine
        go func() {
                for {
                        results, err := w.WaitIO()
                        if err != nil {
                                log.Println(err)
                                return
                        }

                        for _, res := range results {
                                chIO <- res
                        }
                }
        }()

        // main logic loop, like your program core loop.
        go func() {
                var conns []net.Conn
                for {
                        select {
                        case res := <-chIO: // receive IO events from watcher
                                if res.Error != nil {
                                        continue
                                }
                                conns = append(conns, res.Conn)
                        case t := <-ticker.C: // receive ticker events
                                push := []byte(fmt.Sprintf("%s\n", t))
                                // all conn will receive the same 'push' content
                                for _, conn := range conns {
                                        w.Write(nil, conn, push)
                                }
                                conns = nil
                        case conn := <-chConn: // receive new connection events
                                conns = append(conns, conn)
                        }
                }
        }()

        // this loop keeps on accepting connections and send to main loop
        for {
                conn, err := ln.Accept()
                if err != nil {
                        log.Println(err)
                        return
                }
                chConn <- conn
        }
}

Documentation

For complete documentation, see the associated Godoc.

Benchmarks

Testing Directives

On macOS, you need to increase the maximum open files limit to run the benchmarks.

sysctl -w kern.ipc.somaxconn=4096
sysctl -w kern.maxfiles=100000
sysctl -w kern.maxfilesperproc=100000
sysctl -w net.inet.ip.portrange.first=1024
sysctl -w net.inet.ip.portrange.last=65535

ulimit -S -n 65536

Regression

X -> number of concurrent connections, Y -> time of completion in seconds

Best-fit values	 
Slope	8.613e-005 ± 5.272e-006
Y-intercept	0.08278 ± 0.03998
X-intercept	-961.1
1/Slope	11610
 
95% Confidence Intervals	 
Slope	7.150e-005 to 0.0001008
Y-intercept	-0.02820 to 0.1938
X-intercept	-2642 to 287.1
 
Goodness of Fit	 
R square	0.9852
Sy.x	0.05421
 
Is slope significantly non-zero?	 
F	266.9
DFn,DFd	1,4
P Value	< 0.0001
Deviation from horizontal?	Significant
 
Data	 
Number of XY pairs	6
Equation	Y = 8.613e-005*X + 0.08278

FAQ

1. If you encounter an error like:

# github.com/xtaci/gaio [github.com/xtaci/gaio.test]
./aio_linux.go:155:7: undefined: setAffinity
./watcher.go:588:4: undefined: setAffinity
FAIL	github.com/xtaci/gaio [build failed]
FAIL

ensure that gcc/clang is installed.

License

The gaio source code is available under the MIT License.

References

• https://zhuanlan.zhihu.com/p/102890337 -- gaio小记
• https://github.com/xtaci/grasshopper -- A secure chained relayer for UDP made of gaio
• https://github.com/golang/go/issues/15735 -- net: add mechanism to wait for readability on a TCPConn
• https://en.wikipedia.org/wiki/C10k_problem -- C10K
• https://golang.org/src/runtime/netpoll_epoll.go -- epoll in golang
• https://www.freebsd.org/cgi/man.cgi?query=kqueue&sektion=2 -- kqueue
• https://idea.popcount.org/2017-02-20-epoll-is-fundamentally-broken-12/ -- epoll is fundamentally broken
• https://en.wikipedia.org/wiki/Transmission_Control_Protocol#Flow_control -- TCP Flow Control
• http://www.idc-online.com/technical_references/pdfs/data_communications/Congestion_Control.pdf -- Back-pressure

Status

Stable