The current HTTP client instrumentation library httptrace tracks the life cycle of a single request with hooks present at different stages in the outgoing HTTP Request's journey. This limits the use of httptrace in many ways which we believe could be solved by having an instrumentation library which works at a “transport level”. This means instead of placing hooks in the request’s context, one defines these hooks for the Transport. This would also support the current functionality of httptrace.

Proposal

Here is a basic example of the proposed change. Let’s take the example of the GotConn hook in httptrace.ClientTrace. The current method for adding and using this hook is

req, _ := http.NewRequest("GET", "http://example.com", nil)
trace := &httptrace.ClientTrace{
	GotConn: func(connInfo httptrace.GotConnInfo) {
		fmt.Printf("Got Conn: %+v\n", connInfo)
	},
}
req = req.WithContext(httptrace.WithClientTrace(req.Context(), trace))
resp, _ := http.DefaultClient.Do(req)

We propose restructuring http.Transport to contain the hooks instead of ClientTrace. Transport contains a new struct Stats which contains the hooks and other data like counts of idle/active connections and waiting requests.

type Stats struct {
	GotConn func(connInfo GotConnInfo)
        
	// among other hooks and data
}

type Transport struct {
	Stats *Stats
	// and all other fields to stay the same
}

And one would make a request in the following way:-

req, _ := http.NewRequest("GET", "http://example.com", nil)
stats := &http.Stats{
	GotConn: func(connInfo http.GotConnInfo) {
		fmt.Printf("Got Conn: %+v\n", connInfo)
	},
}
transport := http.Transport{Stats: stats}
client := http.Client{Transport: transport}
resp, _ := client.Do(req)

Within Transport, when a request has received a connection, the GotConn hook would run as below. This is in contrast to extracting ClientTrace from the Request’s context and using its hooks.

func (t *Transport) methodName() {
    // a connection has been delivered to the request
    if t.Stats.GotConn != nil {
	  t.Stats.GotConn(GotConnInfo{…})
    }
…
}

The above basic proposal can be extended to all the hooks already present in httptrace. We propose the following additional hooks to be added to http.Stats above:-

1. IdleConnWaitIn, IdleConnWaitOut, ConnsPerHostWaitIn, and ConnsPerHostWaitOut - These hooks will be called when a request (wantConn) enters/exits the idleConnWait and connsPerHostWait queues. For example -

type IdleConnWaitInInfo struct {
    ConnectMethodKey string
    EventTime time.Time
    RemainingInQueue int
}
IdleConnWaitIn(info IdleConnWaitInInfo)

These hooks could take as parameters connectMethodKey of the queue, time of entry/exit, and the number of remaining active wantConns in the queues after the event (an active wantConnis one which has not been delivered a persistConn yet and its context has not exceeded its deadline. Note that this is not equal to the number of wantConns in the queues as some of them could have been delivered a persistConn in another goroutine or their context could have exceeded its deadline.) The IdleConnWaitOut and ConnsPerHostWaitOut hooks could also have an extra parameter error which is nil if the wantConns were removed from the queues because they were successfully delivered a persistConn. If not, error gives the reason for their removal from the queues, for example, if the request was cancelled or its context deadline expired.

2. IdleConnIn, IdleConnOut - IdleConnIn serves the same purpose as the existing PutIdleConn hook from the httptrace package. IdleConnOut is the related hook for when a connection is removed from the idleConn queue. These hooks could take as parameters connectMethodKey of the queue and time of entry/exit. IdleConnOut could take an additional parameter error which is nil if the idle connection was delivered to a wantConn. If not, it represents the reason it was removed from the queue, for example, being too old to be reused.

3. ConnReused - This hook is called after serving a request, if the connection is directly delivered to a wantConn waiting in idleConnWait instead of being added to the idleConn queue. Currently, there is only one hook related to connections, PutIdleConn, which records the latter. ConnReused could take as parameters connectMethodKey of the persistConn (cacheKey field) and time of delivery.

4. MaxIdleConnsHit, MaxIdleConnsPerHost, MaxConnsPerHostHit - When configured capacities of queues like MaxIdleConns, MaxIdleConnsPerHost, MaxConnsPerHost are hit, these hooks should be able to take a "snapshot of the state" of the HTTP Client. They could take as parameters the counts of active wantConns in idleConnWait and connsPerHostWait queues for each connectMethodKey and the length of idleConn queues for each connectMethodKey. They could also take as parameters the state of all active connections like their TCP connection state( this is an optional feature for which we may have to make external calls). These hooks are useful for the user to understand where their server's bottlenecks or the server they are connecting to.

Note: I have already implemented this during an internship.