golang.org/x/oauth2 is a pretty big module with its own dependencies including "google.golang.org/appengine" and "cloud.google.com/go/compute/metadata", but go-github only uses a very small part of it to add a static value to the Authorization header.
The oauth2 dependency can be replaced with a simple RoundTripper that adds the Authorization header.