Problem:
@toolmantim's account was banned (related thread)

Link: https://sourcegraph.com/users/toolmantim/settings/quota
ID: 59727 / 018d5660-4043-745d-bc0d-ca8e69970468

Findings:

• In the past week, 100% of Tim’s chat requests were flagged where he used a Google model, and 0 other requests were flagged (58 flagged out of 77 requests).
• All flagged requests had both the high_max_tokens_to_sample and blocked_phrase flags, and no other.

Based on this code, not having a tokenizer avoids the high_prompt_token_count flag, and indeed, no requests got that one. Only the high_max_tokens_to_sample and blocked_phrase are applied, but those are enough for the request to be flagged all the same.

Redash queries that helped me:

• Cody Gateway abuse flags: Tim's requests from the past week
• Cody Gateway abuse flags: all users, grouped by model, past 8 hours
• Cody Gateway abuse flags: all users, grouped by model, past 8 hours, with user counts

Solution ideas:

1. If we wanted to just do no flagging for Google requests, we could do

func (g *GoogleHandlerMethods) shouldFlagRequest(_ context.Context, _ log.Logger, _ googleRequest) (*flaggingResult, error) {
    // This entirely disables flagging for Google requests.
	result := flaggingResult{
		shouldBlock:       false,
		blockedPhrase:     nil,
		reasons:           nil,
		promptPrefix:      "",
		maxTokensToSample: 0,
		promptTokenCount:  0,
	}
	return &result, nil
}

2. Investigate why this happens, and go with a softer measure.