Fix: AI Request Logging: token count understated for thinking models (thoughtsTokenCount ignored)#680
Conversation
|
The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message. To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook. |
| $usage = $response['usageMetadata']; | ||
| $input = $usage['promptTokenCount'] ?? null; | ||
| $output = $usage['candidatesTokenCount'] ?? null; | ||
| $output = ( $usage['candidatesTokenCount'] ?? 0 ) + ( $usage['thoughtsTokenCount'] ?? 0 ); |
There was a problem hiding this comment.
So previously we would set null here if candidatesTokenCount wasn't set. While that's an unlikely scenario, now if that isn't set, we return 0 instead. Not sure that will actually cause problems but may be best to keep original behavior here and ensure we return null if neither candidatesTokenCount or thoughtsTokenCount is set
There was a problem hiding this comment.
Just to clarify, should the logic return null only if both candidatesTokenCount and thoughtsTokenCount are unset? If either one has a value, we'd keep that value instead of null?
$output = ( isset( $usage['candidatesTokenCount'] ) || isset( $usage['thoughtsTokenCount'] ) )
? ( $usage['candidatesTokenCount'] ?? 0 ) + ( $usage['thoughtsTokenCount'] ?? 0 )
: null;This preserves the original behavior of returning null when neither is set while still correctly summing whichever tokens are available when at least one is present.
…enCount does not exist instead of 0
What?
Closes #662
Why?
Token count mismatched in AI Request Logging and in Google console logs
How?
Updated Google token extraction in Log Data Extractor to compute output tokens as:
candidatesTokenCount + thoughtsTokenCount
Use of AI Tools
Testing Instructions
Enable the AI Request Logging experiment
Configure the Google provider with a thinking model (e.g. gemini-3-flash-preview)
Trigger a text generation (e.g. via the Title Generation experiment)
Compare the token count in Tools → AI Request Logs with the usage in Google AI Studio → Usage
Screenshots or screencast
Changelog Entry