feat(ai): Add tag to log requests made to cloud models while in hybrid mode#9469
feat(ai): Add tag to log requests made to cloud models while in hybrid mode#9469
Conversation
🦋 Changeset detectedLatest commit: 37f8034 The changes in this PR will be included in the next version bump. This PR includes changesets to release 2 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Vertex AI Mock Responses Check
|
Summary of ChangesHello @hsubox76, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request introduces a telemetry enhancement to better understand the usage patterns of hybrid AI inference modes. By appending a "hybrid" tag to the Highlights
🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
There was a problem hiding this comment.
Code Review
This pull request successfully adds a 'hybrid' tag to the x-goog-api-client header for cloud requests made in hybrid mode, which will be useful for tracking. The implementation is correct and includes a new test case. My review includes a few suggestions to improve code clarity, maintainability, and test coverage.
| chromeAdapter | ||
| ); | ||
|
|
||
| generativeModel._apiSettings.inferenceMode = hybridParams.mode; |
There was a problem hiding this comment.
While this works, modifying the _apiSettings property directly after the object has been constructed is not ideal, especially since the _ prefix suggests it's an internal property. A cleaner approach would be to pass the inferenceMode to the GenerativeModel constructor and let it handle setting this property. This would improve encapsulation and make the initialization logic clearer.
.changeset/nasty-squids-push.md
Outdated
| '@firebase/ai': patch | ||
| --- | ||
|
|
||
| Add tag to log requests made to cloud while in hybrid mode. |
There was a problem hiding this comment.
Food for thought: maybe note that this ias an internal change with "Internal: " or something something?
There was a problem hiding this comment.
Sounds good, helpful to the release operator.
Add "hybrid" tag to x-goog-api-client header to help track which requests are fallbacks from hybrid mode. This does not track on-device requests.
The tag is only applied when
getGenerativeModel()was initialized withInferenceMode.PREFER_IN_CLOUDorInferenceMode.PREFER_ON_DEVICE. IfONLY_ON_DEVICE, no cloud calls will be made anyway. IfONLY_IN_CLOUD, this shouldn't count as hybrid mode.