feat(contact-center): real time transcript#4794
Conversation
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3e288ec342
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| private requestRealTimeTranscripts(eventType: string, interactionId: string): void { | ||
| const action = TRANSCRIPT_EVENT_MAP[eventType]; | ||
| if (!action || !this.apiAIAssistant) return; | ||
|
|
||
| this.apiAIAssistant |
There was a problem hiding this comment.
Invoke transcript control events from websocket handling
requestRealTimeTranscripts() is never called from registerTaskListeners() (repo-wide search only finds this definition), so none of the mapped AQM events actually call ApiAIAssistant.sendEvent(). In normal voice or consult flows the SDK never sends the GET_TRANSCRIPTS START/STOP handshake, which means realtime transcription streaming is never requested for any task.
Useful? React with 👍 / 👎.
| if (!this.aiFeature?.realtimeTranscripts?.enable) { | ||
| throw new Error('REAL_TIME_TRANSCRIPTION_NOT_ENABLED'); |
There was a problem hiding this comment.
Propagate AI feature flags before gating history fetch
fetchHistoricTranscripts() now hard-fails when this.aiFeature is unset, but the new ApiAIAssistant instance created in cc.ts is never given the Profile.aiFeature value loaded during cc.register(); repo search only finds the setter definition. After a standard registration flow, cc.apiAIAssistant.fetchHistoricTranscripts(...) will therefore throw REAL_TIME_TRANSCRIPTION_NOT_ENABLED even for orgs that have realtime transcripts enabled, unless the app discovers and calls the internal setter itself.
Useful? React with 👍 / 👎.
| urlMappingPromise, | ||
| aiFeatureFlagsPromise, |
There was a problem hiding this comment.
Keep AI feature lookup from blocking agent registration
Including aiFeatureFlagsPromise in this Promise.all() makes /v2/ai-feature a hard dependency of getAgentConfig(), even though parseAgentConfigs() treats aiFeature as optional. In tenants or regions where that endpoint is not rolled out yet, or returns 403/404 for orgs without the entitlement, cc.register() will now reject before the agent can log in, which is a broader regression than just disabling the new transcript feature.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8c8e9f687c
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| if (Object.values(CC_TASK_EVENTS).includes(payload.data.type || payload.type)) { | ||
| task = | ||
| this.taskCollection[payload.data?.interactionId] || | ||
| this.taskCollection[payload.data?.data?.conversationId]; |
There was a problem hiding this comment.
Resolve live transcript events against interaction IDs
When the websocket sends REAL_TIME_TRANSCRIPTION payloads keyed by conversationId (which this new branch explicitly anticipates), task will still be undefined because taskCollection is written everywhere else by interactionId. In that case the if (task) block is skipped and no transcript event is emitted to the app, so live captions disappear on calls where conversation and interaction IDs differ.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
In that case, shouldn't we be mapping the conversation against the right interaction ID? What are the cases where we can expect conversationId and interactionId to be different?
| [CC_EVENTS.AGENT_CONSULT_ENDED]: 'STOP', | ||
| [CC_EVENTS.PARTICIPANT_LEFT_CONFERENCE]: 'STOP', |
There was a problem hiding this comment.
Keep transcripts running through consult and conference downgrades
Both AGENT_CONSULT_ENDED and PARTICIPANT_LEFT_CONFERENCE are non-terminal in TaskManager: the initiating agent keeps the task after consult end, and conference leave only removes secondary legs. Mapping either event to STOP will therefore send GET_TRANSCRIPTS STOP while the main customer interaction is still active, cutting off live transcription during normal consult/conference flows.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 40773f6399
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| if (TRANSCRIPT_EVENT_MAP[payload.data.type] && transcriptInteractionId) { | ||
| this.requestRealTimeTranscripts(payload.data.type, transcriptInteractionId); |
There was a problem hiding this comment.
Guard transcript control requests to telephony tasks
requestRealTimeTranscripts() now runs for every mapped task event without checking the channel. TaskManager also handles chat/email tasks in this same class, and those tasks do receive AGENT_CONTACT_ASSIGNED/wrap-up lifecycle events, so accepting or ending a digital interaction will now call ApiAIAssistant.sendEvent(GET_TRANSCRIPTS, ...) with a non-voice interaction ID. In tenants where the AI endpoint rejects non-telephony interactions, that turns normal digital-task flows into repeated backend failures and noisy error metrics.
Useful? React with 👍 / 👎.
| const aiFeature: AIFeatureFlags | undefined = | ||
| aiFeatureFlags?.data && aiFeatureFlags.data.length > 0 ? aiFeatureFlags.data[0] : undefined; |
There was a problem hiding this comment.
Avoid choosing an arbitrary AI feature row
/v2/ai-feature is fetched as a paginated list, but parseAgentConfigs() always stores aiFeatureFlags.data[0] as the org's feature state. If the API returns more than one resource, the SDK's behavior now depends on backend ordering: a disabled or stale first row will make fetchHistoricTranscripts() reject even when another returned row enables realtime transcripts, and the opposite ordering can expose the wrong feature set after registration.
Useful? React with 👍 / 👎.
mkesavan13
left a comment
There was a problem hiding this comment.
I haven't reviewed the tests 100% but there are few questions and some change requests.
Kindly address those. Also attach a vidcast of the testing from PR preview or localhost of the latest code changes.
| self?: string; | ||
| }; | ||
| }; | ||
| data: AIFeatureFlags[]; |
There was a problem hiding this comment.
Why is this an array? Is it intended to have only one instance of AIFeatureFlags though?
There was a problem hiding this comment.
This is how we receive it from the backend so need to read the data in the same manner
| if (Object.values(CC_TASK_EVENTS).includes(payload.data.type || payload.type)) { | ||
| task = | ||
| this.taskCollection[payload.data?.interactionId] || | ||
| this.taskCollection[payload.data?.data?.conversationId]; |
There was a problem hiding this comment.
In that case, shouldn't we be mapping the conversation against the right interaction ID? What are the cases where we can expect conversationId and interactionId to be different?
| if (payload.type === CC_EVENTS.REAL_TIME_TRANSCRIPTION) { | ||
| task.emit(payload.type, payload.data.data); | ||
| } | ||
| task.emit(payload.data.type, payload.data); |
There was a problem hiding this comment.
Should we emit only one of these events rather? Seems redundant
There was a problem hiding this comment.
Yes I can optimize this
There was a problem hiding this comment.
Once again, if the CC_EVENTS.REAL_TIME_TRANSCRIPTION is what we receive in payload.type and we emit an event for it, shouldn't we skip sending the event for payload.data.type ?
| public async fetchHistoricTranscripts( | ||
| agentId: string, | ||
| interactionId: string | ||
| ): Promise<HistoricTranscriptsResponse> { |
There was a problem hiding this comment.
Would this include the IVR transcripts? Have we tried?
There was a problem hiding this comment.
I have tried this API but it just returns older live transcripts. Whichever interactionId you share. Doesn't include IVR trasncripts.
| const envMap: Record<string, string> = { | ||
| 'api.intgus1.ciscoccservice.com': 'intgus1', | ||
| 'api.qaus1.ciscoccservice.com': 'qaus1', | ||
| 'api.wxcc-us1.cisco.com': 'produs1', | ||
| 'api.wxcc-eu1.cisco.com': 'prodeu1', | ||
| 'api.wxcc-eu2.cisco.com': 'prodeu2', | ||
| 'api.wxcc-anz1.cisco.com': 'prodanz1', | ||
| 'api.wxcc-ca1.cisco.com': 'prodca1', | ||
| 'api.wxcc-jp1.cisco.com': 'prodjp1', | ||
| 'api.wxcc-sg1.cisco.com': 'prodsg1', | ||
| 'api.wxcc-in1.cisco.com': 'prodin1', | ||
| 'api.loadus1.cisco.com': 'loadus1', | ||
| }; |
There was a problem hiding this comment.
I think the SDK has it's own variation of catalog. Like we did for Mobius during CC SDK initial architecture, can't we put these in the catalog by ourselves?
There was a problem hiding this comment.
Discussed this during DSM
| GET_TRANSCRIPTS: 'GET_TRANSCRIPTS', | ||
| GET_MID_CALL_SUMMARY: 'GET_MID_CALL_SUMMARY', | ||
| GET_POST_CALL_SUMMARY: 'GET_POST_CALL_SUMMARY', | ||
| MID_CALL_SUMMARY_RESPONSE: 'MID_CALL_SUMMARY_RESPONSE', | ||
| POST_CALL_SUMMARY_RESPONSE: 'POST_CALL_SUMMARY_RESPONSE', | ||
| SUGGESTED_RESPONSES_DIGITAL: 'SUGGESTED_RESPONSES_DIGITAL', |
There was a problem hiding this comment.
Does this mean that the Summaries will be available when the API is ready?
What else would be pending to bring those features in?
There was a problem hiding this comment.
Just the API additions would be pending. These also need to invoke the same API: sendEvent with different event name and action and they need to be added as part of APIAIAssistant itself
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c192456891
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| if (Object.values(CC_TASK_EVENTS).includes(payload.data.type)) { | ||
| task = this.taskCollection[payload.data.interactionId]; | ||
| if (payload.data?.type || payload.type) { | ||
| if (Object.values(CC_TASK_EVENTS).includes(payload.data.type || payload.type)) { |
There was a problem hiding this comment.
Handle top-level websocket events without
data
This branch now runs when either payload.data?.type or top-level payload.type is present, but it immediately dereferences payload.data.type without null checks. The websocket layer emits top-level events like Welcome/AGENT_MULTI_LOGIN without nested data.type, so this can throw a TypeError and break message handling during connection/registration.
Useful? React with 👍 / 👎.
| if ( | ||
| !action || | ||
| !this.apiAIAssistant || | ||
| this.apiAIAssistant.aiFeature?.realtimeTranscripts?.enable === false |
There was a problem hiding this comment.
Require explicit AI feature enablement before sending events
The transcript guard only stops when realtimeTranscripts.enable is explicitly false; if feature flags are absent (for example when /v2/ai-feature returns an empty list and aiFeature is unset), execution still sends GET_TRANSCRIPTS events. That means non-entitled orgs can generate repeated failing backend calls and noisy operational metrics on normal task lifecycle events.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b21c1d0e62
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
|
|
||
| this.apiAIAssistant | ||
| .sendEvent( | ||
| this.agentId, |
There was a problem hiding this comment.
Block transcript sends until agentId is available
requestRealTimeTranscripts() forwards this.agentId directly to sendEvent(), but task listeners are registered in TaskManager before ContactCenter.connectWebsocket() finishes getAgentConfig() and calls taskManager.setAgentId(...). If a mapped task event arrives during that registration window (for example while hydrating an already-active interaction), the SDK sends transcript control with an undefined agent ID, so the backend request fails and transcript START/STOP is dropped for that task.
Useful? React with 👍 / 👎.
mkesavan13
left a comment
There was a problem hiding this comment.
Approving with one followup comment.
Please attach the VidCast before merging.
| if (payload.type === CC_EVENTS.REAL_TIME_TRANSCRIPTION) { | ||
| task.emit(payload.type, payload.data.data); | ||
| } | ||
| task.emit(payload.data.type, payload.data); |
There was a problem hiding this comment.
Once again, if the CC_EVENTS.REAL_TIME_TRANSCRIPTION is what we receive in payload.type and we emit an event for it, shouldn't we skip sending the event for payload.data.type ?
|
@mkesavan13 Emission for payload.data.type is an existing code, was already happening. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: bd05b1c5ab
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| } | ||
| if (task) { | ||
| task.emit(payload.data.type, payload.data); | ||
| const eventType = payload.type || payload.data.type; |
There was a problem hiding this comment.
Emit task-specific event type before wrapper type
This now prefers the websocket envelope payload.type when re-emitting task events, but routing task messages carry a top-level wrapper type (typically RoutingMessage) while the actionable event is in payload.data.type. In those normal payloads, task listeners subscribed to backend events (including REAL_TIME_TRANSCRIPTION) will stop firing because they receive RoutingMessage instead of the CC event name.
Useful? React with 👍 / 👎.
In which case, on this particular RTT event, would we be emitting two events? |
|
@rarajes2 has reviewed the PR against task-refactor branch and approved while I have reviewed and approved this. The delta is smaller and already looked into. So, merging this PR. @Kesari3008 - Kindly attach the VidCast to the PR when you have it ready. |
Thank you for your contribution! |
@mkesavan13 Attached the vidcast and screen recording and screenshots |
Thanks @Kesari3008 |
COMPLETES
This pull request addresses
by making the following changes
Change Type
The following scenarios were tested
Screen Recording: https://github.com/user-attachments/assets/eabb7ea4-3fd6-451d-8f4b-27302db0ed3b
Vidcast: https://app.vidcast.io/share/c2d427f6-2b17-40ea-8eda-7ffe73e58483?playerMode=vidcast
Network tab share was not visible in the recording so attached screenshot for that.
The GAI Coding Policy And Copyright Annotation Best Practices
I certified that
Make sure to have followed the contributing guidelines before submitting.