feat(contact-center): real time transcript#4771
feat(contact-center): real time transcript#4771mkesavan13 merged 9 commits intowebex:task-refactorfrom
Conversation
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 75d17aeffa
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| return; | ||
| } | ||
|
|
||
| if (eventData.type === CC_EVENTS.REAL_TIME_TRANSCRIPTION) { |
There was a problem hiding this comment.
Use nested event type when routing transcript chunks
handleWebsocketMessage checks eventData.type === CC_EVENTS.REAL_TIME_TRANSCRIPTION, but this handler otherwise treats notifications as wrapped payloads where the actionable event is in eventData.data.type (it re-emits eventData.data.type just above). If transcript notifications arrive in that common wrapped format, this condition never matches, so task.emit(CC_EVENTS.REAL_TIME_TRANSCRIPTION, ...) is skipped and task-level listeners never receive live transcript updates.
Useful? React with 👍 / 👎.
| orgSettingsPromise, | ||
| tenantDataPromise, | ||
| urlMappingPromise, | ||
| aiFeatureResourcesPromise, |
There was a problem hiding this comment.
Make AI feature lookup non-blocking during config load
getAgentConfig() now waits on aiFeatureResourcesPromise inside Promise.all, and getAIFeatureResources() throws on any non-200. That means a 404/403/temporary failure on /v2/ai-feature will reject the entire config load and fail agent registration, even though this is an optional transcript feature and the parser already has a disabled fallback when feature data is missing. This introduces a full login outage path tied to an ancillary API.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1f4059d521
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| task.emit(CC_EVENTS.REAL_TIME_TRANSCRIPTION, payload); | ||
| // Backward-compatible alias consumed by existing sample apps. | ||
| task.emit(CC_EVENTS.REAL_TIME_TRANSCRIPTION, payload); |
There was a problem hiding this comment.
Emit transcript alias instead of firing same event twice
In TaskManager.setupTaskListeners, the REAL_TIME_TRANSCRIPTION branch emits CC_EVENTS.REAL_TIME_TRANSCRIPTION twice, so every transcript chunk is delivered twice to normal listeners (for example the sample app appends each chunk twice), while the intended backward-compatible alias is never emitted. The inline comment says the second emit is an alias, and the new unit test also expects 'realtimeTranscription', so this currently causes duplicated processing and misses alias subscribers.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 49f4996927
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| const interactionId = | ||
| eventType === CC_EVENTS.REAL_TIME_TRANSCRIPTION | ||
| ? message.data.data.conversationId | ||
| : message.data.interactionId; | ||
| const task = this.taskCollection[interactionId]; |
There was a problem hiding this comment.
Resolve transcript notifications against stored task IDs
This branch looks up the task by message.data.data.conversationId, but every write to taskCollection in this class uses payload.interactionId. The new HistoricTranscriptsResponse type in src/types.ts:866-870 also models conversationId separately from interactionId, so any deployment where those IDs differ will drop every realtime transcript chunk because eventContext.task is undefined and nothing gets emitted to the task.
Useful? React with 👍 / 👎.
| [CC_EVENTS.AGENT_CONSULT_CONFERENCED]: 'START', | ||
| [CC_EVENTS.AGENT_WRAPUP]: 'STOP', | ||
| [CC_EVENTS.AGENT_CONSULT_ENDED]: 'STOP', | ||
| [CC_EVENTS.PARTICIPANT_LEFT_CONFERENCE]: 'STOP', |
There was a problem hiding this comment.
Don't stop transcription on every participant exit
PARTICIPANT_LEFT_CONFERENCE is not equivalent to the conversation ending: src/services/task/state-machine/guards.ts:145-169 uses that event to downgrade from conference back to CONNECTED when the current agent and customer are still on the main leg. Mapping it to STOP means a consulted agent leaving a three-way call will shut transcription off for the rest of the live customer conversation.
Useful? React with 👍 / 👎.
| private requestRealTimeTranscripts(eventType: string, interactionId: string): void { | ||
| const action = TRANSCRIPT_EVENT_MAP[eventType]; | ||
| if (!action || !this.apiAIAssistant) return; |
There was a problem hiding this comment.
Gate transcript control requests to voice interactions
requestRealTimeTranscripts() only checks the backend event type before calling the AI assistant. The task manager already processes AGENT_CONTACT_ASSIGNED for chat tasks too (see test/unit/spec/services/task/TaskManager.ts:1809-1835), so normal chat/email assignment and wrapup events will now POST GET_TRANSCRIPTS even though there is no audio stream to transcribe. That turns every digital interaction into avoidable AI-assistant errors and log noise.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 731461b0ce
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| export const TRANSCRIPT_EVENT_MAP = { | ||
| [CC_EVENTS.AGENT_CONTACT_ASSIGNED]: 'START', | ||
| [CC_EVENTS.AGENT_CONSULTING]: 'START', | ||
| [CC_EVENTS.AGENT_CONSULT_CONFERENCED]: 'START', |
There was a problem hiding this comment.
Start transcript streaming on AGENT_CONTACT hydration
TRANSCRIPT_EVENT_MAP only sends START for AGENT_CONTACT_ASSIGNED and consult events, but TaskManager.handleAgentContact() explicitly recreates already-active tasks from AGENT_CONTACT during multi-session/reconnect recovery. In that path the call is already live and may never emit another AGENT_CONTACT_ASSIGNED, so after a refresh/relogin the SDK rebuilds the task but never posts GET_TRANSCRIPTS START, and the agent stops receiving live transcript chunks for the rest of the interaction.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
What's the reason for keeping the values same?
| [CC_EVENTS.AGENT_WRAPUP]: 'STOP', | ||
| [CC_EVENTS.AGENT_CONSULT_ENDED]: 'STOP', | ||
| [CC_EVENTS.PARTICIPANT_LEFT_CONFERENCE]: 'STOP', |
There was a problem hiding this comment.
Stop transcript streaming on terminal transfer/end paths
STOP is only mapped for AGENT_WRAPUP, AGENT_CONSULT_ENDED, and PARTICIPANT_LEFT_CONFERENCE, but several voice flows finish without any of those events. contact.ts binds blind/vteam/consult transfer success directly to AGENT_*TRANSFERRED, and the task state machine can move TRANSFER_SUCCESS, TRANSFER_CONFERENCE_SUCCESS, or some CONTACT_ENDED cases straight to WRAPPING_UP/TERMINATED. In those scenarios the agent has left the conversation, but no stop request is ever posted, so the transcript stream can remain active after the task is effectively over.
Useful? React with 👍 / 👎.
| private requestRealTimeTranscripts(eventType: string, interactionId: string): void { | ||
| const action = TRANSCRIPT_EVENT_MAP[eventType]; | ||
| if (!action || !this.apiAIAssistant) return; |
There was a problem hiding this comment.
Skip transcript control when realtimeTranscripts is disabled
requestRealTimeTranscripts() only checks for an ApiAIAssistant instance before posting GET_TRANSCRIPTS. The SDK already loads agentConfig.aiFeature, and ApiAIAssistant.fetchHistoricTranscripts() refuses to run when realtimeTranscripts.enable is false, so this looks like an optional feature. As written, tenants where /v2/ai-feature is empty or has realtimeTranscripts.enable === false will still send transcript control requests on every mapped voice event, creating avoidable AI-assistant errors/noise for every call.
Useful? React with 👍 / 👎.
| export class ApiAIAssistant { | ||
| private webex: WebexSDK; | ||
| private metricsManager: MetricsManager; | ||
| private aiFeature: AIFeatureFlags; |
There was a problem hiding this comment.
Maybe we can keep the Flag suffix
There was a problem hiding this comment.
It will be multiple flags not just real time transcript
There was a problem hiding this comment.
Right, I was talking about the variable name - aiFeature to aiFeatureFlags
| let hostname = ''; | ||
| try { | ||
| hostname = new URL(wccApiGatewayUrl).hostname.toLowerCase(); | ||
| } catch (_error) { | ||
| hostname = wccApiGatewayUrl.toLowerCase(); | ||
| } |
There was a problem hiding this comment.
Can wccApiGatewayUrl be hostname as well as a full url ?
There was a problem hiding this comment.
Then we wouldn't require the try-catch. The code in the catch should be enough and the name wccApiGatewayUrl should also be something like wccApiGatewayHostname
| export const TRANSCRIPT_EVENT_MAP = { | ||
| [CC_EVENTS.AGENT_CONTACT_ASSIGNED]: 'START', | ||
| [CC_EVENTS.AGENT_CONSULTING]: 'START', | ||
| [CC_EVENTS.AGENT_CONSULT_CONFERENCED]: 'START', |
There was a problem hiding this comment.
What's the reason for keeping the values same?
packages/@webex/contact-center/src/services/task/TaskManager.ts
Outdated
Show resolved
Hide resolved
|
|
||
| // Register task listeners | ||
| function registerTaskListeners(task) { | ||
| task.on('REAL_TIME_TRANSCRIPTION', (payload) => { |
There was a problem hiding this comment.
Should we keep the event name consistent with other event names ?
There was a problem hiding this comment.
This event is received differently in terms the payload structure is different, we are not keeping it as part of task object. It is independently present inside cc object so kept the name different in that manner
There was a problem hiding this comment.
Should we add a comment in the code describing why we have kept it different ? will be easy for other devs to understand
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b6f935d0c8
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| }, | ||
| ['operational'] | ||
| ); | ||
| if (error instanceof Error && error.message === AI_ASSISTANT_ERRORS.BASE_URL_NOT_AVAILABLE) { |
There was a problem hiding this comment.
Remove undeclared error enum from transcript fetch catch
fetchHistoricTranscripts() references AI_ASSISTANT_ERRORS.BASE_URL_NOT_AVAILABLE, but AI_ASSISTANT_ERRORS is not defined or imported in this module. Any failure that enters this catch block (for example a network error or base-URL resolution failure) will throw a new ReferenceError at this line, masking the real cause and bypassing the existing getErrorDetails(...) error normalization path.
Useful? React with 👍 / 👎.
|
|
||
| this.apiAIAssistant | ||
| .sendEvent( | ||
| this.agentId, |
There was a problem hiding this comment.
Guard transcript requests until agent ID is initialized
requestRealTimeTranscripts() sends transcript control events with this.agentId but never verifies that it has been set. TaskManager starts consuming websocket events immediately, while cc.connectWebsocket() sets taskManager.setAgentId(...) only after awaiting getAgentConfig(...); during that window (for example reconnect/hydration with active interactions), sendEvent(...) can be called with an undefined agentId, causing failed START/STOP requests and missed transcript streaming.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
webex-js-sdk/packages/@webex/contact-center/src/types.ts
Lines 43 to 48 in cf3e799
as const to AI assistant event constant objects
AIAssistantEventType (and AIAssistantEventName below) are declared as plain objects, so Enum<typeof ...> widens to string instead of the intended literal union. That means ApiAIAssistant.sendEvent() now type-checks arbitrary event types/names, and typos are only caught at runtime when the backend rejects the request. Marking these objects as const preserves the compile-time contract for this new public API.
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
Fixed the test issue and address the remaining comments |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f354f57549
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| const interactionId = | ||
| eventType === CC_EVENTS.REAL_TIME_TRANSCRIPTION | ||
| ? message.data.data.conversationId | ||
| : message.data.interactionId; |
There was a problem hiding this comment.
Guard interaction lookup when data is missing
prepareEventContext now accepts top-level event types via message.data?.type || message.type, but it immediately dereferences message.data to compute interactionId. If a websocket notification arrives as a top-level event without a data object (for example a bare {"type":"Welcome"} payload), this will throw at runtime and abort message handling for that event path. Add a null check before reading message.data.interactionId / message.data.data.conversationId so non-task top-level events are safely ignored.
Useful? React with 👍 / 👎.
COMPLETES #https://jira-eng-sjc12.cisco.com/jira/browse/WXCC-13637
https://jira-eng-sjc12.cisco.com/jira/browse/WXCC-11818
This pull request addresses
Real Time Transcription Implementation
by making the following changes
Change Type
The following scenarios were tested
Vidcast: https://app.vidcast.io/share/c2d427f6-2b17-40ea-8eda-7ffe73e58483?playerMode=vidcast
Network tab share was not visible in the recording so attached screenshot for that.
The GAI Coding Policy And Copyright Annotation Best Practices
I certified that
Make sure to have followed the contributing guidelines before submitting.