Skip to content

feat(contact-center): real time transcript#4771

Merged
mkesavan13 merged 9 commits intowebex:task-refactorfrom
Kesari3008:real-time-transcript
Mar 27, 2026
Merged

feat(contact-center): real time transcript#4771
mkesavan13 merged 9 commits intowebex:task-refactorfrom
Kesari3008:real-time-transcript

Conversation

@Kesari3008
Copy link
Copy Markdown
Contributor

@Kesari3008 Kesari3008 commented Mar 12, 2026

COMPLETES #https://jira-eng-sjc12.cisco.com/jira/browse/WXCC-13637

https://jira-eng-sjc12.cisco.com/jira/browse/WXCC-11818

This pull request addresses

Real Time Transcription Implementation

by making the following changes

Change Type

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation update
  • Tooling change
  • Internal code refactor

The following scenarios were tested

Vidcast: https://app.vidcast.io/share/c2d427f6-2b17-40ea-8eda-7ffe73e58483?playerMode=vidcast
Network tab share was not visible in the recording so attached screenshot for that.

Screenshot 2026-03-27 at 12 02 16 PM Screenshot 2026-03-27 at 12 02 24 PM

The GAI Coding Policy And Copyright Annotation Best Practices

  • GAI was not used (or, no additional notation is required)
  • Code was generated entirely by GAI
  • GAI was used to create a draft that was subsequently customized or modified
  • Coder created a draft manually that was non-substantively modified by GAI (e.g., refactoring was performed by GAI on manually written code)
  • Tool used for AI assistance (GitHub Copilot / Other - specify)
    • Github Copilot
    • Other - Please Specify
  • This PR is related to
    • Feature
    • Defect fix
    • Tech Debt
    • Automation

I certified that

  • I have read and followed contributing guidelines
  • I discussed changes with code owners prior to submitting this pull request
  • I have not skipped any automated checks
  • All existing and new tests passed
  • I have updated the documentation accordingly

Make sure to have followed the contributing guidelines before submitting.

@Kesari3008 Kesari3008 requested a review from a team as a code owner March 12, 2026 19:53
@Kesari3008 Kesari3008 added the validated If the pull request is validated for automation. label Mar 12, 2026
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 75d17aeffa

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

return;
}

if (eventData.type === CC_EVENTS.REAL_TIME_TRANSCRIPTION) {
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Use nested event type when routing transcript chunks

handleWebsocketMessage checks eventData.type === CC_EVENTS.REAL_TIME_TRANSCRIPTION, but this handler otherwise treats notifications as wrapped payloads where the actionable event is in eventData.data.type (it re-emits eventData.data.type just above). If transcript notifications arrive in that common wrapped format, this condition never matches, so task.emit(CC_EVENTS.REAL_TIME_TRANSCRIPTION, ...) is skipped and task-level listeners never receive live transcript updates.

Useful? React with 👍 / 👎.

orgSettingsPromise,
tenantDataPromise,
urlMappingPromise,
aiFeatureResourcesPromise,
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Make AI feature lookup non-blocking during config load

getAgentConfig() now waits on aiFeatureResourcesPromise inside Promise.all, and getAIFeatureResources() throws on any non-200. That means a 404/403/temporary failure on /v2/ai-feature will reject the entire config load and fail agent registration, even though this is an optional transcript feature and the parser already has a disabled fallback when feature data is missing. This introduces a full login outage path tied to an ancillary API.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1f4059d521

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +345 to +347
task.emit(CC_EVENTS.REAL_TIME_TRANSCRIPTION, payload);
// Backward-compatible alias consumed by existing sample apps.
task.emit(CC_EVENTS.REAL_TIME_TRANSCRIPTION, payload);
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Emit transcript alias instead of firing same event twice

In TaskManager.setupTaskListeners, the REAL_TIME_TRANSCRIPTION branch emits CC_EVENTS.REAL_TIME_TRANSCRIPTION twice, so every transcript chunk is delivered twice to normal listeners (for example the sample app appends each chunk twice), while the intended backward-compatible alias is never emitted. The inline comment says the second emit is an alias, and the new unit test also expects 'realtimeTranscription', so this currently causes duplicated processing and misses alias subscribers.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 49f4996927

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +415 to 419
const interactionId =
eventType === CC_EVENTS.REAL_TIME_TRANSCRIPTION
? message.data.data.conversationId
: message.data.interactionId;
const task = this.taskCollection[interactionId];
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Resolve transcript notifications against stored task IDs

This branch looks up the task by message.data.data.conversationId, but every write to taskCollection in this class uses payload.interactionId. The new HistoricTranscriptsResponse type in src/types.ts:866-870 also models conversationId separately from interactionId, so any deployment where those IDs differ will drop every realtime transcript chunk because eventContext.task is undefined and nothing gets emitted to the task.

Useful? React with 👍 / 👎.

[CC_EVENTS.AGENT_CONSULT_CONFERENCED]: 'START',
[CC_EVENTS.AGENT_WRAPUP]: 'STOP',
[CC_EVENTS.AGENT_CONSULT_ENDED]: 'STOP',
[CC_EVENTS.PARTICIPANT_LEFT_CONFERENCE]: 'STOP',
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Don't stop transcription on every participant exit

PARTICIPANT_LEFT_CONFERENCE is not equivalent to the conversation ending: src/services/task/state-machine/guards.ts:145-169 uses that event to downgrade from conference back to CONNECTED when the current agent and customer are still on the main leg. Mapping it to STOP means a consulted agent leaving a three-way call will shut transcription off for the rest of the live customer conversation.

Useful? React with 👍 / 👎.

Comment on lines +718 to +720
private requestRealTimeTranscripts(eventType: string, interactionId: string): void {
const action = TRANSCRIPT_EVENT_MAP[eventType];
if (!action || !this.apiAIAssistant) return;
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Gate transcript control requests to voice interactions

requestRealTimeTranscripts() only checks the backend event type before calling the AI assistant. The task manager already processes AGENT_CONTACT_ASSIGNED for chat tasks too (see test/unit/spec/services/task/TaskManager.ts:1809-1835), so normal chat/email assignment and wrapup events will now POST GET_TRANSCRIPTS even though there is no audio stream to transcribe. That turns every digital interaction into avoidable AI-assistant errors and log noise.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 731461b0ce

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +87 to +90
export const TRANSCRIPT_EVENT_MAP = {
[CC_EVENTS.AGENT_CONTACT_ASSIGNED]: 'START',
[CC_EVENTS.AGENT_CONSULTING]: 'START',
[CC_EVENTS.AGENT_CONSULT_CONFERENCED]: 'START',
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Start transcript streaming on AGENT_CONTACT hydration

TRANSCRIPT_EVENT_MAP only sends START for AGENT_CONTACT_ASSIGNED and consult events, but TaskManager.handleAgentContact() explicitly recreates already-active tasks from AGENT_CONTACT during multi-session/reconnect recovery. In that path the call is already live and may never emit another AGENT_CONTACT_ASSIGNED, so after a refresh/relogin the SDK rebuilds the task but never posts GET_TRANSCRIPTS START, and the agent stops receiving live transcript chunks for the rest of the interaction.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the reason for keeping the values same?

Comment on lines +91 to +93
[CC_EVENTS.AGENT_WRAPUP]: 'STOP',
[CC_EVENTS.AGENT_CONSULT_ENDED]: 'STOP',
[CC_EVENTS.PARTICIPANT_LEFT_CONFERENCE]: 'STOP',
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Stop transcript streaming on terminal transfer/end paths

STOP is only mapped for AGENT_WRAPUP, AGENT_CONSULT_ENDED, and PARTICIPANT_LEFT_CONFERENCE, but several voice flows finish without any of those events. contact.ts binds blind/vteam/consult transfer success directly to AGENT_*TRANSFERRED, and the task state machine can move TRANSFER_SUCCESS, TRANSFER_CONFERENCE_SUCCESS, or some CONTACT_ENDED cases straight to WRAPPING_UP/TERMINATED. In those scenarios the agent has left the conversation, but no stop request is ever posted, so the transcript stream can remain active after the task is effectively over.

Useful? React with 👍 / 👎.

Comment on lines +704 to +706
private requestRealTimeTranscripts(eventType: string, interactionId: string): void {
const action = TRANSCRIPT_EVENT_MAP[eventType];
if (!action || !this.apiAIAssistant) return;
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Skip transcript control when realtimeTranscripts is disabled

requestRealTimeTranscripts() only checks for an ApiAIAssistant instance before posting GET_TRANSCRIPTS. The SDK already loads agentConfig.aiFeature, and ApiAIAssistant.fetchHistoricTranscripts() refuses to run when realtimeTranscripts.enable is false, so this looks like an optional feature. As written, tenants where /v2/ai-feature is empty or has realtimeTranscripts.enable === false will still send transcript control requests on every mapped voice event, creating avoidable AI-assistant errors/noise for every call.

Useful? React with 👍 / 👎.

export class ApiAIAssistant {
private webex: WebexSDK;
private metricsManager: MetricsManager;
private aiFeature: AIFeatureFlags;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we can keep the Flag suffix

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It will be multiple flags not just real time transcript

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, I was talking about the variable name - aiFeature to aiFeatureFlags

Comment on lines +49 to +54
let hostname = '';
try {
hostname = new URL(wccApiGatewayUrl).hostname.toLowerCase();
} catch (_error) {
hostname = wccApiGatewayUrl.toLowerCase();
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can wccApiGatewayUrl be hostname as well as a full url ?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's hostname

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then we wouldn't require the try-catch. The code in the catch should be enough and the name wccApiGatewayUrl should also be something like wccApiGatewayHostname

Comment on lines +87 to +90
export const TRANSCRIPT_EVENT_MAP = {
[CC_EVENTS.AGENT_CONTACT_ASSIGNED]: 'START',
[CC_EVENTS.AGENT_CONSULTING]: 'START',
[CC_EVENTS.AGENT_CONSULT_CONFERENCED]: 'START',
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the reason for keeping the values same?


// Register task listeners
function registerTaskListeners(task) {
task.on('REAL_TIME_TRANSCRIPTION', (payload) => {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we keep the event name consistent with other event names ?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This event is received differently in terms the payload structure is different, we are not keeping it as part of task object. It is independently present inside cc object so kept the name different in that manner

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we add a comment in the code describing why we have kept it different ? will be easy for other devs to understand

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b6f935d0c8

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

},
['operational']
);
if (error instanceof Error && error.message === AI_ASSISTANT_ERRORS.BASE_URL_NOT_AVAILABLE) {
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Remove undeclared error enum from transcript fetch catch

fetchHistoricTranscripts() references AI_ASSISTANT_ERRORS.BASE_URL_NOT_AVAILABLE, but AI_ASSISTANT_ERRORS is not defined or imported in this module. Any failure that enters this catch block (for example a network error or base-URL resolution failure) will throw a new ReferenceError at this line, masking the real cause and bypassing the existing getErrorDetails(...) error normalization path.

Useful? React with 👍 / 👎.


this.apiAIAssistant
.sendEvent(
this.agentId,
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Guard transcript requests until agent ID is initialized

requestRealTimeTranscripts() sends transcript control events with this.agentId but never verifies that it has been set. TaskManager starts consuming websocket events immediately, while cc.connectWebsocket() sets taskManager.setAgentId(...) only after awaiting getAgentConfig(...); during that window (for example reconnect/hydration with active interactions), sendEvent(...) can be called with an undefined agentId, causing failed START/STOP requests and missed transcript streaming.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

* @public
* @example
* function makeRequest(method: HTTP_METHODS) { ... }
* @ignore
*/
export type HTTP_METHODS = Enum<typeof HTTP_METHODS>;

P2 Badge Add as const to AI assistant event constant objects

AIAssistantEventType (and AIAssistantEventName below) are declared as plain objects, so Enum<typeof ...> widens to string instead of the intended literal union. That means ApiAIAssistant.sendEvent() now type-checks arbitrary event types/names, and typos are only caught at runtime when the backend rejects the request. Marking these objects as const preserves the compile-time contract for this new public API.

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Copy link
Copy Markdown
Contributor

@rarajes2 rarajes2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some questions and minor comments added. UTs are failing for some token issue, please check.

@Kesari3008
Copy link
Copy Markdown
Contributor Author

Some questions and minor comments added. UTs are failing for some token issue, please check.

Fixed the test issue and address the remaining comments

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f354f57549

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +398 to +401
const interactionId =
eventType === CC_EVENTS.REAL_TIME_TRANSCRIPTION
? message.data.data.conversationId
: message.data.interactionId;
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Guard interaction lookup when data is missing

prepareEventContext now accepts top-level event types via message.data?.type || message.type, but it immediately dereferences message.data to compute interactionId. If a websocket notification arrives as a top-level event without a data object (for example a bare {"type":"Welcome"} payload), this will throw at runtime and abort message handling for that event path. Add a null check before reading message.data.interactionId / message.data.data.conversationId so non-task top-level events are safely ignored.

Useful? React with 👍 / 👎.

@mkesavan13 mkesavan13 changed the title feat(ccsdk): real time transcript implementation feat(contact-center): real time transcript Mar 26, 2026
@mkesavan13 mkesavan13 merged commit 2683141 into webex:task-refactor Mar 27, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

validated If the pull request is validated for automation.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants