Skip to content

feat(contact-center): real time transcript#4794

Merged
mkesavan13 merged 9 commits intowebex:nextfrom
Kesari3008:Real-Time-Transcript
Mar 26, 2026
Merged

feat(contact-center): real time transcript#4794
mkesavan13 merged 9 commits intowebex:nextfrom
Kesari3008:Real-Time-Transcript

Conversation

@Kesari3008
Copy link
Copy Markdown
Contributor

@Kesari3008 Kesari3008 commented Mar 20, 2026

COMPLETES

This pull request addresses

by making the following changes

Change Type

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation update
  • Tooling change
  • Internal code refactor

The following scenarios were tested

Screen Recording: https://github.com/user-attachments/assets/eabb7ea4-3fd6-451d-8f4b-27302db0ed3b
Vidcast: https://app.vidcast.io/share/c2d427f6-2b17-40ea-8eda-7ffe73e58483?playerMode=vidcast
Network tab share was not visible in the recording so attached screenshot for that.

Screenshot 2026-03-27 at 12 02 16 PM Screenshot 2026-03-27 at 12 02 24 PM

The GAI Coding Policy And Copyright Annotation Best Practices

  • GAI was not used (or, no additional notation is required)
  • Code was generated entirely by GAI
  • GAI was used to create a draft that was subsequently customized or modified
  • Coder created a draft manually that was non-substantively modified by GAI (e.g., refactoring was performed by GAI on manually written code)
  • Tool used for AI assistance (GitHub Copilot / Other - specify)
    • Github Copilot
    • Other - Please Specify
  • This PR is related to
    • Feature
    • Defect fix
    • Tech Debt
    • Automation

I certified that

  • I have read and followed contributing guidelines
  • I discussed changes with code owners prior to submitting this pull request
  • I have not skipped any automated checks
  • All existing and new tests passed
  • I have updated the documentation accordingly

Make sure to have followed the contributing guidelines before submitting.

@Kesari3008 Kesari3008 requested a review from a team as a code owner March 20, 2026 13:18
@Kesari3008 Kesari3008 added the validated If the pull request is validated for automation. label Mar 20, 2026
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3e288ec342

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +766 to +770
private requestRealTimeTranscripts(eventType: string, interactionId: string): void {
const action = TRANSCRIPT_EVENT_MAP[eventType];
if (!action || !this.apiAIAssistant) return;

this.apiAIAssistant
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Invoke transcript control events from websocket handling

requestRealTimeTranscripts() is never called from registerTaskListeners() (repo-wide search only finds this definition), so none of the mapped AQM events actually call ApiAIAssistant.sendEvent(). In normal voice or consult flows the SDK never sends the GET_TRANSCRIPTS START/STOP handshake, which means realtime transcription streaming is never requested for any task.

Useful? React with 👍 / 👎.

Comment on lines +170 to +171
if (!this.aiFeature?.realtimeTranscripts?.enable) {
throw new Error('REAL_TIME_TRANSCRIPTION_NOT_ENABLED');
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Propagate AI feature flags before gating history fetch

fetchHistoricTranscripts() now hard-fails when this.aiFeature is unset, but the new ApiAIAssistant instance created in cc.ts is never given the Profile.aiFeature value loaded during cc.register(); repo search only finds the setter definition. After a standard registration flow, cc.apiAIAssistant.fetchHistoricTranscripts(...) will therefore throw REAL_TIME_TRANSCRIPTION_NOT_ENABLED even for orgs that have realtime transcripts enabled, unless the app discovers and calls the internal setter itself.

Useful? React with 👍 / 👎.

Comment on lines 109 to +110
urlMappingPromise,
aiFeatureFlagsPromise,
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Keep AI feature lookup from blocking agent registration

Including aiFeatureFlagsPromise in this Promise.all() makes /v2/ai-feature a hard dependency of getAgentConfig(), even though parseAgentConfigs() treats aiFeature as optional. In tenants or regions where that endpoint is not rolled out yet, or returns 403/404 for orgs without the entitlement, cc.register() will now reject before the agent can log in, which is a broader regression than just disabling the new transcript feature.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 8c8e9f687c

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

if (Object.values(CC_TASK_EVENTS).includes(payload.data.type || payload.type)) {
task =
this.taskCollection[payload.data?.interactionId] ||
this.taskCollection[payload.data?.data?.conversationId];
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Resolve live transcript events against interaction IDs

When the websocket sends REAL_TIME_TRANSCRIPTION payloads keyed by conversationId (which this new branch explicitly anticipates), task will still be undefined because taskCollection is written everywhere else by interactionId. In that case the if (task) block is skipped and no transcript event is emitted to the app, so live captions disappear on calls where conversation and interaction IDs differ.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In that case, shouldn't we be mapping the conversation against the right interaction ID? What are the cases where we can expect conversationId and interactionId to be different?

Comment on lines +92 to +93
[CC_EVENTS.AGENT_CONSULT_ENDED]: 'STOP',
[CC_EVENTS.PARTICIPANT_LEFT_CONFERENCE]: 'STOP',
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Keep transcripts running through consult and conference downgrades

Both AGENT_CONSULT_ENDED and PARTICIPANT_LEFT_CONFERENCE are non-terminal in TaskManager: the initiating agent keeps the task after consult end, and conference leave only removes secondary legs. Mapping either event to STOP will therefore send GET_TRANSCRIPTS STOP while the main customer interaction is still active, cutting off live transcription during normal consult/conference flows.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 40773f6399

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +577 to +578
if (TRANSCRIPT_EVENT_MAP[payload.data.type] && transcriptInteractionId) {
this.requestRealTimeTranscripts(payload.data.type, transcriptInteractionId);
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Guard transcript control requests to telephony tasks

requestRealTimeTranscripts() now runs for every mapped task event without checking the channel. TaskManager also handles chat/email tasks in this same class, and those tasks do receive AGENT_CONTACT_ASSIGNED/wrap-up lifecycle events, so accepting or ending a digital interaction will now call ApiAIAssistant.sendEvent(GET_TRANSCRIPTS, ...) with a non-voice interaction ID. In tenants where the AI endpoint rejects non-telephony interactions, that turns normal digital-task flows into repeated backend failures and noisy error metrics.

Useful? React with 👍 / 👎.

Comment on lines +187 to +188
const aiFeature: AIFeatureFlags | undefined =
aiFeatureFlags?.data && aiFeatureFlags.data.length > 0 ? aiFeatureFlags.data[0] : undefined;
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Avoid choosing an arbitrary AI feature row

/v2/ai-feature is fetched as a paginated list, but parseAgentConfigs() always stores aiFeatureFlags.data[0] as the org's feature state. If the API returns more than one resource, the SDK's behavior now depends on backend ordering: a disabled or stale first row will make fetchHistoricTranscripts() reject even when another returned row enables realtime transcripts, and the opposite ordering can expose the wrong feature set after registration.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor

@mkesavan13 mkesavan13 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't reviewed the tests 100% but there are few questions and some change requests.

Kindly address those. Also attach a vidcast of the testing from PR preview or localhost of the latest code changes.

self?: string;
};
};
data: AIFeatureFlags[];
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this an array? Is it intended to have only one instance of AIFeatureFlags though?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is how we receive it from the backend so need to read the data in the same manner

if (Object.values(CC_TASK_EVENTS).includes(payload.data.type || payload.type)) {
task =
this.taskCollection[payload.data?.interactionId] ||
this.taskCollection[payload.data?.data?.conversationId];
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In that case, shouldn't we be mapping the conversation against the right interaction ID? What are the cases where we can expect conversationId and interactionId to be different?

Comment on lines 566 to 569
if (payload.type === CC_EVENTS.REAL_TIME_TRANSCRIPTION) {
task.emit(payload.type, payload.data.data);
}
task.emit(payload.data.type, payload.data);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we emit only one of these events rather? Seems redundant

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes I can optimize this

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Once again, if the CC_EVENTS.REAL_TIME_TRANSCRIPTION is what we receive in payload.type and we emit an event for it, shouldn't we skip sending the event for payload.data.type ?

Comment on lines +155 to +158
public async fetchHistoricTranscripts(
agentId: string,
interactionId: string
): Promise<HistoricTranscriptsResponse> {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would this include the IVR transcripts? Have we tried?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have tried this API but it just returns older live transcripts. Whichever interactionId you share. Doesn't include IVR trasncripts.

Comment on lines +56 to +68
const envMap: Record<string, string> = {
'api.intgus1.ciscoccservice.com': 'intgus1',
'api.qaus1.ciscoccservice.com': 'qaus1',
'api.wxcc-us1.cisco.com': 'produs1',
'api.wxcc-eu1.cisco.com': 'prodeu1',
'api.wxcc-eu2.cisco.com': 'prodeu2',
'api.wxcc-anz1.cisco.com': 'prodanz1',
'api.wxcc-ca1.cisco.com': 'prodca1',
'api.wxcc-jp1.cisco.com': 'prodjp1',
'api.wxcc-sg1.cisco.com': 'prodsg1',
'api.wxcc-in1.cisco.com': 'prodin1',
'api.loadus1.cisco.com': 'loadus1',
};
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the SDK has it's own variation of catalog. Like we did for Mobius during CC SDK initial architecture, can't we put these in the catalog by ourselves?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Discussed this during DSM

Comment on lines +834 to +839
GET_TRANSCRIPTS: 'GET_TRANSCRIPTS',
GET_MID_CALL_SUMMARY: 'GET_MID_CALL_SUMMARY',
GET_POST_CALL_SUMMARY: 'GET_POST_CALL_SUMMARY',
MID_CALL_SUMMARY_RESPONSE: 'MID_CALL_SUMMARY_RESPONSE',
POST_CALL_SUMMARY_RESPONSE: 'POST_CALL_SUMMARY_RESPONSE',
SUGGESTED_RESPONSES_DIGITAL: 'SUGGESTED_RESPONSES_DIGITAL',
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this mean that the Summaries will be available when the API is ready?

What else would be pending to bring those features in?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just the API additions would be pending. These also need to invoke the same API: sendEvent with different event name and action and they need to be added as part of APIAIAssistant itself

@Kesari3008 Kesari3008 changed the title Real time transcript feat(contact-center): real time transcript Mar 23, 2026
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c192456891

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

if (Object.values(CC_TASK_EVENTS).includes(payload.data.type)) {
task = this.taskCollection[payload.data.interactionId];
if (payload.data?.type || payload.type) {
if (Object.values(CC_TASK_EVENTS).includes(payload.data.type || payload.type)) {
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Handle top-level websocket events without data

This branch now runs when either payload.data?.type or top-level payload.type is present, but it immediately dereferences payload.data.type without null checks. The websocket layer emits top-level events like Welcome/AGENT_MULTI_LOGIN without nested data.type, so this can throw a TypeError and break message handling during connection/registration.

Useful? React with 👍 / 👎.

if (
!action ||
!this.apiAIAssistant ||
this.apiAIAssistant.aiFeature?.realtimeTranscripts?.enable === false
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Require explicit AI feature enablement before sending events

The transcript guard only stops when realtimeTranscripts.enable is explicitly false; if feature flags are absent (for example when /v2/ai-feature returns an empty list and aiFeature is unset), execution still sends GET_TRANSCRIPTS events. That means non-entitled orgs can generate repeated failing backend calls and noisy operational metrics on normal task lifecycle events.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b21c1d0e62

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".


this.apiAIAssistant
.sendEvent(
this.agentId,
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Block transcript sends until agentId is available

requestRealTimeTranscripts() forwards this.agentId directly to sendEvent(), but task listeners are registered in TaskManager before ContactCenter.connectWebsocket() finishes getAgentConfig() and calls taskManager.setAgentId(...). If a mapped task event arrives during that registration window (for example while hydrating an already-active interaction), the SDK sends transcript control with an undefined agent ID, so the backend request fails and transcript START/STOP is dropped for that task.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor

@mkesavan13 mkesavan13 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving with one followup comment.

Please attach the VidCast before merging.

Comment on lines 566 to 569
if (payload.type === CC_EVENTS.REAL_TIME_TRANSCRIPTION) {
task.emit(payload.type, payload.data.data);
}
task.emit(payload.data.type, payload.data);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Once again, if the CC_EVENTS.REAL_TIME_TRANSCRIPTION is what we receive in payload.type and we emit an event for it, shouldn't we skip sending the event for payload.data.type ?

@Kesari3008
Copy link
Copy Markdown
Contributor Author

@mkesavan13 Emission for payload.data.type is an existing code, was already happening.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: bd05b1c5ab

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

}
if (task) {
task.emit(payload.data.type, payload.data);
const eventType = payload.type || payload.data.type;
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Emit task-specific event type before wrapper type

This now prefers the websocket envelope payload.type when re-emitting task events, but routing task messages carry a top-level wrapper type (typically RoutingMessage) while the actionable event is in payload.data.type. In those normal payloads, task listeners subscribed to backend events (including REAL_TIME_TRANSCRIPTION) will stop firing because they receive RoutingMessage instead of the CC event name.

Useful? React with 👍 / 👎.

@mkesavan13
Copy link
Copy Markdown
Contributor

@mkesavan13 Emission for payload.data.type is an existing code, was already happening.

In which case, on this particular RTT event, would we be emitting two events?

@mkesavan13
Copy link
Copy Markdown
Contributor

@rarajes2 has reviewed the PR against task-refactor branch and approved while I have reviewed and approved this. The delta is smaller and already looked into. So, merging this PR.

@Kesari3008 - Kindly attach the VidCast to the PR when you have it ready.

@mkesavan13 mkesavan13 merged commit 1d1e663 into webex:next Mar 26, 2026
11 checks passed
@github-actions
Copy link
Copy Markdown

🎉 Your changes are now available!
Released in: v3.11.0-next.63
📖 View full changelog →
Packages Updated Version
webex 3.11.0-next.63
@webex/contact-center 3.11.0-next.26

Thank you for your contribution!
🤖 This is an automated message. For questions, please refer to the documentation.

@Kesari3008
Copy link
Copy Markdown
Contributor Author

@rarajes2 has reviewed the PR against task-refactor branch and approved while I have reviewed and approved this. The delta is smaller and already looked into. So, merging this PR.

@Kesari3008 - Kindly attach the VidCast to the PR when you have it ready.

@mkesavan13 Attached the vidcast and screen recording and screenshots

@mkesavan13
Copy link
Copy Markdown
Contributor

@rarajes2 has reviewed the PR against task-refactor branch and approved while I have reviewed and approved this. The delta is smaller and already looked into. So, merging this PR.

@Kesari3008 - Kindly attach the VidCast to the PR when you have it ready.

@mkesavan13 Attached the vidcast and screen recording and screenshots

Thanks @Kesari3008

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

validated If the pull request is validated for automation.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants