Skip to content

Latest commit

 

History

History
216 lines (181 loc) · 8.18 KB

closed-captions-teams-interop-web.md

File metadata and controls

216 lines (181 loc) · 8.18 KB
title description author ms.service ms.subservice ms.topic ms.date ms.author
include file
Web how-to guide for enabling Closed captions during a Teams interop call.
valindrae
azure-communication-services
calling
include
07/21/2023
kpunjabi

Prerequisites

Note

Please note that you will need to have a voice calling app using Azure Communication Services calling SDKs to access the closed captions feature that is described in this guide.

Models

Name Description
CaptionsCallFeature API for Captions
CaptionsCommon Base class for captions
StartCaptionOptions Closed caption options like spoken language
TeamsCaptionHandler Callback definition for handling CaptionsReceivedEventType event
TeamsCaptionsInfo Data structure received for each CaptionsReceivedEventType event

Get closed captions feature

External Identity users

If you're building an application that allows Azure Communication Services users to join a Teams meeting.

let captionsCallFeature: SDK.CaptionsCallFeature = call.feature(SDK.Features.Captions);

Microsoft 365 users

let captionsCallFeature: SDK.CaptionsCallFeature = teamsCall.feature(SDK.Features.Captions);

Get teams captions object

You need to get and cast the Teams Captions object to utilize Teams Captions specific features

let teamsCaptions: SDK.TeamsCaptions;
if (captionsCallFeature.captions.kind === 'TeamsCaptions') {
    teamsCaptions = captionsCallFeature.captions as SDK.TeamsCaptions;
}

Subscribe to listeners

Add a listener to receive captions active/inactive status

const captionsActiveChangedHandler = () => {
    if (teamsCaptions.isCaptionsFeatureActive) {
        /* USER CODE HERE - E.G. RENDER TO DOM */
    }
}
teamsCaptions.on('CaptionsActiveChanged', captionsActiveChangedHandler);

Add a listener for captions data received

Handle the returned TeamsCaptionsInfo data object.

Note: The object contains a resultType prop that indicates whether the data is a partial caption or a finalized version of the caption. ResultType partial indicates live unedited caption, while final indicates a finalized interpreted version of the sentence (i.e includes punctuation and capitalization).

let currentCaptionLanguage : string;
const captionsReceivedHandler : TeamsCaptionsHandler = (data: TeamsCaptionsInfo) => { 
    /** USER CODE HERE - E.G. RENDER TO DOM 
     * data.captionLanguage
     * data.captionText
     * data.resultType
     * data.speaker
     * data.spokenText
     * data.timeStamp
    */
   // Example code:
   // Create a dom element, i.e. div, with id "captionArea" before proceeding with the sample code
    if (!this._currentCaptionLanguage || this._currentCaptionLanguage === data.captionLanguage) {
        let mri: string;
        switch (data.speaker.identifier.kind) {
            case 'communicationUser': { mri = data.speaker.identifier.communicationUserId; break; }
            case 'microsoftTeamsUser': { mri = data.speaker.identifier.microsoftTeamsUserId; break; }
            case 'phoneNumber': { mri = data.speaker.identifier.phoneNumber; break; }
        }
        const outgoingCaption = `prefix${mri.replace(/:/g, '').replace(/-/g, '')}`;

        let captionArea = document.getElementById("captionArea");
        const captionText = `${data.timestamp.toUTCString()}
            ${data.speaker.displayName}: ${data.captionText ?? data.spokenText}`;

        let foundCaptionContainer = captionArea.querySelector(`.${outgoingCaption}[isNotFinal='true']`);
        if (!foundCaptionContainer) {
            let captionContainer = document.createElement('div');
            captionContainer.setAttribute('isNotFinal', 'true');
            captionContainer.style['borderBottom'] = '1px solid';
            captionContainer.style['whiteSpace'] = 'pre-line';
            captionContainer.textContent = captionText;
            captionContainer.classList.add(newClassName);

            captionArea.appendChild(captionContainer);
        } else {
            foundCaptionContainer.textContent = captionText;

            if (captionData.resultType === 'Final') {
                foundCaptionContainer.setAttribute('isNotFinal', 'false');
            }
        }
    }
}; 
teamsCaptions.on('CaptionsReceived', captionsReceivedHandler);

Add a listener to receive spoken language changed status

const spokenLanguageChangedHandler = () => {
    if (teamsCaptions.activeSpokenLanguage !== currentSpokenLanguage) {
        /* USER CODE HERE - E.G. RENDER TO DOM */
    }
}
teamsCaptions.on('SpokenLanguageChanged', spokenLanguageChangedHandler)

Add a listener to receive caption language changed status

const captionLanguageChangedHandler = () => {
    if (teamsCaptions.activeCaptionLanguage !== currentCaptionLanguage) {
        /* USER CODE HERE - E.G. RENDER TO DOM */
    }
}
teamsCaptions.on('CaptionLanguageChanged', captionLanguageChangedHandler)

Start captions

Once you've set up all your listeners, you can now start adding captions.

try {
    await teamsCaptions.startCaptions({ spokenLanguage: 'en-us' });
} catch (e) {
    /* USER ERROR HANDLING CODE HERE */
}

Stop captions

try {
    teamsCaptions.stopCaptions(); 
} catch (e) {
    /* USER ERROR HANDLING CODE HERE */
}

Unsubscribe to listeners

teamsCaptions.off('CaptionsActiveChanged', captionsActiveChangedHandler);
teamsCaptions.off('CaptionsReceived', captionsReceivedHandler);

Spoken language support

Get a list of supported spoken languages

Get a list of supported spoken languages that your users can select from when enabling closed captions. The property returns an array of languages in bcp 47 format.

const spokenLanguages = teamsCaptions.supportedSpokenLanguages;

Set spoken language

Pass a value in from the supported spoken languages array to ensure that the requested language is supported. By default, if contoso provides no language or an unsupported language, the spoken language defaults to 'en-us'.

// bcp 47 formatted language code
const language = 'en-us'; 

// Altneratively, pass a value from the supported spoken languages array
const language = spokenLanguages[0]; 

try {
    teamsCaptions.setSpokenLanguage(language);
} catch (e) {
    /* USER ERROR HANDLING CODE HERE */
}

Caption language support

Get a list of supported caption languages

If your organization has an active Teams premium license, you can allow your users to use translated captions provided by Teams captions. As for users with a Microsoft 365 identity, if the meeting organizer doesn't have an active Teams premium license, captions language check is done against the Microsoft 365 users account.

The property returns an array of two-letter language codes in ISO 639-1 standard.

const captionLanguages = teamsCaptions.supportedCaptionLanguages;

Set caption language

// ISO 639-1 formatted language code
const language = 'en'; 

// Altneratively, pass a value from the supported caption languages array
const language = captionLanguages[0];
try {
    teamsCaptions.setCaptionLanguage(language);
} catch (e) {
    /* USER ERROR HANDLING CODE HERE */
}