Contact Support

Customers who viewed this article also viewed

banner icon

Identify Changes in NetScaler build files with

File Integrity Monitoring

Learn More Watch Video
CTX253754 {{tooltipText}}

Troubleshooting HDX Optimization for Microsoft Teams

Applicable Products

  • Citrix Workspace App
  • Citrix Virtual Apps and Desktops

Objective

This is a Live article - updated frequently with the latest info and known issues

-------------------------------------------------------------------------------------------------------------------------------------------------

Recommended Teams version : latest version. Update history for Microsoft Teams App Version here.

The following article provides guidelines for troubleshooting the HDX optimization for Microsoft Teams in Citrix Virtual Apps and Desktops.

Citrix strongly recommends the latest current release Workspace app version. Most of the new features require an upgrade of the Workspace app, hence the Current Release model is the preferred delivery mechanism.

-------------------------------------------------------------------------------------------------------------------------------------------------
Live updates on known issues

  • 12/19/23 Citrix has published a support article covering Teams 2.1 support including APM and App Layering https://support.citrix.com/article/CTX585013/microsoft-teams-21-supported-for-vdidaas .
  • 12/13/23   Microsoft has released Teams 2.1 for VDI in early December 2023.  This version supports Optimization with the Citrix HDX Optimization engine (WebRTC).   See the following Microsoft support article for more information https://learn.microsoft.com/en-us/microsoftteams/new-teams-vdi-requirements-deploy.
  • 9/25/23 Mac users running CWA 2308 who have the Secondary Ringer configured may occasionally encounter an issue where sounds for messages or ringtones may play continuously without stopping. To stop the sound from playing the user must disconnect the desktop session. To completely prevent this issue from happening users should not use Secondary Ringer in 2308 Mac CWA.
  • 9/12/2023 On January 31, 2024, Microsoft will retire the Microsoft Teams support for Single-window UI when using VDI Teams optimization and support only the Multi-Window experience. Microsoft gave notice of this deprecation on 9/8/2023 in the M365s Admin Center (Post ID: MC674419) Customers must upgrade their VDA and CWA to supported versions in order to retain Optimized video calling and screen sharing.

    Public details about the Multi-Window feature can be found in the Tech Community article: New Meeting and Calling Experience in Microsoft Teams!

  • 9/8/2023 Microsoft has enabled Secondary Ringtone 100% for CWA release 2307.1 and greater. 
  • 8/20/2023   Microsoft has enabled Simulcast 100% for CWA release 2305 and greater.
  • 4/19/2023 - Microsoft has reverted changes it made to the Teams software which we jointly believe have mitigated the frequency of occurrence of the "floating" video or black square issue.  If this is still happening  please try restarting the Teams client.  If it is still occurring please file a support ticket with Citrix.
  • 4/11/2023 – Several customers have reported problems with video in calls where the video is “floating” on the desktop window and / or may be a black square.  They are also reporting crashes in HDXRTCEngine.exe associated with this occurring.  Citrix and Microsoft are working together to get to the root of this issue and fix it ASAP. 
  • 4/4/2023 - Desktop Sharing in older VDAs (1912 CU5 or below) that do not support multi-window mode may no longer allow sharing of more than one Desktop due to changes in Teams support for Single-Window mode.   Citrix recommends upgrading to at least 1912 CU6+ to get or retain this functionality.
  • 4/4/2023 - Citrix has received reports that some desktop applications are not sharable in Optimized Teams.  If you have any Windows applications that are not shareable, please contact Citrix Support to troubleshoot this issue. 
  • 4/3/2023 - The issue with CWA on Mac documented here impacts Screen Sharing and Video Conferencing on the Mac platform.  Please follow the instructions to upgrade your CWA to the fixed version to regain functionality.
  • 3/16/2023 - Citrix has received new reports of HDXRTCMediaEngine.exe crashes during meetings with video and screen sharing enabled which we are investigating.  We suspect that this is related to the Intel Driver previously mentioned in this article.
  • 3/16/2023 - Citrix has fixed a problem in the Javascript “Shim” that runs inside of Teams where some Point-to-Point calls were failing to connect sporadically.  This bug is fixed in the latest version of Microsoft Teams (Shim version 1.13.0.2).  To get the latest Shim update you must log out and log back into Teams and it will be automatically downloaded.
  • 1/26/2023 - The issue with the Intel graphics device driver has been fixed.  Please contact your device manufacturer to download the latest version of the driver. See HDX Microsoft Teams Optimization Crash or Video Freeze on newer Intel-based clients for more information.
  • 1/26/2023 - Background blurring and image effects are available in release 2301 for Mac.  This requires MultiWindow as a prerequisite  which is available in VDA 2112 or higher and Teams 1.5.00.11865 or higher.
  • 1/26/2023 - Background blurring is available in release 2212 for Linux.  This requires MultiWindow as a prerequisite  which is available in VDA 2112 or higher and Teams 1.5.00.11865 or higher. Image effects will be delivered in a subsequent Linux release.
  • 11/18/2022- Citrix is aware of an issue with graphics device drivers for Windows endpoints that have the Intel I5 1245U 1.6GHZ processor using the UHD Graphics 770 adapter. Endpoints using this driver can experience call freezes or disconnections in meetings or calls. See HDX Microsoft Teams Optimization Crash or Video Freeze on newer Intel-based clients for more information.
  • 8/31/2022: Background Blurring and effects (default image templates) are now enabled in Ring 4 (GA) with CWA for Windows 2207. This requires MultiWindow as a pre-requisite (VDA 2112 or higher, and Teams 1.5.00.11865 or higher). 
  • 7/27/2022: CWA 2207 for Windows is GA. This version will support Background Blurring and Effects once Microsoft rolls out a feature flag (check for updates here). Please note that MultiWindow is a pre-requisite (VDA upgrade to 2112 or higher) for Background Blurring and Effects.
  • 6/17/2022: critical hotfix for 1912 LTSR, 2203 LTSR and CR VDAs (single session or multisession). In these VDAs, WebSocketService.exe can crash and cause Teams call failures ("Still connecting to remote devices. Calling is not available yet"). See CTX460173. [Internal reference: CVADHELP-17758]
  • 6/7/2022: Simultaneous outgoing video and outgoing screensharing are now enabled in Ring 4 (GA). CWA 2109 or higher for Windows/Linux/Mac required. No VDA dependency.
  • 6/6/2022: Background blurring and effects tech preview is now available with CWA for Windows 2206 EAR. No VDA or Teams versions dependency. The feature is controlled via client-side regkeys.
  • 6/3/2022: MultiWindow (a.k.a pop-out chat, a.k.a new meeting experience) is now enabled in Ring 4 (GA). CWA for Windows 2112.1 or higher, CWA for Linux/Mac 2203 or higher, VDA 2112 or higher and Teams 1.5.00.11865 or higher required. A Teams sign out/sign in and restart will be required.
  • 5/25/2022: Simultaneous outgoing video and outgoing screensharing are now enabled in Teams Public Preview Channel (a.k.a Ring 3.6, check here for how to enable your account). This feature requires CWA for Windows/Linux/Mac 2109 or higher. Thanks to this feature, users can share their webcam and screen at the same time. In the past, the webcam would have been disabled the moment a user also started to screenshare.
  • 5/9/2022: Give and Take Control of shared content is now enabled in Ring 4 (G). CWA 2112.1 for Windows (or higher) and CWA 2203 for MAC (or higher) required.
  • 5/2/2022: Live Captions are now enabled in Ring 4 (GA). CWA 2109 for Windows/Linux/Mac or higher required.
  • 4/11/2022: WebRTC 1.0 is now enabled in GCC. See note from 3/7/2022.
  • 3/22/2022: VDA 2103 can cause Teams users to fail to connect to a meeting (HDX-34394). Issue is fixed in VDA 2106 or higher.
  • 3/7/2022: Gallery View performance improvements (WebRTC 1.0) are now enabled in Ring4 (GA). CWA 2109 or higher can now provide better user experience in conference calls with Gallery View (ability to switch resolutions more gracefully by the sender, which in itself smoothens bandwidth estimation drops and video freezes on the receiver side; and also faster ramp-up time to a desired bitrate/resolution at the beginning of the video call or after packet loss).
  • 2/22/2022: Dynamic e911 is now available in Ring4 (GA). See edocs 'Microsoft Phone System' section for more details
  • 2/10/2022: App Sharing is now available in Ring4 (GA). See edocs 'App Sharing' section for more details
  • 2/10/2022: The Citrix API (colloquially known as shim) was auto-updated in Teams today to version 1.8.0.12. No action required by the IT Admins in either VDA, CWA or Teams client msi versions. Users might need to sign out/sign back in. This release introduced numerous fixes:
    • Self-preview video disappears after changing to different audio device in Teams UI/Settings/Devices [CVADHELP-17416]
    • Can't put PSTN call on hold for a second time [HDX-24447] [MS174365]
    • Toggling “Audio off” --> "Audio on" in Teams UI when joining a meeting does not work [HDX-35200]
    • “Audio off” doesn’t work when user is invited to join a meeting, so user still hears incoming audio [HDX-29206]
    • Performance improvements for enumerateDevice issues that could spike endpoint’s CPU [HDX-36752] [HDX-36494]
    • UI Performance improvements during video calls (sluggish Teams UI) [CVADHELP-18844] [HDX-37058]
  • 11/05/2021 : VDA 1912 CU4 released, including critical fixes for CtxSvcHost.exe (Citrix HDX Teams Redirection Service) exiting unexpectedly because of a faulting CtxTeamsSvc.dll module, and also for CtxHdxWebSocketService.exe (HTML5 Video Redirection Service) exiting unexpectedly. As a result, Microsoft Teams might fail to load optimized ("Citrix HDX Not Connected") or optimized calls drop suddenly while in progress. These fixes are also available in VDA 2106 or higher [CVADHELP-16918] [CVADHELP-17341] [CVADHELP-17146] [CVADHELP-17505]
  • 11/1/2021 : Citrix and Microsoft strongly recommend to update to a Teams version higher than 1.3.00.28778. Any version equal or lower will not be able to run new features for VDI. Public Microsoft Teams roadmap is available here.
  • 9/29/2021: Workspace app 2109 for Windows, Linux and Mac is now GA. These releases include improvements in conference call performance, but the availability of this feature depends on an upcoming Teams update.
  • 9/24/2021 : for Remote PC scenarios, Citrix strongly recommends Teams version 1.4.00.22472 or higher, as this version includes improved detection logic for HDX or local sessions to the client machine. Older Teams versions might disable certain features/UI elements otherwise for local sessions. Check edocs for further details.
  • 9//9/2021: when optimized, drag and drop of files in the VDA into Teams chat fails and Teams might crash. Microsoft working on a fix (Microsoft case ref. 267297731 and 240731800).
  • 6/25/2021: Call Quality Dashboard (CQD) in Teams Admin Center is now pulling up HDX optimized endpoint data
  • 5/20/2021: New blog on Single Server Scalability considerations and testing. See here.
  • 5/12/2021: VDA 1912 CU3 released, including important fix for WebSocketAgent.exe (Citrix HDX HTML5 Video Redirection Agent Service) stopping intermittently. As a result, incoming calls are not displayed in Microsoft Teams toaster (called party does not receive any notifications). [CVADHELP-15611]
  • 5/11/2021: Large Gallery View 7x7 and Together Mode are now supported. Please sign out and sign back into Teams, no CWA nor VDA upgrade required. [Please note that Gallery and Large Gallery are very different - see edocs, The former is a 2x2 layout where the endpoint decodes 4 individual video streams, with maximum quality 360p. The latter is a 7x7 layout (i.e up to 49 participants) where the endpoint decodes a single composite video feed created by the Conference Server, with maximum quality 720p)
  • 12/8/2020: Microsoft introduced a new public tech preview methodology for Teams. Citrix recommends you enroll.
  • 11/15/2020: VDA 1912 LTSR CU2 released, including important fixes for bugs causing Teams to load unoptimized . See Release notes (HDX-25074)
  • In some scenarios, even though "Citrix HDX Not Connected" is displayed, attempting to make a call will trigger Teams to connect to the Citrix VDA services and optimization will succeed. This is caused by a delay in the Teams UI refresh.
  • If Teams is configured to Auto-start, the issue in the bullet above might also manifest. We recommend disabling auto-start by deleting the Teams regkeys in HKLM\Software\Microsoft\Windows\CurrentVersion\Run, HKLM\Software\WOW6432Node\Microsoft\Windows\CurrentVersion\Run and HKCU\Software\Microsoft\Windows\CurrentVersion\Run
  • Note that currently it is not possible to disable Auto-start even if using the command line flag OPTIONS="noAutoStart=true" with the machine-wide MSI and the ALLUSER=1 property. Deleting the regkey described in the bullet above should fix this.
  • If the Virtual Desktop does not have a GPU/vGPU, we recommend "Disable GPU hardware acceleration" in Teams/Settings in order to improve performance and scalability, otherwise Teams will use the VDA's CPU to render UI elements/animations. This setting ("disableGpu":true") is stored in %Appdata%\Microsoft\Teams inside desktop-config.json, so you could also use a logon script to edit that file and set the value to true.
  • In some scenarios, when the VM's NTP time sync server is updated via GPOs, this can create a clock drift when compared to the time when the Citrix HTML5 Video Redirection service is started. This drift then translates into invalid/expired Citrix certificates and Teams will fail to load in optimized mode. We recommend changing the HTML5 Video Redirection Service ("CtxHdxWebSocketService") Startup Type to "Automatic (Delayed Start)" instead of the default "Automatic". Citrix fixed this problem in CVAD 2006 VDA. [HDX-24350]
  • When installing Teams in a machine that has a VDA (either VM or Remote PC), Teams will disable background effects.
  • Background blurring and live events are known limitations in VDI. Please contact Microsoft for timelines and more information.

-------------------------------------------------------------------------------------------------------------------------------------------------


Instructions



Please note that, generally, fully quitting Teams (Systray icon / right click / quit) and restarting it clears all the components involved and can fix whatever issue you were experiencing.

Also, keep in mind that Teams feature set is different when comparing optimized vs non-optimized. When Teams is not optimized, it runs on the VDA with its native media engine and all the features are supported (although multimedia rendering happens on the VDA, incurring in a CPU tax)


[Warning: If your VDA is 7.15 (any CU) and you still want to use Teams in unoptimized mode (the only mode supported in these VDA versions), scroll here. Teams HDX optimization will not be backported to the LTSR 7.15].

Step-by-step:

Microsoft Teams optimization mechanisms can be divided into 5 main areas

1. Media Engine instantiation (HdxRtcEngine.exe)
1.1 Teams must launch in optimized mode
1.2 
Teams must open a WebSocket connection (wss://127.0.0.1:9002) to the “Citrix HDX HTML5 Video redirection” service in the VDA. This action triggers the establishment of the virtual channel CTXMTOP
1.3 
CWA’s HDX Engine (wfica32.exe) must spawn the process HdxRtcEngine.exe (the actual WebRTC media engine in CWA)
 
2.Media Engine peripheral acquisition
2.1 WebRTC media engine maps and relays the names of the peripherals to Teams
2.2 
WebRTC media engine acquires and captures all peripheral devices I/O after user selection
2.3 WebRTC media engine must be able to display the video self-preview

 
3.Media Engine transport layer establishment
3.1 SDP file offer/answer negotiation
3.2 
ICE Candidates discovery (STUN/TURN)
3.3 
Secure RealTime Protocol (SRTP) end to end connection
 
4. Media Engine multimedia stream processing
4.1 Codecs: Decoding/encoding and rendering of audio-video tracks and screensharing
4.2 Echo Cancellation, noise reduction, jitter/packet loss concealment, lip synchronization, image enhancement
 

5. Monitoring
More information on how to leverage HDX Monitor to monitor Teams optimization can be found here.

 

Proper troubleshooting requires the Admin to verify each area progressively, moving down the list.

Please note that the presence of a Citrix Gateway as an HDX proxy should have no impact, so you can discard that from your troubleshooting. If you are also using the Gateway for VPN, make sure you are allowing the client machine to reach the O365 Teams servers.


#1: Media Engine instantiation

The quickest way to determine if the user is running in optimized mode is to look for HdxRtcEngine.exe running on the user’s client machine:

User-added image

For Linux, run:
#ps -ef | grep -i hdxrtcengine

If the process is not found, then point #1.1 should be investigated first.



User-added image

#1.1: Teams must launch in optimized mode


Click on the avatar / About / Version. “Citrix HDX Optimized" should be displayed. This means Teams is ready for an optimized call.

User-added image

If the legend is not there, and instead you see "Citrix HDX Not Connected" then Teams did not load in VDI mode correctly.

User-added image

Citrix HDX Not Connected means Teams loaded the Citrix API correctly, but it could not complete an end-to-end check because there was an error in subsequent parts of the stack. Most likely, VDA Services (Citrix HDX HTML5 Video Redirection, Citrix HDX Teams Redirection) or in Workspace app.

If there is no legend with the word Citrix, then Teams did not load in VDI mode at all:

User-added image

The VDA regkey MsTeamsRedir is set by a Citrix service (CtxSvcHost.exe) when the user logs on or reconnects to the virtual desktop. Teams will read this key when starting, and if the value is 1, it will load in VDI mode.

User-added image


The value is set to 1 automatically if and only if:

  1. Microsoft Teams Redirection policy is enabled in Studio (and by default is Allowed)
  2. CWA has loaded the Teams Virtual Channel (VDTeamsn.dll) and the CWA version is 1907 or higher

If = 0 or missing – trouble!

Action: Quit Teams from the systray itself (right click on the icon / Quit).
Sometimes, if Teams auto-updated while the session was in disconnected state, it can fail to read the key properly upon reconnections.

Lastly,  if the regkey value is 1 but Teams still fails to display "Citrix HDX Optimized" or "Citrix HDX Not Connected", there is likely a problem with Teams itself. Common issues could be erroneous cached items (settings.json and storage.json) in 
AppData\Roaming\Microsoft\Teams , or even dual Teams installations (under %appdata% and Program Files). Customers are advised to contact Microsoft support in this case.


#1.2: WebSocket phase

The VDA Service “Citrix HDX HTML5 Video Redirection” [WebSocketService.exe] is the point of entry for Teams into the VDA. In other words, Teams must open a secure websocket connection to this service, which listens at 127.0.0.1:9002 in the VDA.
If WebSocketService.exe is not actively listening, optimization will fail. In some cases, the Service will show as “Running” in the services.msc console, but when you run netstat you will not see the socket.
Therefore, you must make sure it is running and LISTENING :


`User-added image


After a successful connection, you will see the state change to “ESTABLISHED”:

User-added image

[Note: WebSocketService.exe listens also at 9001 – this socket is used for other HDX features (Browser Content Redirection and HTML5 Video Redirection, and they are independent of Teams)]

-Proxies: If an explicit proxy server is configured in the VDA, make sure connections to localhost are not routed through it. Otherwise, redirection will fail. You must "bypass proxy servers for local address" 127.0.0.1:9002

-PAC files: make sure your VDA pac file script is returning DIRECT for wss://127.0.0.1:9002. If not, optimization will fail. One way to do this:

 shExpMatch(url, "wss://127.0.0.1:9002/*")

-Anti-Virus: Some security software products interfere with the proper functioning of WebSocketService.exe and its certificates. While the actual service (Citrix HDX HTML5 Video Redirection) might be running in the services.msc console, the localhost 127.0.0.1:9002 TCP socket is never in listening mode as seen in netstat. Trying to restart the service causes it to hang (“Stopping…”). Ensure you apply the proper exclusions for the WebScoketService.exe process.


Once there is a successful secure WebSocket (wss) connection, WebSocketService.exe will create a new process in the user’s session called WebSocketAgent.exe:

User-added image

This process can also be spotted in Director:

User-added image

[Note: WebSocketAgent.exe is also used for BCR and HTML5 Video Redirection]

Action: if WebSocketAgent.exe is not being created while 127.0.0.1:9002 is ESTABLISHED, restart the WebSocketService.exe service. Relaunching Teams might be required. Also, in some rare cases, you might see WebSocketAgent.exe coming up but closing within a second, and in this case this could be related to a crash in HdxRtcEngine.exe in the client machine (hence make sure to collect EventViewer entries on the client).

The last part of this second phase is the establishment of the Teams virtual channel (CTXMTOP), and this process is ultimately handled by the “Citrix HDX Teams Redirection” Service (CtxSvcHost.exe in the VDA).
CTXMTOP is a Command & Control virtual channel only – no media is exchanged between CWA and VDA. All the media is handled by CWA in a Client-Fetch-Client-Render manner.

Action: If WebSocketAgent.exe is running on the VDA but HdxRtcEngine.exe is not running on the client, restart “Citrix HDX Teams Redirection”.

Notes on HdxRtcEngine.exe :
If you are locking down the endpoint in Kiosk-mode, or restricting users to interact only with whitelisted apps by using GPOs for "Assigned Access Configuration", then make sure HdxRtcEngine.exe is allowed to run.
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\AssignedAccessConfiguration\Profiles\
Some Anti-Virus programs might interfere with HdxRtcEngine.exe (which will open and close instantly) . Make sure you are applying proper exclusions.



#1.3: HdxRtcEngine.exe

Once the virtual channel is established, the Citrix HDX Engine in CWA (wfica32.exe) will spawn the media engine on the client:

User-added image

At this point, Teams is ready to handle calls.

If HdxRtcEngine.exe launches, then the respective webrpc log will be created.

-Windows Logs:
They can be found on the user’s machine in %userprofile%\AppData\Local\Temp\HdxRtcEngine folder.
Each HDX session will create its own separate folder.


-Mac Logs:
1.    vdwebrtc log -- records the execution of Virtual Channel vdwebrtc
Location:  /Users/<User Name>/Library/Logs/Citrix Workspace/CitrixViewer_<Y_M_D_H_M_S>.txt

2.    HdxRtcEngine log -- records the execution of process HdxRtcEngine
Location: /Users/<User Name>/Library/Logs/HdxRtcEngine/hdxrtcengine/<W_M_D_H_M_S_Y>/hdxrtcengine.log
Note that  Hdxrtcengine log is enabled by default.

-Linux Logs:
The /tmp/webrpc/<current date>/ and /tmp/hdxrtcengine/<current date>/ directory will have log files.


#2:Media Engine peripheral acquisition

In Teams, go to Settings à Devices. All the peripherals should now be mapped, and the video self-preview visible. You should be able to toggle between the cameras too.
The "Make a test call" button is a very handy tool to assess peripheral acquisition and performance. Go to Settings/Devices/Audio devices.


User-added image
Note: when Teams enumerates Playback/Recording peripherals, it will select the ones that are defaults at the endpointOpen mmsys.cpl on the endpoint, and you can see what the default devices are in the user's client machine. These are the ones that will be auto-selected by Teams.
You can toggle these devices in the middle of a call - in that case, a 1 sec audio glitch might be experienced while the new devices are remapped.

User-added image


The peripheral acquisition process is captured in the webrpc.txt logs.
All device enumeration entries will have “webrtcapi.RPCStubs” in the respective lines.

=============================
Thu Nov 28 15:08:30 2019
=============================

Nov 28 15:08:30.947 webrtcapi.DeviceManager Info: Audio device IMMNotificationClient registration successful.
Nov 28 15:08:30.948 webrtcapi.VideoOverlay Info: setProvider. set overlay provider (current = 0, new = 10168640)!
Nov 28 15:08:30.949 webrtcapi.DeviceManager Info: operator (). Device monitor message window created
Nov 28 15:08:31.287 webrtcapi.RPCStubs Info: -> device id = {0.0.1.00000000}.{100c5b9e-8f9f-4bc4-8ff2-196c45c88f76} label =  Headset Microphone (Jabra EVOLVE LINK MS) groupId = 1445567130

Nov 28 15:08:31.305 webrtcapi.RPCStubs Info: -> device id = {0.0.1.00000000}.{ee2e77ed-f871-4c65-8476-a40942ea913f} label = Microphone Array (Realtek High Definition Audio(SST)) groupId = 152748168
Nov 28 15:08:31.348 webrtcapi.RPCStubs Info: -> device id = {0.0.0.00000000}.{c11106bd-8a8e-441e-8bba-1740b9b9934b} label = Headset Earphone (Jabra EVOLVE LINK MS) groupId = 1445567130
Nov 28 15:08:31.351 webrtcapi.RPCStubs Info: -> device id = {0.0.0.00000000}.{d10485d5-ddbb-4568-a3ee-214d73dc6b09} label = Speakers (Realtek High Definition Audio(SST)) groupId = 152748168
Nov 28 15:08:31.385 webrtcapi.RPCStubs Info: -> device id = \\?\display#int3470#4&1835d135&0&uid13424#{65e8773d-8f56-11d0-a3b9-00a0c9223196}\{bf89b5a5-61f7-4127-a279-e187013d7caf} label = Microsoft Camera Front groupId =
Nov 28 15:08:31.397 webrtcapi.RPCStubs Info: -> device id = \\?\display#int3470#4&1835d135&0&uid13424#{65e8773d-8f56-11d0-a3b9-00a0c9223196}\{7c9bbcea-909c-47b3-8cf9-2aa8237e1d4b} label = Microsoft Camera Rear groupId =
[…]
Nov 28 15:15:01.197 webrtcapi.CameraCapture Info: start. capabilities: width = 1280 height = 720 fps = 30 fmt = 9
Nov 28 15:15:01.553 webrtcapi.DxVideoRenderer Info: configure. window = 203588 frameWidth =1280 frameHeight = 720
[…]

 

#2.1 Errors
Any error in the acquisition process will be displayed in the webrpc logs.
E.g.:
 
============================= Sun Mar 29 13:23:57 2020 =============================
Mar 29 13:23:57.232 webrtcapi.WebRTCEngine Info: init. initializing...
Mar 29 13:23:57.520 webrtcapi.WebRTCEngine Error: init. couldn't create audio module!

HdxTeams.exe supports only these specific audio device formats (channels, bit depth, and sample rate):

  • Playback Devices: {1 or 2 channels}, 16 bit, frequencies up to 96000 Hz (or equal)
  • Recording Devices: {1,2 or 4 channels}, 16 bit, frequencies up to 96000 Hz (or equal)

Even if one speaker or microphone does not match the expected settings, device enumeration in Teams fails and None displays under Settings > Devices.

As a workaround, open the Sound Control Panel (mmsys.cpl), select the playback or recording device, go to Properties > Advanced and change the settings to a supported mode.
Also, "Restore Defaults" under Advanced can be used to recover from this error.

Alternatively, disable the specific device.

Important: The error in the table above is fixed in Citrix Workspace app 2009.6 for Windows - so any peripheral (even Studio-Quality soundcards) is mapped.

If Teams loaded in unoptimized mode, it will try to acquire the endpoint’s peripherals leveraging the traditional HDX technologies for Audio, Microphone and Webcam redirection (see section on 7.15 for a description of unoptimized mode).
In this case, the camera name will show the “Citrix HDX Web Camera” legend – this is an unequivocal sign that Teams is unoptimized.


User-added image

3. Media Engine transport layer establishment

In this phase, errors are generally equivalent to call failures, or one-way audio/video only where one peer cannot hear/see the other.

The call setup process is handled primarily by Teams, not HDX WebRTC engine In other words, the hosted Teams app will utilize the signaling channel back to O365 Teams servers who will in turn contact the other peer.
Teams does rely on HDX WebRTC engine for the call negotiation by requesting an SDP file, which is a compilation of all the media and transport address characteristics of the endpoint.
HDX WebRTC engine relies on hole punching techniques (ICE/STUN/TURN) to eventually establish the connection to the other peer or conference server.

Enterprise network environments can be a challenge in this phase.
Proxies, Firewalls, symmetric NATs, VPNs, Deep packet inspectors, Session Border Controllers, etc etc can all affect the call establishment process; hence it is important you understand your network topology and consult with your networking team if necessary.


Teams conference servers are all hosted in O365 with the IP Blocks {52.114.0.0/12} ; {52.120.0.0/12} ; {13.107.64.0/18} and UDP ports 3478-3481. TCP 443 can be used as a fallback.
Peer to peer calls will be attempted via direct connections, but if that fails a TURN server (also hosted in O365, same IP:PORTS as above) might be used.


Important note for DNS : make sure the endpoint can do DNS resolutions. These Teams Relay Servers FQDN must be resolved by the endpoint (depending on GEO and Ring):
  • worldaz.relay.teams.microsoft.com
  • usaz.relay.teams.microsoft.com
  • euaz.relay.teams.microsoft.com
  • inaz.relay.teams.microsoft.com
  • uaeaz.relay.teams.microsoft.com
  • turn.gov.teams.microsoft.us
  • turn.dod.teams.microsoft.us

#3.1 When the user initiates or receives a call, the signaling channel between the Teams hosted app and the Teams servers in O365 will negotiate an SDP file, containing all the required information for HdxRtcEngine.exe to execute the call.
Please note that the SDP file can be updated multiple times after the initial single pass negotiation,
containing new info about ICE candidates or new call parameters (resolution, fps, etc).


Each peer maintains two simultaneous descriptions:
  • local one describing itself ("webrtcapi.RTCPeerConnection Info: setLocalDescription”)
  • a remote one describing the other peer (setRemoteDescription)
The SDP files are logged in webrpc.txt. Look for the “>>> begin:sdp” entries.
The caller will ‘set’ his SDP offer first (without any ICE candidates), which can be seen in the webrpc.txt logs as setLocalDescription.


#3.2 Interactive Connectivity Establishment
The caller now will ask STUN servers to generate ICE candidates. It will then send -over the signaling channel- individual ICE candidates as soon as they are discovered (a.k.a “trickle ICE”) by the local ICE agent .

Example #1 : VDI caller (behind Firewall blocking UDP) joining a conference

 
                                                                                               TRICKLE ICE snippet
[...]
>>> begin:sdp
candidate:780896439 1 udp 2122063615 10.0.0.140 52811 typ host generation 0 ufrag 9gep network-id 1 network-cost 10
candidate:1938109490 1 udp 24911871 52.114.188.57 51757 typ relay raddr 73.205.xxx.xxx rport 5039 generation 0 ufrag 9gep network-id 1 network-cost 10
<<< end:sdp
[...]
ICE Candidates can be [IPv4 or IPv6] : [UDP or TCP].  The 'type' parameter is important, and can take four values (host, server reflexive, peer reflexive and relay).
Conference servers will receive an SDP offer through the signaling channel and answer with its own SDP, which can be seen in the webrpc.txt logs as “webrtcapi.RTCPeerConnection Info: setRemoteDescription”.
Of utmost importance are the “a=candidate” lines in the SDP 
                                                                                               SDP ANSWER:
[…]
a=candidate:1 1 UDP 36044286 52.114.138.158 3480 typ relay raddr 10.0.136.233 rport 3480
a=candidate:2 1 TCP 18086910  52.114.138.158 3478 typ relay raddr 10.0.136.233 rport 3478
[…]

The VDI user caller will register remote ICE candidate and begin connectivity checks (STUN Bind request/response). Sometimes, the calling peer can be behind a Firewall that blocks STUN/UDP packets, and in such a case there will no answer to the STUN Bind requests:

User-added image

[Warning: STUN Bind requests are essential for a healthy call setup, as this allows candidate discovery with optimal call routing (especially in p2p calls). If your branch office Firewalls are blocking STUN and UDP destination ports 3478-81, then a transport relay in O365 must be contacted using TCP which adds additional latency and degrades call quality experience].

Since the Firewall is blocking UDP, the calling peer must attempt a TCP connection to the other candidate:2
If that also fails, it will have to use his transport relay (“relay raddr”) seen in the Trickle ice snippet above (52.114.188.57).


User-added image

Important note for branch-office proxies: HdxRtcEngine.exe cannot read explicit proxy configurations on the endpoint.

This was fixed in:

  • CWA for Windows 2012 (Negotiate:Kerberos, NTLM, Basic/Digest)
  • CWA for MAC 2104 (anonymous authentication only)
  • CWA for Linux 2101 (anonymous authentication only)

or higher versions. In those releases, the HDX media engine will read the system proxy settings and honor the configuration (e.g. explicit proxies or pac files).


#3.3 RTP media flows
If the connection was established using TCP through a TURN server, you will not be able to decode the traffic in Wireshark as RTP since it is encrypted using TLS.

In case of a peer-to-peer connection or conference via UDP, then Wireshark will show more detailed info that can be used to understand the traffic flow between the participants.

User-added image

Wireshark recommendations:


RTP cannot be automatically analyzed by Wireshark. Applying the Display filter ‘rtp’ won’t result in any RTP packets displayed.
Instead, first set the Display Filter to ‘udp’ to spot UDP traffic. After the connection is established, there will be UDP traffic sent regularly between the peers. RTP over UDP traffic will begin with 90 hexa (or 80 hexa if no RTP header extensions are present) in Data, the first octet of the RTP header.


User-added image

By selecting one of these UDP packets and selecting "Analyze/Decode As" … then selecting RTP, Wireshark will process the packets as RTP.

The WebRTC media engine in Workspace app (HdxRtcEngine.exe) uses the Secure RTP and DTLS protocols for multimedia streams that are offloaded to the client. SRTP provides confidentiality and authentication to RTP, using symmetric keys to encrypt media and control messages, and uses the AES encryption cipher.



4. Media Engine multimedia stream processing

Once the call set up is finalized and the ICE connectivity checks are completed, the media will be exchanged between the parties.
Currently we support video resolutions up to 720p@30fps, with 360p being common [H264 and VP9].

-The video streams are captured in the webrp.txt logs as "DxVideoRenderer ":
webrtcapi.DxVideoRenderer Info: configure. window = 657170 frameWidth = 640 frameHeight = 360
-The webcam captures entries in webrp.txt are"CameraCapture":
webrtcapi.CameraCapture Info: start. capabilities: width = 640 height = 360 fps = 30 fmt = 9

Lip sync issues might be caused by high CPU usage on the endpoint. The encoding process (camera capture and subsequent encoding) is more expensive in CPU terms than decoding. Some endpoints might not have enough processing power (see here for system requirements), resulting in 100% CPU utilization and causing audio/video to be out of sync.

Important note: Workspace app for Windows 1912 LTSR and 2002 CR (and any posterior version) introduced a performance estimator that is designed to measure the encoding capabilities of the endpoint's CPU. As a result, the video encoding is limited to a certain resolution (240p, 360p, 720p, 1080p).
This is particularly useful for endpoints that have low CPU speeds/amount of cores.
The estimator is run on HdxRtcEngine.exe launch time, transparent to the user.
The estimator might return a value that is lower for VP9 vs H264, since VP9 consumes more CPU. If you would like to tweak the resolution values, or disable VP9 and use H264 exclusively, this can be controlled via regkeys on the endpoint. See here for more details.


Citrix Workspace app (Windows 2103.1, Linux 2103, Mac 2102) does not support VP9 anymore for p2p video calls. H264 will be used instead for every type of call now (meetings or p2p), preserving client CPU.



The RTP protocol can be decoded in Wireshark to understand what type of codecs are being used and how much bandwidth is consumed.

#4.1 As an example we will investigate a p2p audio call between two VDI users. HdxRtcEngine.exe will select the Opus codec for this scenario, which can be seen clearly in the SDP Offer/Answer negotiation.
The calling party will state all his supported codecs, and the callee will then reply with his preferred choice in the SDP answer.

[Note: The notification audio (ringtone) is played by Teams out of the VDA’s default playout device.]

User-added image

The call bandwidth usage can then be plotted in Wireshark using the Statisticsà IO Graph option.
Apply the proper filters to display only the traffic associated with the call.
We can see that the typical bandwidth for a voice call is ~40-70 kbps (each way).


User-added image
Green: outgoing | Blue: incoming | Red: outgoing + incoming


Screensharing

In the Skype for Business optimization pack (RTOP), screensharing was not optimized. In other words, there was no offloading of such streams and therefore they were rendered server-side natively by Skype for Business.

But in terms of HDX optimization for Microsoft Teams, incoming screensharing is no different than incoming webcam video streams, since Teams relies on video-based-screen-sharing. Hence it is optimized by default.

However, a VDI user sharing his monitor (i.e. outgoing screensharing) will result in HdxRtcEngine.exe performing display captures at the endpoint (with H.264). Currently it is only possible to share the primary (default) monitor with CWA 2105 or older.
HdxRtcEngine.exe crops the sharing to the Desktop Viewer window only (if CWA's Desktop Viewer is in Windowed mode).

User-added image

If Desktop Viewer (cdviewer.exe) is in full screen mode, then of course the entire monitor is shared.
In multi-monitor setups, only the primary monitor is shared, and the screen picker button in Teams will label the preview as 'Screen #1'.

User-added image

Multimonitor setups are now supported with CWA 2106 for Win/Linux/Mac or higher.
In the Windows case, Desktop Viewer must be enabled in StoreFront/ICA file and in full screen mode in order for the picker to display all available monitors.

Troubleshooting screensharing issues:

Common issues with screensharing, as perceived by the other peer, are:
- low resolution or low fps
- black window instead of the actual screen
- screen freeze from the sender's side

The webrpc.txt logs (AppData\Local\Temp\HdxRtcEngine in the user's device, not the VDA) are the best source for troubleshooting such issues.
The outgoing screensharing flow is as follows:

[CreateDisplayTrack --> VideoTrackSource (id = ########) --> startVideoTrackSource --> DesktopCapture.start]

Oct 01 10:06:24.222 webrtcapi.WebRTCEngine Info: createDisplayTrack. id = DV-9d1651420da4 sourceId = 
Oct 01 10:06:24.223 webrtcapi.VideoTrackSource Info: VideoTrackSource. id = 03B28CB8 source-id = 0 width = 1920 height = 1080 fps = 15
Oct 01 10:06:24.224 webrtcapi.MediaStreamTrack Trace: MediaStreamTrack. video track-id = DV-9d1651420da4 label = display-track state = 0
Oct 01 10:06:24.224 webrtcapi.WebRTCEngine Trace: startVideoTrackSource. first capture consumer, source-id = 0
Oct 01 10:06:24.224 webrtcapi.DesktopCapture Info: start.


HdxRtcEngine.exe contains a bandwidth estimator that will allow Teams to pick the best resolution that can be sustained by the network, and sometimes it can take a few seconds for it to converge to the right value. So screensharing might actually start with a low resolution before it jumps to a higher one.

In the webrpc logs, look for "webrtcapi.VideoTrackSource Info: setOptions" to spot what resolution and fps is being applied:
 

[2716.921] Oct 04 11:35:10.670 webrtcapi.VideoTrackSource Info: setOptions. id = 03457800 width = 1920 height = 1080 fps = 15



In the logs, also the parameter "webrtcapi.DesktopCapture" can be used for quick finds, and if there are errors they would be logged as "webrtcapi.DesktopCapture Error:". 
Black windows can be caused by issues on the sender's side (user capturing) or the receiver's side (user rendering the stream).
For example, if the error is on the sender's side the following entry could be seen:
webrtcapi.DesktopCapture Error: OnCaptureResult. invalid cropping rectangle (x = 0 y = 0 width = 0 height = 0)



Monitoring Teams with HDX Monitor

On 1912 VDAs (or higher), you can monitor an optimized active call using HDX Monitor (3.5.2 or higher).
Important : make sure you are matching the HDX monitor version with the right VDA version (HDX Monitor is packaged with the CVAD .iso)
To turn this on, create the following WebrtcDirectorIntegration regkey on the VDA:

HKLM\Software\Citrix\HDXMediaStream\
Or
HKLM\
SOFTWARE\WOW6432Node\Citrix\HdxMediaStream

- reg key value:
name: WebrtcDirectorIntegration
type: DWORD
value: enable(1), disable(0)


The information is displayed under the WebRTC icon :

User-added image

The following data content will be populated when an active Teams redirection is going on.
Please see below for the detailed field description:


User-added image

  • Call Device Used Audio: audio device used for the last active call
  • Call Device Used Video: video device used for the last active call
  • Call Direction: incoming or outgoing
  • Call Duration Audio: last active audio call duration in seconds
  • Call Duration Video: last active video call duration in seconds
  • Call Duration Conference: last active conference call duration in seconds
  • Call Duration Screenshare: last active desktop sharing call duration in seconds
  • Call Establishment Duration Incoming: last active incoming call establishment in sec
  • Call Establishment Duration Outgoing: last active outgoing call establishment in sec
  • Call State: Active or Idle
  • Call Type: Audio,  Video, Conference, Screenshare
  • Version - Endpoint OS: not filled currently
  • Version - Receiver: CWA version
  • Version - Typescript: API version
  • Version - Vda: not filled currently
  • Version - Webrpc: webrpc version
  • Version - WebrtcCodec: webrtc codec version




Guidelines for 7.15 LTSR

If you are using 7.15 LTSR (with any CU), please note that the new HDX Optimization for Microsoft Teams is not (and will not be) available in this version and the only way to use Teams is through the legacy HDX technologies:


While these technologies do offload some of the CPU/RAM/BW from the VDA, they do not fully optimize the experience for Real Time traffic and will create additional loads on the VDA resources.
The best example is the rendering of a video feed from another peer / meeting. This is done using Thinwire graphics technologies and it renders on the VDA, which is expensive in CPU terms.
Admins concerned about Single Server Scalability might want to disable video conferencing policies on the Microsoft Teams admin center. See here for more info.

A secondary method for redirecting peripherals is available, and some customers might prefer to use Generic USB Redirection.
With HDX Generic USB Redirection technology, the peripherals (e.g webcam or headset) is virtually detached from the client device and attached to the XenApp/XenDesktop session. This provides all the native functionalities of the webcam/peripheral in the virtual session.
HDX Generic USB Redirection requires the device drivers for the webcam/peripheral to be available on both the client device as well as on the VDA.
Bandwidth usage for webcams using HDX Generic USB Redirection technology can vary based on the vendor and model of the device, but it is significantly higher compared to use it over HDX RealTime Webcam Video Compression. HDX Generic USB for webcams/peripherals is recommended to be used only under LAN conditions where bandwidth and specially latency are not constraints.
Refer the following link regarding more information on HDX Generic USB configuration: Configure USB Support.

 

Known Issues with Microsoft Teams and 7.15 VDA:

  • Webcam redirection will not work with 64-bit Teams, only with 32-bit. This is known limitation in 7.15 (any CU) - see here for more details.
  • In Windows Server, Teams.exe might crash, or the Teams logon screen might become blank and no input fields are presented to the user. Please add Teams.exe to the VDA registry under SfrHook, and make sure the Flag value REG_DWORD is set to 204 Hexadecimal  
  • After you add the SfrHook key, you must reboot your VDA.
  • As a last resort, you can disable ALL hooking for Teams by following CTX107825
User-added image
 

 

Limiting Video Resolutions 

Administrators who have users on lower-performance client endpoints may choose to limit incoming or outgoing video resolutions to decrease the impacts of encoding and decoding video on those endpoints.  In Citrix Workspace app 2010, Citrix introduced the capability to limit these resolutions using client configuration options.  
Note: Users running with restricted resolutions impact the overall video quality of the conference because the Microsoft Teams server will be forced to use the lowest-common-denominator resolution for all conference participants. 

Call constraints are disabled by default on the client with Citrix Workspace app 2210. To enable, administrators must set client-side configurations as follows:

 

Windows

Configure the below client registry keys in HKEY_CURRENT_USER\SOFTWARE\Citrix\HDXMediaStream.  All keys are DWORDs.
Name Type Mandatory Acceptable values
EnableSimulcast Int YES 1 to 3 (Set it to 1)
MaxOutgoingResolution int YES 180,240,360,540,720,1080 (Teams supported Resolutions)
MaxIncomingResolution int YES 180,240,360,540,720,1080 (Teams supported Resolutions)
MaxIncomingStreams int YES 1 to 8
MaxSimulcastLayers int YES 1 to 3 (set it to 1)
MaxVideoFrameRate int NO 1 to 30
MaxScreenshareFrameRate int NO 1 to 15

 

Mac

To configure the Mac HDX RTC Engine for 360p, you should execute the following commands:
defaults write com.citrix.HdxRtcEngine EnableSimulcast -int 1
defaults write com.citrix.HdxRtcEngine MaxOutgoingResolution -int 360
defaults write com.citrix.HdxRtcEngine MaxIncomingResolution -int 360
defaults write com.citrix.HdxRtcEngine MaxIncomingStreams -int 3
defaults write com.citrix.HdxRtcEngine MaxSimulcastLayers -int 1




Linux

On the endpoint settings in the Linux config.json.  The following example assumes that there isn’t config.json file present.
mkdir -p /var/.config/citrix/hdx_rtc_engine

Create a config.json file in that directory.
The file contents should look like this:
{
    "EnableSimulcast":1,
    "MaxOutgoingResolution":360,
    "MaxIncomingResolution":360,
    "MaxIncomingStreams":3,
    "MaxSimulcastLayers":1
}
 

Additional Resources

Release notes history:

 
  • 3/23/2022: CWA for Windows 2203 LTSR and 2204 CR are now GA. These versions allow App Protection users to be able to see incoming video and screensharing, and also introduce support for Secondary Ringtone (pending update from Microsoft, refer to edocs).
  • 3/15/2022: A fix for the issue below has been rolled out by Microsoft. Users are required to sign out / sign back in or Exit Teams.
  • 3/14/2022: Customers in Public GCC or GCC High Office 365 Rings might be experiencing intermittent webcam failures (Teams detects the camera but not able to acquire video) if the Workspace app is equal or higher to 2109 (on any OS platform). This issue has been reported recently in March. Microsoft is investigating [Microsoft IcM Reference #294654482] [Citrix Reference CVADHELP-19623]
  • 3/8/2022: Fix for P2P call failure is now completely rolled out by Microsoft. No Teams client/Workspace app/VDA upgrade required (the fix was applied to the Teams backend services)
  • 2/25/2022: P2P calls (not conference) between an HDX optimized Teams user and a user on native Teams desktop client 1.5.00.2164 or higher can intermittently fail a few seconds after establishment. User would see "Hold on, looks like something went wrong – we are trying to get you back on the Call", the call would resume and subsequently interrupt. The issue is seen irrespective of the CWA or VDA versions. The problem is not seen in P2P calls between two optimized users. Citrix and Microsoft are investigating. References for Microsoft IcM# cases (279646315 and 291199411) and Citrix SR# cases (81036409 and 81046791)
  • 2/4/2022: App Sharing (i.e. sharing a single app and not a full desktop) is now available in Ring 3.6 (Public Preview Channel). This is an opt-in program, controlled via IT admin policy. Ring 4 (GA) is not yet available.
  • 12/9/2020: Gallery View (2x2) is now on public tech preview. See here for more details.
  • 12/8/2020: Microsoft introduced a new public tech preview methodology for Teams. Citrix recommends you enroll.
  • 12/8/2020: Citrix Workspace app for Windows 2012 is GA, adding support for proxy servers and mouse pointer capture. See Release notes.
  • 12/2/2020: Citrix Workspace app for MAC 2012 is GA, adding support for screensharing. See Release notes.
  • 11/15/2020: VDA 1912 LTSR CU2 released, including important fixes for bugs causing Teams to load unoptimized . See Release notes.
  • 11/11/2020 : "Make a test call" is now available under Teams/Settings/Devices/Audio devices. Sign out / sign back into Teams required.
  • 11/2/2020 : Citrix Workspace app for Windows 2010 is GA. See fixed issues (around screensharing)
  • 10/29/2020 : Citrix Workspace app for Linux 2010 is GA. See fixed issued
  • 10/15/2020 : support for DTMF has been enabled in Teams. Sign out / sign back into Teams required. (Workspace app for Windows 2006.1 or higher)
  • 10/8/2020 : a bug has been reported in Workspace app for Windows 2009.6 for outgoing screensharing failures (in setups where the monitor has high DPI or multi-monitor with high resolutions). Fixed in 2010.
  • 9/24/2020 : Citrix Workspace app for Windows 2009.6 GA. See Release notes.
  • 9/23/2020 : Citrix Workspace app for MAC 2009 supports Teams optimization
  • 9/8/2020 : In Windows Server (any version, with any VDA 1906.2 or higher), session disconnect/reconnect might result in Teams failing to load in optimized mode ("Citrix HDX Not Connected" in About/Version). Citrix and Microsoft rolled out the fix on Sept-8-2020, deployed already in GA (a.k.a Ring4). This did not require any component upgrade (Teams, VDA or Workspace app) and was rolled out via feature flags (only sign out/sign in required). [RFWIN-15624] [Microsoft Teams Bug ID: 873138].
  • Teams version 1.3.0.13565 now includes a feature that can control the fallback mechanism from optimized to non-optimized (e.g. disabling Webcams or disabling both audio and webcams so there is no impact to server-side resources). See "fallback" section in edocs.
  • 8/22/2020 : Timezone Redirection mismatch when Teams is running in VDI: please upgrade Teams to get the fix (to version 1.3.00.21759)
  • 8/16/2021: multimonitor screensharing is now available with CWA for Win/Mac/Linux 2106 or higher. Log out/log back into Teams required. Edocs link here.
  • 8/12/2021: Teams Keyboard Shortcuts such as Ctrl+E (Global Search box) , Ctrl+F (Find in existing context), etc. are now working in VDI. Log out/log back into Teams required.
  • 8/11/2021: CWA 1912 CU5 is now available. Customers using CWA 1912 are encouraged to test and update their clients, since CU5 contains numerous bug fixes and performance improvements not available in any previous CU. Release notes here.
  • 8/11/2020 : Optimization is now available for Microsoft Teams in GCC High
  • 8/10/2021: CWA 2108 for MAC is now available. This release fixes a bug where an optimized user from MAC is seen by other peers in the conference call in slow motion (10x speed reduction).
  • 8/10/2021: DTMF (CWA for Windows 2102, Linux/Mac 2101 or higher and CWA for Windows 1912 CU5) is now available. Please sign out and sign back into Teams.
  • 8/9/2021: Hidden chat channels are now visible/clickable in optimized Teams when running in a virtual desktop - Microsoft issue (incident ref 241949910)
  • 7/21/2020 : Screensharing from Chat window is now available in GA (Ring4). Sign out and sign back into Teams to enable the feature.
  • 6/16/2021: Citrix Workspace app for Windows 2106 is available, addressing a bug where incoming video feeds might flicker [CVADHELP-17398]
  • 5/25/2021: Bandwidth estimation improvements rolled out in Teams for better performance/video quality in Gallery View 2x2 (sign out/sign in to Teams)
  • 5/24/2021: Gallery View 2x2 is now also available in Microsoft 365 Government - GCC High
  • 5/23/2021: Hidden chat channels are not visible in Teams when running in a virtual desktop - Microsoft issue (incident reference 241949910) ; currently working on the fix)
  • 5/10/2021: Citrix Workspace app 2105 for Windows is GA (release notes here). This release fixes incoming video/screensharing seen as black screens, failure to receive video feeds when joining a meeting from Outlook, video rendering flickers and HdxRtcEngine crashes in p2p calls (HDX-30176).
  • 4/01/2021: endpoint/client devices with multiple NICs (e.g. VPN, Wireless, Ethernet) might experience optimized webrtc traffic not being routed through the proper interface, hence degrading call quality because of routing metrics and different Layer 3 performances (CVADHELP-16480). This was fixed in CWA 2105 for Windows.
  • 03/23/2021: Citrix Workspace app (Windows 2103.1, Linux 2103, Mac 2102) does not support VP9 anymore for p2p video calls. H264 will be used instead for every type of call now (meetings or p2p), preserving client CPU.
  • 03/17/2021: Endpoints with quad-channel microphones (e.g. HP EliteBook laptops) might fail to capture audio with CWA for Windows 2009.6 or higher, up to 2103 (HDX-29725). This was fixed in CWA for Windows 2105 or higher, and it requires the creation of a client-side REG_DWORD regkey with Value 1 in [HKCU\Software\Citrix\HDXMediaStream ; Key: DisableADM2].
  • In Windows 10 VDAs, disconnecting from a session by using the tsdiscon.exe Microsoft utility (as described in CTX225970 option#2) will cause Teams to load un-optimized when reconnecting to the session. We recommend using the Desktop Viewer toolbar for disconnecting.
  • In P2P calls, VDI users might experience a 'black screen' when another peer shares his screen. If the user then minimizes the Teams UI (e.g. by clicking on the chat button) so it is positioned in the upper left corner, the screen share becomes visible. Maximizing it shows the black screen again. Sign out/Sign in back into Teams fixes the issue. [CVADHELP-14716]
  • In some scenarios, even though "Citrix HDX Not Connected" is displayed, attempting to make a call will trigger Teams to connect to the Citrix VDA services and optimization will succeed. This is caused by a delay in the Teams UI refresh.
  • If Teams is configured to Auto-start, the issue in the bullet above might also manifest. We recommend disabling auto-start by deleting the Teams regkeys in HKLM\Software\Microsoft\Windows\CurrentVersion\Run, HKLM\Software\WOW6432Node\Microsoft\Windows\CurrentVersion\Run and HKCU\Software\Microsoft\Windows\CurrentVersion\Run
  • Note that currently it is not possible to disable Auto-start even if using the command line flag OPTIONS="noAutoStart=true" with the machine-wide MSI and the ALLUSER=1 property. Deleting the regkey described in the bullet above should fix this.
  • With Workspace app 1912 or 2002, peer-to-peer calls (not conferences) might result in lower encoding resolutions for the VDI user sharing his webcam, depending on the endpoint CPU. You can try either disabling VP9, or forcing the resolution to medium. See here for instructions. Citrix fixed this issue in Workspace app for Windows 2006.1 [RFWIN-17890]
  • Endpoints with high-end sound cards (Studio quality) might fail to load in optimized mode. See section #2.1 below. Clicking on Teams/Settings/Devices will show "None". Citrix is aware of this issue and working on a fix in Workspace app. [RFWIN-17892] [Issue fixed in Workspace app for Windows 2009.6]
  • If the Virtual Desktop does not have a GPU/vGPU, we recommend "Disable GPU hardware acceleration" in Teams/Settings in order to improve performance and scalability, otherwise Teams will use the VDA's CPU to render UI elements/animations. This setting ("disableGpu":true") is stored in %Appdata%\Microsoft\Teams inside desktop-config.json, so you could also use a logon script to edit that file and set the value to true.
  • In some scenarios, when the VM's NTP time sync server is updated via GPOs, this can create a clock drift when compared to the time when the Citrix HTML5 Video Redirection service is started. This drift then translates into invalid/expired Citrix certificates and Teams will fail to load in optimized mode. We recommend changing the HTML5 Video Redirection Service ("CtxHdxWebSocketService") Startup Type to "Automatic (Delayed Start)" instead of the default "Automatic". Citrix fixed this problem in CVAD 2006 VDA. [HDX-24350]
  • When installing Teams in a machine that has a VDA (either VM or Remote PC), Teams will disable background effects.

********************************************************************************************