Overview

The Video SDK provides you with an option to access real-time raw audio, video, and share data during a session, so you can apply additional effects and enhance the user experience. Use a processor component to apply effects or transformations to each piece of data in real-time.

An example of raw share data is when a user shares their screen.

  • Receive and send raw share video data using the same logic as video data, except get the share pipe instead of the video pipe in the raw data pipe method.
  • Receive and send raw share audio data using the same method used for raw audio data.

Follow the process outlined in this flowchart to capture and process raw data.

Flowchart outlining the raw data flow

To subscribe to remote raw audio, video, and share frames, see Receive raw data. To feed your own frames into a session, see Send raw data.

This sample app demonstrates raw video and audio access in a headless Docker environment.

Prerequisites

The Video SDK for Linux requires pulseaudio to enable audio raw data functions.

If you're using a Docker container to create your app, do the following.

  1. Install pulseaudio before implementing the audio raw data function.

  2. Create a configuration file at ~/.config/zoomus.conf with the following contents.

    [General]
system.audio.type=default

Select raw data memory mode

In order to utilize raw data of any type, you must first select the memory mode you wish to use. The SDK supports heap-based and stack-based memory modes.

Stack-based memory

  • Variables are allocated automatically and deallocated after the data leaves scope.
  • Variables are not accessible from or transferable to other threads.
  • Typically has faster access than heap-based memory allocation.
  • Memory space is managed by the CPU and will not become fragmented.
  • Variables cannot be resized.

See Stack-based memory allocation for details.

Specify with:

ZoomVideoSDKRawDataMemoryModeStack

Heap-based memory

  • Variables are allocated and deallocated manually. You must allocate and free variables to avoid memory leaks.
  • Variables can be accessed globally.
  • Has relatively slower access than stack-based memory allocation.
  • Has no guarantee on the efficiency of memory space and can become fragmented.
  • Variables can be resized.

See Heap-based dynamic memory allocation for details.

Specify with:

ZoomVideoSDKRawDataMemoryModeHeap

Specify memory mode

After determining which memory mode is right for you, specify it when the SDK is initialized. This must be done for audio, video, and share individually. To specify a raw data memory mode, provide one of these enum cases to the ZoomVideoSDKInitParams during SDK initialization.

ZoomVideoSDKInitParams init_params;
init_params.videoRawDataMemoryMode = ZoomVideoSDKRawDataMemoryModeHeap;
init_params.shareRawDataMemoryMode = ZoomVideoSDKRawDataMemoryModeHeap;
init_params.audioRawDataMemoryMode = ZoomVideoSDKRawDataMemoryModeHeap;

From the developer blog

For a walkthrough, see the following blog post.