Send raw data

You can feed your own video, audio, and share frames into a session, which is useful for headless apps with no camera or microphone, or to send processed media.

Before sending raw data, set the raw data memory mode.

Send raw video data

You can send video data by using sendVideoFrame within ZMVideoSDKVideoSender. To obtain a ZMVideoSDKVideoSender you must assign a ZMVideoSDKVideoSource. The onInitialize within ZMVideoSDKVideoSource will provide a ZMVideoSDKVideoSender.

func onInitialize(_ sender: ZMVideoSDKVideoSender, supportedCapbilityList: [Any], suggest suggestCapbility: ZMVideoSDKVideoCapability) {
    // Store video raw data sender.
    self.videoSender = sender
}
// Call sendVideoFrame to send a frame buffer of raw data.
self.videoSender?.sendVideoFrame(frameBuffer, width: width, height: height, frameLength: frameLength, rotation: rotation, format: format)

- (void)onInitialize:(ZMVideoSDKVideoSender *)sender supportedCapbilityList:(NSArray *)supportedCapList suggestCapbility:(ZMVideoSDKVideoCapability *)suggestCap {
    // Store video rawdata sender.
    self.videoSender = sender;
}
// Call sendVideoFrame to send a frame buffer of raw data.
[self.videoSender sendVideoFrame:frameBuffer width:width height:height frameLength:frameLength rotation:rotation format:format];

In the onInitialize:(ZMVideoSDKVideoSender *)sender supportedCapbilityList:(NSArray *)supportedCapList suggestCapbility:(ZMVideoSDKVideoCapability *)suggestCap callback:

• The supportedCapList parameter is the supported capability list. It combines the supported resolution of the session and the device to create a list of supported capabilities.
• The suggestCap parameter is the suggested capability. It combines the maximum capability of the session itself and the device to get the real maximum as the suggested capability.

The format parameter is a ZMVideoSDKFrameDataFormat: either ZMVideoSDKFrameDataFormat_I420_Limited or ZMVideoSDKFrameDataFormat_I420_Full.

Pre-process raw video data

Pre-process raw video data using onPreProcessRawData within ZMVideoSDKVideoSourcePreProcessor.

func onPreProcessRawData(_ rawData: ZMVideoSDKYUVProcessDataI420!) {
    // Perform preprocess actions here.
}

- (void)onPreProcessRawData:(ZMVideoSDKYUVProcessDataI420 *)rawData {
    // Perform preprocess actions here.
}

Send raw audio data

Follow these steps to send raw or processed audio data of a user from the user's device.

1. Create an instance of ZMVideoSDKVirtualAudioMic.
2. Pass that instance into ZMVideoSDKSessionContext.
3. Obtain ZMVideoSDKAudioSender from the onMicInitialize callback.
4. Use the send method to send raw audio data.

VirtualMicExample.swift

class VirtualMicExample: NSObject, ZMVideoSDKVirtualAudioMic {
    var audioSender: ZMVideoSDKAudioSender?
    func onMicInitialize(_ rawdataSender: ZMVideoSDKAudioSender) {
        // Virtual Microphone has initialized, store rawdataSender to send raw audio data into session later.
        audioSender = rawdataSender
    }
    func onMicStartSend() {
        guard let audioSender = audioSender else { return }
        // Virtual Microphone can begin sending raw audio into session.
        // Provide your audio data info here.
        guard let yourAudioBuffer = UnsafeMutablePointer<Int8>(mutating: ("buffer here" as NSString).utf8String) else { return }
        let yourAudioDataLength = UInt()
        let yourAudioSampleRate = UInt()
        let yourAudioChannel: ZMVideoSDKAudioChannel = .ZMVideoSDKAudioChannel_Mono
        let sendRawAudioReturnStatus = audioSender.send(yourAudioBuffer, dataLength: yourAudioDataLength, sampleRate: yourAudioSampleRate, channel: yourAudioChannel)
        switch sendRawAudioReturnStatus {
        case .ZMVideoSDKErrors_Success:
            print("Call to send raw audio succeeded")
        default:
            print("Call to send raw audio produced an error: \(sendRawAudioReturnStatus)")
        }
    }
    func onMicStopSend() {
        // Virtual Microphone has stopped sending raw audio.
    }
    func onMicUninitialized() {
        // Virtual Microphone has been destroyed.
        audioSender = nil
    }
}
// Create a VirtualMicExample object and assign it to the ZMVideoSDKSessionContext object virtualAudioMicDelegate you have created before joining a session.
let virtualAudioMic = VirtualMicExample()
sessionContext.virtualAudioMicDelegate = virtualAudioMic

VirtualMicExample.h

#import <Foundation/Foundation.h>
NS_ASSUME_NONNULL_BEGIN
@interface VirtualMicExample : NSObject <ZMVideoSDKVirtualAudioMic>
@property (nonatomic, strong) ZMVideoSDKAudioSender *audioSender;
@end
NS_ASSUME_NONNULL_END

VirtualMicExample.m

#import "VirtualMicExample.h"
#import <AVFoundation/AVFoundation.h>
#import <AssetsLibrary/AssetsLibrary.h>
@implementation VirtualMicExample
- (instancetype)init
{
    self = [super init];
    if (self) {
        _audioSender = nil;
    }
    return self;
}
- (void)onMicInitialize:(ZMVideoSDKAudioSender *_Nonnull)rawdataSender;
{
    // Virtual Microphone has initialized, store rawdataSender to send raw audio data into session later.
    if (self.audioSender != rawdataSender) {
        self.audioSender = rawdataSender;
    }
}
- (void)onMicStartSend;
{
    // Virtual Microphone can begin sending raw audio into session.
    if (!self.audioSender) {
        return;
    }
    // Provide your audio data info here.
    unsigned char* yourAudioBuffer;
    NSUInteger yourAudioDataLength;
    NSUInteger yourAudioSampleRate;
    ZMVideoSDKErrors sendRawAudioReturnStatus = [self.audioSender send:(char *)yourAudioBuffer dataLength:yourAudioDataLength sampleRate:yourAudioSampleRate channel:ZMVideoSDKAudioChannel_Mono];
    if (sendRawAudioReturnStatus == ZMVideoSDKErrors_Success) {
        // Call to send raw audio succeeded.
    } else {
        NSLog(@"Call to send raw audio produced an error: %@", @(sendRawAudioReturnStatus));
    }
}
- (void)onMicStopSend;
{
    // Virtual Microphone has stopped sending raw audio.
}
- (void)onMicUninitialized;
{
    // Virtual Microphone has been destroyed.
    self.audioSender = nil;
}
@end
// Create a VirtualMicExample object and assign it to the ZMVideoSDKSessionContext object virtualAudioMicDelegate you have created before joining a session.
VirtualMicExample *virtualAudioMic = [VirtualMicExample new];
sessionContext.virtualAudioMicDelegate = virtualAudioMic;

The channel parameter is a ZMVideoSDKAudioChannel: either ZMVideoSDKAudioChannel_Mono or ZMVideoSDKAudioChannel_Stereo. The dataLength must be even, and the sampling bits must be 16.

Send raw share data

Follow these steps to send raw screen share data.

1. Have your class conform to the ZMVideoSDKShareSource protocol.
2. Use the sendShareFrame method in onShareSendStarted to send raw screen share data.

RawShareDataExample.swift

class RawShareDataExample: NSObject, ZMVideoSDKShareSource {
    var shareSender: ZMVideoSDKShareSender?
    func onShareSendStarted(_ rawDataSender: ZMVideoSDKShareSender?) {
        shareSender = rawDataSender
        // sendShareFrame sends one frame of data, therefore you will most likely need a thread for the full frame.
        shareSender?.sendShareFrame(frameBuffer, width: width, height: height, frameLength: frameLength, format: format)
    }
    func onShareSendStopped() {
        // Stop sending raw share data.
        shareSender = nil
        // If you have a thread for onShareSendStarted, you will most likely need to cancel it and set it to nil.
    }
}

RawShareDataExample.h

#import <Foundation/Foundation.h>
@interface RawShareDataExample : NSObject <ZMVideoSDKShareSource>
@property (nonatomic, strong) ZMVideoSDKShareSender *shareSender;
@end

RawShareDataExample.m

#import "RawShareDataExample.h"
@implementation RawShareDataExample
- (void)onShareSendStarted:(ZMVideoSDKShareSender *)rawDataSender
{
    self.shareSender = rawDataSender;
    // sendShareFrame sends one frame of data, therefore you will most likely need a thread for the full frame.
    [self.shareSender sendShareFrame:frameBuffer width:width height:height frameLength:frameLength format:format];
}
- (void)onShareSendStopped {
    // Stop sending raw share data.
    self.shareSender = nil;
    // If you have a thread for onShareSendStarted, you will most likely need to cancel it and set it to nil.
}
@end

The format parameter is a ZMVideoSDKFrameDataFormat, the same enum used when sending raw video.