AVSampleBufferDisplayLayer: 'kCMSampleBufferAttachmentKey_PostNotificationWhenConsumed' not firing on real device

Originator:Chylis88
Number:rdar://33453254 Date Originated:July 21 2017, 3:11
Status:Open Resolved:
Product:iOS SDK Product Version:
Classification: Reproducible:Yes
 
When setting the 'kCMSampleBufferAttachmentKey_PostNotificationWhenConsumed' attachment on a CMSampleBufferRef and enqueueing the CMSampleBufferRef into a AVSampleBufferDisplayLayer, I expect the 'kCMSampleBufferConsumerNotification_BufferConsumed' notification to be posted when the buffer has been consumed. 

It works as expected when running in the simulator, but the notification is never posted when running on a real device.
    




Code to reproduce issue:

#import "ViewController.h"
@import AVFoundation;
@import UIKit;
@import CoreVideo;

@interface SampleBufferView : UIView
@property (nonatomic, readonly) AVSampleBufferDisplayLayer *sampleBufferDisplayLayer;
@end

@implementation SampleBufferView

+ (Class)layerClass {
    return [AVSampleBufferDisplayLayer class];
}
- (AVSampleBufferDisplayLayer *)sampleBufferDisplayLayer {
    return (AVSampleBufferDisplayLayer *)self.layer;
}
@end

@interface ViewController ()
@property (nonatomic, readonly) SampleBufferView *sampleBufferView;
@end

@implementation ViewController

///This callback is only called when running in simulator - not when running on an actual device.
void cmSampleBufferConsumedNotificationCallback (CFNotificationCenterRef center,
                           void * observer,
                           CFStringRef name,
                           const void * object,
                           CFDictionaryRef userInfo) {
    NSLog(@"Successfully received notification: '%@'", name);
}

- (void)loadView {
    self.view = [SampleBufferView new];
}

-(void)viewDidLoad
{
    [super viewDidLoad];

    //Register observer for the 'kCMSampleBufferConsumerNotification_BufferConsumed' notification
    CFNotificationCenterRef center = CFNotificationCenterGetLocalCenter();
    CFNotificationCenterAddObserver(center, NULL, cmSampleBufferConsumedNotificationCallback,
                                    kCMSampleBufferConsumerNotification_BufferConsumed,
                                    NULL,
                                    CFNotificationSuspensionBehaviorDeliverImmediately);

}

- (SampleBufferView *)sampleBufferView {
    return (SampleBufferView *)self.view;
}

- (void)viewDidAppear:(BOOL)animated {
    [super viewDidAppear:animated];
    [self displayTestBuffer];
}

- (void)displayTestBuffer {

    NSDictionary *attributes = @{ (id)kCVPixelBufferIOSurfacePropertiesKey: @{} };

    CVPixelBufferRef pixelBuffer;
    if (CVPixelBufferCreate(kCFAllocatorDefault,
                            1920,
                            1080,
                            kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange,
                            (__bridge CFDictionaryRef)attributes,
                            &pixelBuffer) != kCVReturnSuccess) {
        [NSException raise:NSInternalInconsistencyException
                    format:@"Failure in CVPixelBufferCreate"];
        return;
    }

    CMVideoFormatDescriptionRef formatDescription;
    if (CMVideoFormatDescriptionCreateForImageBuffer(kCFAllocatorDefault,
                                                     pixelBuffer,
                                                     &formatDescription) != noErr) {
        [NSException raise:NSInternalInconsistencyException
                    format:@"Failure in CMVideoFormatDescriptionCreateForImageBuffer"];
        return;
    }

    CMSampleBufferRef sampleBuffer;
    if (CMSampleBufferCreateReadyWithImageBuffer(kCFAllocatorDefault,
                                                 pixelBuffer,
                                                 formatDescription,
                                                 &kCMTimingInfoInvalid,
                                                 &sampleBuffer) != noErr) {
        [NSException raise:NSInternalInconsistencyException
                    format:@"Failure in CMSampleBufferCreateReadyWithImageBuffer"];
        return;
    }

    //Set attachment 'kCMSampleBufferAttachmentKey_PostNotificationWhenConsumed' - will only post a notification when running in simulator, not when running on a real device
    CFDictionaryRef userInfo = CFDictionaryCreate(NULL, NULL, NULL, 0, NULL, NULL);
    CMSetAttachment(sampleBuffer, kCMSampleBufferAttachmentKey_PostNotificationWhenConsumed,
                    userInfo, kCMAttachmentMode_ShouldPropagate);
    CFRelease(userInfo);


    //Set attachment 'kCMSampleAttachmentKey_DisplayImmediately'
    CFArrayRef attachmentsArray = CMSampleBufferGetSampleAttachmentsArray(sampleBuffer, true);
    CFMutableDictionaryRef attachments = (CFMutableDictionaryRef)CFArrayGetValueAtIndex(attachmentsArray, 0);
    CFDictionarySetValue(attachments, kCMSampleAttachmentKey_DisplayImmediately, kCFBooleanTrue);

    [self.sampleBufferView.sampleBufferDisplayLayer enqueueSampleBuffer:sampleBuffer];

    CVPixelBufferRelease(pixelBuffer);
    CFRelease(formatDescription);
    CFRelease(sampleBuffer);
}

@end

Comments

Response from Apple

iOS engineering has conducted a preliminary investigation of this issue, and reported the following via your bug report:

"I do not think this is expected to work. It happens to work on the simulator because the developer is listening for kCMSampleBufferConsumerNotification_BufferConsumed globally. When the notification is posted within the same process, they can receive it, but this does not propagate across processes."


Please note: Reports posted here will not necessarily be seen by Apple. All problems should be submitted at bugreport.apple.com before they are posted here. Please only post information for Radars that you have filed yourself, and please do not include Apple confidential information in your posts. Thank you!