AVFoundation

RSS for tag

Work with audiovisual assets, control device cameras, process audio, and configure system audio interactions using AVFoundation.

Posts under AVFoundation tag

200 Posts

Post

Replies

Boosts

Views

Activity

AVPlayer reports unexpected ~26-hour timeline
Hello, We are seeing an issue where AVPlayer sometimes reports time values close to 95,443 seconds (~26.5 hours), even though the actual content duration is much shorter. For example, in normal cases we see: currentTime: ~355 seconds duration: 6958 seconds seekableTimeRange: 0...6958 seconds However, in some cases AVPlayer reports values around: currentTime: ~95443 seconds duration: ~95449 seconds We checked the underlying MPEG-TS segments with ffprobe. The video and audio PTS values are around 6426 seconds and are consistent with the HLS EXTINF durations, so we do not see a 95,443-second timestamp directly in the media segments. Could you please help us understand: 1- Why can AVPlayer expose values close to 95,443 seconds? 2- Could this be related to MPEG-TS PTS wrap-around or AVFoundation timeline mapping? 3- How does AVPlayer calculate currentTime and AVPlayerItem.duration for HLS MPEG-TS content? Interestingly, Audio Only / Now Playing mode shows the expected timeline, while video playback can expose the incorrect values. Thank you.
0
0
126
4h
iPad (9th gen) headphone output ~2.5 to 4.5 dB quieter on iPadOS 27 (FB24980117)
Since iPadOS 27, the 3.5 mm headphone output on iPad (9th generation) is 2.4 to 4.5 dB quieter than on iPadOS 26, depending on frequency, with the same content, system volume and headphones. It's still present in iPadOS 27.2 beta 2 (24B5089g). It isn't app-specific. It reproduces in Safari: Connect wired headphones to the 3.5 mm jack and set system volume to 100%. Open https://www.szynalski.com/tone-generator/, set 3000 Hz, set the generator volume to 50%, and play. Measure with a sound level meter. At 3 kHz we see 2.5 to 2.9 dB less on iPadOS 27 than on iPadOS 26, across three different headphone models. Audio settings are identical between the two OS versions and Bluetooth is off. The only session-level difference we found is AVAudioSession.outputLatency, which went from 97 to 65 frames. Filed as FB24980117.
0
0
223
11h
Can SensitiveContentAnalysis gate FairPlay-protected on-demand video before presentation?
Is there a supported public API on iOS/iPadOS 27 to apply SensitiveContentAnalysis to FairPlay Streaming-protected, on-demand HLS playback in AVPlayer, so that sensitive video is withheld before presentation, including after seeking or resuming, while preserving FairPlay protection? The intended behavior is on-device analysis of the actual protected presentation, with playback held until the required safety decision is available. This includes preventing the first sensitive frame from appearing and applying the corresponding audio intervention. The app must not obtain raw content keys or create a decrypted copy or clear proxy for analysis. I reviewed the public APIs and the iPhoneOS 27.0 SDK headers in Xcode 27.0 (27A266a): SCSensitivityAnalyzer's video-analysis entry point accepts a file URL; I have not found an entry point that attaches to an AVContentKeySession or a protected AVPlayerItem. SCVideoStreamAnalyzer is documented in terms of conference-call participants and incoming/outgoing streams. It accepts a capture-device input, a VTDecompressionSession, or individual pixel buffers. Its decompression-session path is documented to produce blank frames when sensitive content is detected, but I have not found a supported way to attach it to AVPlayer's FairPlay-protected decoding path. SCVideoStreamAnalyzer's initializer throws if both Communication Safety and Sensitive Content Warnings are off. Analysis would only be required when the relevant device policy is enabled. AVPlayer.h, lines 853–856, documents that allowsCaptureOfClearKeyVideo defaults to NO and has no effect on FairPlay Streaming content. I therefore cannot treat that property as a route to analyze protected frames through AVPlayerItemVideoOutput. Is SCVideoStreamAnalyzer supported for this protected on-demand use case? If so, which API connects it to the protected playback path, and what guarantees cover withholding sensitive frames and coordinating audio at initial playback, seek, and resume? If the requirement is unsupported with public APIs, confirmation of that limitation would help determine the product architecture. This is an API feasibility question based on documentation and header inspection. I do not yet have a focused reproduction project or a physical-device result, and I am not reporting a reproduced framework failure. References: https://developer.apple.com/documentation/sensitivecontentanalysis/scvideostreamanalyzer https://developer.apple.com/documentation/sensitivecontentanalysis/scsensitivityanalyzer/videoanalysishandler https://developer.apple.com/documentation/avfoundation/avplayer/allowscaptureofclearkeyvideo
0
0
228
1d
Does activeLockedVideoFrameDuration limit fully manual exposure to half a frame?
On an iPhone 18 Pro running iOS 27, I’m seeing an interaction between locked frame timing and custom exposure. I configure the wide camera for 4K at 29.97 fps: input.activeLockedVideoFrameDuration = CMTime(value: 1001, timescale: 30000) With aperture and ISO fixed, I set exposure using the iOS 27 setExposureModeCustom(lensAperture:duration:iso:completionHandler:) API. A controlled comparison produced: 10 ms requested: timing lock ON → 10.000 ms; OFF → 10.000 ms. 16 ms requested: timing lock ON → 16.129 ms; OFF → 16.129 ms. 20 ms requested: timing lock ON → 16.667 ms; OFF → 20.000 ms. “OFF” means ordinary minimum and maximum frame durations both set to 1001/30000 seconds. These are exposure values from per-frame camera metadata after valid completion timestamps and a settling period. The behaviour occurs both idle and during recording. With the timing lock enabled, delivered frame timestamps show exact 30000/1001 pacing. Without it, fully manual capture measured approximately 29.964 fps. Separately, with shutter fixed at 20 ms but ISO automatic, the locked-timing configuration held 20 ms and exact 29.97. That suggests exposure ownership affects the behaviour. The documentation (https://developer.apple.com/documentation/avfoundation/avcapturedeviceinput/activelockedvideoframeduration) recommends exposure durations of half a frame or less to maintain full dynamic range, but doesn’t explicitly describe a duration clamp. Is this expected behaviour? Does fully manual exposure impose a half-frame ceiling when locked frame timing is active? Is there an API to query that effective limit? The format’s advertised exposure range includes 20 ms. Is there a supported way to retain exact fractional frame timing and a longer fully manual exposure? I can provide a minimal reproducer and device diagnostics.
0
0
51
1d
Possible iOS 27 Simulator bug with NowPlaying: metadata only appears after second lock cycle, playback buttons missing
Hi, I’m testing the new NowPlaying framework on iOS 27 using Xcode 27 and an iPhone 17 Simulator, and I’m seeing some behavior that I’m trying to determine is a Simulator issue or something incorrect in my implementation. I have a local AVPlayer playing an MP3 and a MediaSession conforming to MediaSessionRepresentable. The session publishes MusicContent with static artwork, AnimatedArtwork using the .tall aspect ratio, a MediaPlaybackSnapshot, and commands for play, pause, previous, next, and seek. The strange behavior is: I start playback and call requestToBecomeApplicationPrimary(). It succeeds and isApplicationPrimary becomes true, but isSystemPrimary remains false. I lock the Simulator for the first time. There is no Now Playing UI or metadata. I unlock it and return to the app. After recreating/re-registering the MediaSession and calling requestToBecomeApplicationPrimary() again, isSystemPrimary sometimes becomes true. When I lock the Simulator a second time, the Now Playing UI finally appears and the animated 3:4 artwork works correctly. However, the Lock Screen card still does not show the expected play/pause/previous/next buttons. It only shows the title, artist, progress bar, and volume control, despite those commands being included in commands. When the second session succeeds, the framework also starts requesting the artwork as expected: applicationPrimary: true systemPrimary: true Animated preview requested: (1206, 1608) ratio: tall Static artwork requested Animated video requested: (1206, 1608) ratio: tall Calling requestToBecomeSystemPrimary() directly has also consistently returned: internalFailure Playback itself continues correctly throughout all of this. So I’m trying to understand whether anyone else is seeing similar behavior with the iOS 27 Simulator: MediaSession becoming application-primary but not system-primary on the initial playback Now Playing metadata only appearing after a lock/unlock/re-registration cycle media commands being supplied but playback buttons missing from the Lock Screen requestToBecomeSystemPrimary() returning internalFailure I have not yet tested the same build on a physical iOS 27 device, so I’m particularly interested in whether this is currently a known Simulator limitation/bug or whether there is another step required when publishing a local playback MediaSession. Thanks!
0
0
273
3d
CMLogItem.timestamp: which clock is it on, and how to convert to AVCaptureSession.synchronizationClock / host time?
We receive live relative-altitude updates from CMAltimeter.startRelativeAltitudeUpdates(to:withHandler:) on iPhone. CMAltitudeData inherits CMLogItem.timestamp, which the documentation describes as the time when the item is valid and as seconds since device boot. We need to determine whether a pressure measurement's event time falls inside an application operation bounded by a supported monotonic clock. Could Apple clarify the supported contract for CMAltitudeData.timestamp on iOS? Does CMAltitudeData.timestamp use the same epoch and rate as mach_absolute_time(), DispatchTime.uptimeNanoseconds, CACurrentMediaTime(), or ProcessInfo.systemUptime? If only some are compatible, which ones? Does it advance or pause during device sleep, device lock, application suspension, and background execution? In particular, are its sleep/suspension semantics guaranteed to match any of the clocks above? Is there a supported API for converting CMLogItem.timestamp to a host CMClock/Mach time, or for sampling “now” in the exact same clock domain used by CMAltitudeData.timestamp? Are the answers contractual across supported iPhone hardware and iOS releases, or are they implementation details that applications should not rely on? If direct comparison is not supported, what Apple-supported clock or conversion mechanism should an application use to compare a CMAltitudeData event time with two application-side monotonic operation boundaries? The question concerns clock semantics only. A minimal reproducer can be supplied if requested, but no application identifier, production data, sensor values, or user data is required to answer it.
6
0
434
3d
iPhone Duo: Does CameraCaptureAccessory accept touch, what counts as an active camera session, and can both front cameras run at once?
After watching "Leverage multiple displays and scenes on iPhone Duo" and "Build a great camera experience for iPhone Duo", we have four questions about the outer display while the device is open: Does a view hosted in CameraCaptureAccessory receive touch input on the outer display, or is it display-only? The accessory is described as available when the app "has an active camera session." Does that require a running preview layer or an attached output (photo, movie or data), or is a running AVCaptureSession with an input enough? Is a session whose main purpose is to host the accessory a supported use? Can AVCaptureMultiCamSession run the inner (under-display) front camera and the outer front camera at the same time on iPhone Duo? If so, which device-type combinations are supported, and are there thermal or power limits? Is CameraCaptureAccessory the only way for an app to show content on the outer display while the inner display is live, or are scene accessories that don't depend on the camera planned? Our use case is not photography. We're exploring a companion view for a second person sitting across from the device owner, so we want to understand whether the camera accessory is the right tool before designing around it.
4
0
91
5d
Camera direction and rotation in the half-folded pose on iPhone Duo
"Build a great camera experience for iPhone Duo" covers the device open and the device closed, including reconfiguring the session from AVCaptureDeviceDirectionCoordinator's change handler when the forward-facing camera changes. It does not cover the partially open pose, which "Leverage multiple displays and scenes on iPhone Duo" treats as a first-class hinge state with a continuous angle. Consider a camera app that keeps a live preview on screen and adopts AVCaptureDevice.RotationCoordinator — the documented remedy for cases like the square front sensor on iPhone 17, where a hard-coded 90° leaves the preview tilted — applying videoRotationAngleForHorizonLevelPreview to both the preview and the capture connection. In the partially open pose the two halves are at different physical attitudes: one stands up, the other lies flat. Which camera does AVCaptureDeviceDirectionCoordinator report as forward-facing while the hinge status is partially open? Does the hinge angle affect that decision? RotationCoordinator derives its angle from device attitude. With the two halves at different attitudes, which one does videoRotationAngleForHorizonLevelPreview reflect — the half carrying the active camera, the half hosting the preview layer, or a single device-level reference? If an app deliberately keeps the rear camera on the scene while its preview is drawn on the half facing the user, is that a supported configuration, or is the coordinator expected to override that choice? Is any of this modelled in the iPhone Duo simulator, or does the partially open pose require real hardware to test? Question 4 matters for planning: building against a rotation value that only appears on device would be worth knowing about before the hardware ships.
4
0
124
5d
Simultaneous capture and fold transitions in a live-streaming app on iPhone Duo
I build Lumecast, a live-streaming app (custom AVCapture pipeline feeding WebRTC). Three questions:   1. Does iPhone Duo support AVCaptureMultiCamSession — specifically front + rear simultaneously? Our co-casting feature would let a solo host fill two panes with their face and what they're looking at. How should a continuous capture session (live broadcast, can't pause) respond to camera direction changes as the device opens and closes? Is there a recommended pattern for swapping to the Virtual Front Camera mid-session without a visible gap for viewers?  3. Which cameras are available to third-party apps in each posture (folded / half-open / fully open), and does availability change at a specific hinge state? Goal: a host folds or unfolds mid-broadcast and their audience never sees a dropped frame.
3
0
133
5d
Is AVCaptureMultiCamSession supported on iPhone Duo
is AVCaptureMultiCamSession supported on iPhone Duo — specifically streaming two cameras that face opposite directions simultaneously (e.g. .builtInInnerUltraWideCamera + .builtInOuterUltraWideCamera, or the Virtual Front Camera plus a backward-facing camera)? I noticed the article recommends single-input reconfiguration over multi-cam; is that a performance recommendation, or is opposite-direction multi-cam unavailable on this device? Our use case is a live co-cast where one host fills two video panes: their face and what they're looking at.
0
0
44
5d
Where is the API for variable aperture of the iPhone 18 Pro?
According to Apple's press release (https://www.apple.com/pt/newsroom/2026/09/apple-debuts-iphone-18-pro-and-iphone-18-pro-max/): Variable aperture also provides more control for creative pros by giving them the ability to manually adjust any of the four aperture settings in the Camera app, and an API is available to developers for even more control across the aperture range in their apps. Can a DTS engineer specify the quoted API please? I can't see it anywhere in the documentation yet. What does it mean "even more control across the aperture range"? Will it be possible to set the aperture in a 0.0-1.0 range?
3
3
995
5d
Picture-In-Picture not starting with Custom Player
Team, I am trying to start PictureInPicture for my video from custom player as mentioned in the following link, but I am getting an error Steps: Created a player using AVPlayer. Initiated AVPictureInPictureController with AVPlayerLayer Added observer on PIP button enable status When PIP button is tapped executed pipController.startPictureInPicture() statement. PIP is not getting started & getting following error Project created in Xcode11 & currently using Xcode13.2 Error Domain=AVKitErrorDomain Code=-1001 "Failed to start picture in picture." UserInfo={NSLocalizedDescription=Failed to start picture in picture., NSLocalizedFailureReason=The UIScene for the content source has an activation state other than UISceneActivationStateForegroundActive, which is not allowed.} I didn't get any solutions online, plz help me on this issue. FYI: PIP button/mode is working if I use AVPlayerViewController(with default playerItem buttons) instead of AVPlayer & AVPictureInPictureController with custom controls Thanks in advance. Adopting Picture in Picture in a Custom Player
3
2
1.7k
6d
`LockedCameraCaptureManager` practically unusable since iOS 26
Somewhere since iOS 26, the LockedCameraCapture framework gets in an unpredictable state after opening the main app from the LockedCamera extension using LockedCameraCaptureSession.openApplication(for userActivity:). (Feedback with sample code to reproduce: FB21966835) Opening the extension from the lock screen again doesn’t open the extension but puts the lock screen in a state as if it has. Content updated from LockedCameraCaptureManager.shared.sessionContentUpdates comes in inconsistently, usually needs the app to be opened again or the extension to be opened. This makes using this extension impossible for me as I use it to record video files that manually need to be imported when the app is launched (so not through PhotoKit). Does anybody have a suggestion to circumvent this issue or how to get this fixed?
2
3
1.2k
6d
AVPlayerItemSampleBufferOutput
I am using AVPlayer, but I am unable to retrieve PCM data from .m3u8 streams via the AVPlayerItemSampleBufferOutput API in iOS 27 beta. I have tried both the Objective-C and Swift APIs: the Objective-C delegate methods are not being called, and the Swift methods nextAvailableSampleBuffer() and nextSampleBuffer() are returning no data.
2
1
672
1w
fail to get HLS realtime stream via AVPlayerItemSampleBufferOutputDelegate
Hi, I'm trying to do some realtime audio processing on audio served from an HLS stream (i.e. an AVPlayer created using an M3U HTTP URL). And I find that new APIs(AVPlayerItemSampleBufferOutput) are available in iOS 27.0 to achieve it. However, I failed to get the available data via the AVPlayerItemSampleBufferOutputDelegate. And I found some error in the system log: I create AVPlayerItemSampleBufferOutput and set the delegate after receiving the AVPlayerItemStatusReadyToPlay event. And here's my code: @interface OCAudioSamplebuffer () <AVPlayerItemSampleBufferOutputDelegate> @property (nonatomic, strong) AVPlayerItemSampleBufferOutput *bufferOutput; @property (nonatomic, strong) dispatch_queue_t bufferOutputQueue; @property (nonatomic, strong) AVPlayerItem *playerItem; @end - (void)playItem:(AVPlayerItem *)item { if (@available(iOS 27, *)) { AVAudioSession *audioSession = [AVAudioSession sharedInstance]; if([NSThread mainThread]){ [audioSession setCategory:AVAudioSessionCategoryPlayback withOptions:AVAudioSessionCategoryPlayback error:nil]; [audioSession setActive:YES error:nil]; }else{ dispatch_async(dispatch_get_main_queue(), ^{ [audioSession setCategory:AVAudioSessionCategoryPlayback withOptions:AVAudioSessionCategoryPlayback error:nil]; [audioSession setActive:YES error:nil]; }); } AVPlayerItemSampleBufferOutputAudioConfiguration *cfg = [[AVPlayerItemSampleBufferOutputAudioConfiguration alloc] init]; CMFormatDescriptionRef formatDescription = [self createPCMFormatDescriptionWithSampleRate:44100.0 channels:2 isFloat:YES]; cfg.requestedAudioFormat = formatDescription; if (formatDescription) { NSLog(@"create PCM Format Description success"); CFRelease(formatDescription); } else { NSLog(@"fail to create PCM Format Description"); } self.bufferOutput = [[AVPlayerItemSampleBufferOutput alloc] initWithConfiguration:cfg]; NSLog(@"create buffer success, PCM Format Description%@---,%@", cfg.requestedAudioFormat, formatDescription); self.bufferOutputQueue = dispatch_queue_create("audioSamplebufferQueue", DISPATCH_QUEUE_CONCURRENT); [self.bufferOutput setDelegate:self queue:self.bufferOutputQueue]; [item addOutput:self.bufferOutput]; self.playerItem = item; } else { // Fallback on earlier versions } } Any help. Thank you
2
0
937
1w
iOS 27 beta: Opening Notification Center pauses AVPlayer playback
Since iOS 27 beta we are seeing a behavior change in our video streaming app and I would like to know whether others can reproduce it. Behavior on iOS 27 beta: Fully opening the Notification Center pauses playback. Audio continues for about 5 more seconds, then the app is suspended. Closing the Notification Center leaves the player paused. On iOS 26 the same build keeps playing in this situation. Setup: AVQueuePlayer playing video with audio, not muted audiovisualBackgroundPlaybackPolicy = .automatic (default) AVAudioSession category .playback, UIBackgroundModes: audio entitlement AVPictureInPictureController attached with canStartPictureInPictureAutomaticallyFromInline = true Filed as FB23893965 Questions: Can anyone reproduce this on iOS 27 beta? Is this intentional or a regression?
1
3
843
1w
How can an app detect a TrueDepth pipeline failure when no depth output or drop callback is delivered?
How can an app detect a TrueDepth pipeline failure when no depth output or drop callback is delivered? We are using AVFoundation to stream synchronized RGB and depth data from the front-facing TrueDepth camera. On a small number of physical devices, the capture session and RGB stream remain healthy, but the depth pipeline becomes silent or returns unusable data. We would like to understand the intended public-API behavior and the supported way to detect this state without relying on private system logs or an application-defined timeout. We need an AVFoundation-only solution rather than ARKit. Capture configuration Our setup follows Apple's TrueDepth streaming sample: Discover .builtInTrueDepthCamera. Select an activeFormat with a nonempty supportedDepthDataFormats list. Set a supported activeDepthDataFormat, typically DepthFloat16. Add AVCaptureVideoDataOutput and AVCaptureDepthDataOutput. Enable the .depthData connection. Test both direct depth delegate delivery and AVCaptureDataOutputSynchronizer. Observe capture-session runtime errors, interruptions, connection state, and system pressure. The same code and configuration continuously deliver valid RGB and depth frames on healthy control devices. Observed failure modes 1. No depth callback of any kind The video delegate continues to receive RGB frames at the expected rate. The depth connection reports enabled and active. The capture session remains running. No AVCaptureSessionRuntimeErrorNotification or interruption is reported. System pressure is nominal. depthDataOutput(_:didOutput:timestamp:connection:) is never called. depthDataOutput(_:didDrop:timestamp:connection:reason:) is also never called. With AVCaptureDataOutputSynchronizer, RGB synchronization points may continue, but no AVCaptureSynchronizedDepthData object is present for the depth output. 2. One depth frame, followed by permanent silence On other affected devices, one depth frame is delivered shortly after startup, while hundreds of later RGB frames continue with no further depth output or drop callback. 3. A depth object is delivered, but the map is unusable We have also observed an AVDepthData object whose map is empty or has no usable spatial information, for example: a zero-sized/empty depth representation; an unfiltered map containing only NaN; or a filtered map containing a single constant value across nearly all pixels. Metadata such as depth quality, accuracy, calibration availability, supported formats, and connection state does not reliably distinguish these frames from healthy ones. Relevant documented behavior The documentation for AVCaptureSynchronizedDepthData.depthDataWasDropped says that a dropped depth object is different from a synchronization timestamp for which no depth capture occurred. In the latter case, no AVCaptureSynchronizedDepthData object is present in the collection. The documentation for depthDataOutput(_:didDrop:timestamp:connection:reason:) says that the callback reports captured depth data that wasn't processed, with an empty-shell AVDepthData object. This appears to leave a diagnostic gap: if the TrueDepth pipeline fails before a depth object is produced, the application may receive neither an output callback nor a drop callback. System-log correlation We do not want to depend on these private and undocumented strings, but sysdiagnose/device logs from affected devices correlate the public-API silence with messages such as: Can't activate Pearl projector: no projector token found PROJECTOR ON DENIED Object too close (no depth) GMC status: -2 Projector GMC hasn't completed yet - dropping depth/dx buffers Pulses exceed limit. Projector Off In the first group, depth delivery never starts. In the second group, depth may never start or may stop after one frame. RGB capture can remain active in both cases. These logs suggest that depth or disparity buffers can be rejected before they reach AVCaptureDepthDataOutput, which would explain why the application sees neither didOutput nor didDrop. However, none of these projector, IR, calibration, protection, or token states appears to be exposed by a public API. Questions Is it expected that AVCaptureDepthDataOutput delivers neither didOutput nor didDrop when no depth object is produced upstream? Is the absence of AVCaptureSynchronizedDepthData from a synchronization collection the only public signal for this condition? Is there a supported API, notification, KVO property, or error that reports that the TrueDepth IR/projector/depth provider is unavailable, disabled, protected, stalled, or unable to produce data? Can an app distinguish a healthy but temporarily delayed depth stream from a pipeline that will never deliver depth, other than by implementing its own timeout and frame-count watchdog? Is there a documented validity rule for an empty, all-NaN, or near-constant AVDepthData.depthDataMap? Should applications treat these as unavailable depth even when the AVDepthData metadata reports a normal quality or accuracy? What is the recommended recovery procedure: rebuild AVCaptureDepthDataOutput, restart the capture session, switch cameras and switch back, wait for a system notification, or stop retrying until the device state changes? ARKit can report ARError.sensorFailed when a required sensor does not deliver input. Is there an AVFoundation equivalent that identifies which required input failed, without adopting ARKit? If no public signal currently exists, would Apple consider exposing a depth-provider status/error callback that distinguishes at least: captured but dropped; no depth capture for the timestamp; provider temporarily unavailable; and provider disabled or failed until recovery/service? We can provide a minimal sample project, affected-device sysdiagnose, and synchronized timestamp traces in Feedback Assistant if that would help. Please let us know which diagnostics and logging profiles would be most useful.
0
0
99
1w
AVCaptureVideoDataOutput stops zooming while AVCaptureVideoPreviewLayer continues — physical wide / ultra-wide / telephoto only
We use a single AVCaptureSession with AVCaptureVideoPreviewLayer and AVCaptureVideoDataOutput (preview-sized buffers, BGRA). When we increase videoZoomFactor, beyond a certain zoom level the image from AVCaptureVideoDataOutput no longer zooms further, while AVCaptureVideoPreviewLayer continues to zoom with the same zoom control. The preview and the video-data output therefore diverge. This behavior appears when the active camera is a physical lens device — wide, ultra-wide, or telephoto (e.g. builtInWideAngleCamera, builtInUltraWideCamera, builtInTelephotoCamera, or similar). It does not appear when the active input is a virtual / multi-camera (e.g. triple camera, dual-wide, or other system multi-camera). Are there known conditions under which this mismatch between preview and video-data output is expected? Thank you.
2
0
896
2w
AVAssetDownloadConfiguration does not download supplemental tracks on iOS 16
We're downloading HLS content for offline playback using AVAssetDownloadConfiguration. It's critical for our app that all subtitle tracks are downloaded together with the video, so users can watch any subtitle and audio option while offline. On iOS 16, not all tracks are downloaded, and when the user goes offline, playback fails with NSURLErrorNotConnectedToInternet (error -1009) because AVFoundation tries to fetch the missing subtitle segments over the network. On iOS 17 and later, everything works as expected — all subtitle and audio tracks are downloaded. However, on iOS 16 we observe two issues: First issue: Supplemental tracks are not downloaded on iOS 16 (device) Only the default audio track is downloaded. All non-default supplemental tracks — both subtitles and additional audio tracks — exhibit the same problem: Some supplemental tracks listed in the manifest don't even get a directory created in the .movpkg package Some supplemental tracks do get a directory created, but it contains only StreamInfoBoot.xml and StreamInfoRoot.xml — no actual segment data (.frag files) is downloaded. The StreamInfoBoot.xml for these directories shows NO, 0, and 0, with Supplemental Additionally, a Manifest.m3u8 stream directory is created with Complete=NO and no segment data. On iOS 17+ devices, all supplemental track directories are created and all contain segment data. Second issue: Downloads fail entirely on iOS 16 Simulator We receive the following error: Error Domain=NSURLErrorDomain Code=-16090 UserInfo={ NSLocalizedDescription=The operation could not be completed, _NSURLErrorFailingURLSessionTaskErrorKey=BackgroundAVAssetDownloadTask <DEC0D555-6C05-416C-91B8-F490F5A29046>.<3>, _NSURLErrorRelatedURLSessionTaskErrorKey=( "BackgroundAVAssetDownloadTask <DEC0D555-6C05-416C-91B8-F490F5A29046>.<3>" ), NSLocalizedFailureReason=An unknown error occurred (-16090) } On iOS 17+ Simulators, downloads work without issues. Our setup: let urlAsset = AVURLAsset(url: manifestURL) let preferredMediaSelection = try await urlAsset.load(.allMediaSelections) let config = AVAssetDownloadConfiguration(asset: urlAsset, title: title) config.primaryContentConfiguration.mediaSelections = preferredMediaSelection let task = assetDownloadURLSession.makeAssetDownloadTask(downloadConfiguration: config) task.resume() Example HLS manifest: #EXTM3U #EXT-X-VERSION:7 #EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="French",LANGUAGE="fr",DEFAULT=NO,AUTOSELECT=YES,FORCED=NO,URI="s_25/stream.m3u8" #EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="French (forced)",LANGUAGE="fr-x-cc",DEFAULT=NO,AUTOSELECT=YES,FORCED=YES,URI="s_31/stream.m3u8" #EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="English",LANGUAGE="en",DEFAULT=NO,AUTOSELECT=YES,FORCED=NO,URI="s_37/stream.m3u8" #EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="Kazakh",LANGUAGE="kk",DEFAULT=NO,AUTOSELECT=YES,FORCED=NO,URI="s_43/stream.m3u8" #EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="Uzbek",LANGUAGE="uz",DEFAULT=NO,AUTOSELECT=YES,FORCED=NO,URI="s_49/stream.m3u8" #EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="aud_ec3",NAME="French 5.1",LANGUAGE="fr",DEFAULT=YES,AUTOSELECT=YES,CHANNELS="6",URI="a_18/stream.m3u8" #EXT-X-STREAM-INF:AUDIO="aud_ec3",SUBTITLES="subs",BANDWIDTH=1931408,VIDEO-RANGE=SDR,CODECS="hvc1.1.6.L90.90,ec-3",RESOLUTION=1024x428,FRAME-RATE=25,HDCP-LEVEL=NONE v_6/stream.m3u8 #EXT-X-SESSION-KEY:METHOD=SAMPLE-AES,URI="skd://XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX",KEYFORMAT="com.apple.streamingkeydelivery",KEYFORMATVERSIONS="1" Questions: What is the correct way to make sure all supplemental tracks (subtitles, non-default audio) are actually downloaded on an iOS 16 real device? We pass asset.load(.allMediaSelections) to primaryContentConfiguration.mediaSelections — is there a different configuration, or should we use auxiliaryContentConfigurations, or something else entirely? Is there any way to make asset downloads work on the iOS 16 Simulator at all? We always get NSURLErrorDomain error -16090, while iOS 17+ Simulators work fine. If the supplemental tracks download bug on iOS 16 cannot be fixed or worked around from the app side, do you have any recommended approach for handling incomplete .movpkg packages at offline playback time?
1
0
393
2w
Custom AVVideoCompositing on a composition-backed AVPlayerItem fails with AVErrorUnknown Xcode 27 beta 2 / beta 3
Trivial pass-through compositor fails on Xcode 27 (beta 2, beta 3); error code -11800 underlying error -12784. Repro included https://github.com/BugorBN/avplayer-custom-compositor-repro It works well on Xcode26 and lower
Replies
4
Boosts
3
Views
857
Activity
1h
AVPlayer reports unexpected ~26-hour timeline
Hello, We are seeing an issue where AVPlayer sometimes reports time values close to 95,443 seconds (~26.5 hours), even though the actual content duration is much shorter. For example, in normal cases we see: currentTime: ~355 seconds duration: 6958 seconds seekableTimeRange: 0...6958 seconds However, in some cases AVPlayer reports values around: currentTime: ~95443 seconds duration: ~95449 seconds We checked the underlying MPEG-TS segments with ffprobe. The video and audio PTS values are around 6426 seconds and are consistent with the HLS EXTINF durations, so we do not see a 95,443-second timestamp directly in the media segments. Could you please help us understand: 1- Why can AVPlayer expose values close to 95,443 seconds? 2- Could this be related to MPEG-TS PTS wrap-around or AVFoundation timeline mapping? 3- How does AVPlayer calculate currentTime and AVPlayerItem.duration for HLS MPEG-TS content? Interestingly, Audio Only / Now Playing mode shows the expected timeline, while video playback can expose the incorrect values. Thank you.
Replies
0
Boosts
0
Views
126
Activity
4h
iPad (9th gen) headphone output ~2.5 to 4.5 dB quieter on iPadOS 27 (FB24980117)
Since iPadOS 27, the 3.5 mm headphone output on iPad (9th generation) is 2.4 to 4.5 dB quieter than on iPadOS 26, depending on frequency, with the same content, system volume and headphones. It's still present in iPadOS 27.2 beta 2 (24B5089g). It isn't app-specific. It reproduces in Safari: Connect wired headphones to the 3.5 mm jack and set system volume to 100%. Open https://www.szynalski.com/tone-generator/, set 3000 Hz, set the generator volume to 50%, and play. Measure with a sound level meter. At 3 kHz we see 2.5 to 2.9 dB less on iPadOS 27 than on iPadOS 26, across three different headphone models. Audio settings are identical between the two OS versions and Bluetooth is off. The only session-level difference we found is AVAudioSession.outputLatency, which went from 97 to 65 frames. Filed as FB24980117.
Replies
0
Boosts
0
Views
223
Activity
11h
Can SensitiveContentAnalysis gate FairPlay-protected on-demand video before presentation?
Is there a supported public API on iOS/iPadOS 27 to apply SensitiveContentAnalysis to FairPlay Streaming-protected, on-demand HLS playback in AVPlayer, so that sensitive video is withheld before presentation, including after seeking or resuming, while preserving FairPlay protection? The intended behavior is on-device analysis of the actual protected presentation, with playback held until the required safety decision is available. This includes preventing the first sensitive frame from appearing and applying the corresponding audio intervention. The app must not obtain raw content keys or create a decrypted copy or clear proxy for analysis. I reviewed the public APIs and the iPhoneOS 27.0 SDK headers in Xcode 27.0 (27A266a): SCSensitivityAnalyzer's video-analysis entry point accepts a file URL; I have not found an entry point that attaches to an AVContentKeySession or a protected AVPlayerItem. SCVideoStreamAnalyzer is documented in terms of conference-call participants and incoming/outgoing streams. It accepts a capture-device input, a VTDecompressionSession, or individual pixel buffers. Its decompression-session path is documented to produce blank frames when sensitive content is detected, but I have not found a supported way to attach it to AVPlayer's FairPlay-protected decoding path. SCVideoStreamAnalyzer's initializer throws if both Communication Safety and Sensitive Content Warnings are off. Analysis would only be required when the relevant device policy is enabled. AVPlayer.h, lines 853–856, documents that allowsCaptureOfClearKeyVideo defaults to NO and has no effect on FairPlay Streaming content. I therefore cannot treat that property as a route to analyze protected frames through AVPlayerItemVideoOutput. Is SCVideoStreamAnalyzer supported for this protected on-demand use case? If so, which API connects it to the protected playback path, and what guarantees cover withholding sensitive frames and coordinating audio at initial playback, seek, and resume? If the requirement is unsupported with public APIs, confirmation of that limitation would help determine the product architecture. This is an API feasibility question based on documentation and header inspection. I do not yet have a focused reproduction project or a physical-device result, and I am not reporting a reproduced framework failure. References: https://developer.apple.com/documentation/sensitivecontentanalysis/scvideostreamanalyzer https://developer.apple.com/documentation/sensitivecontentanalysis/scsensitivityanalyzer/videoanalysishandler https://developer.apple.com/documentation/avfoundation/avplayer/allowscaptureofclearkeyvideo
Replies
0
Boosts
0
Views
228
Activity
1d
Does activeLockedVideoFrameDuration limit fully manual exposure to half a frame?
On an iPhone 18 Pro running iOS 27, I’m seeing an interaction between locked frame timing and custom exposure. I configure the wide camera for 4K at 29.97 fps: input.activeLockedVideoFrameDuration = CMTime(value: 1001, timescale: 30000) With aperture and ISO fixed, I set exposure using the iOS 27 setExposureModeCustom(lensAperture:duration:iso:completionHandler:) API. A controlled comparison produced: 10 ms requested: timing lock ON → 10.000 ms; OFF → 10.000 ms. 16 ms requested: timing lock ON → 16.129 ms; OFF → 16.129 ms. 20 ms requested: timing lock ON → 16.667 ms; OFF → 20.000 ms. “OFF” means ordinary minimum and maximum frame durations both set to 1001/30000 seconds. These are exposure values from per-frame camera metadata after valid completion timestamps and a settling period. The behaviour occurs both idle and during recording. With the timing lock enabled, delivered frame timestamps show exact 30000/1001 pacing. Without it, fully manual capture measured approximately 29.964 fps. Separately, with shutter fixed at 20 ms but ISO automatic, the locked-timing configuration held 20 ms and exact 29.97. That suggests exposure ownership affects the behaviour. The documentation (https://developer.apple.com/documentation/avfoundation/avcapturedeviceinput/activelockedvideoframeduration) recommends exposure durations of half a frame or less to maintain full dynamic range, but doesn’t explicitly describe a duration clamp. Is this expected behaviour? Does fully manual exposure impose a half-frame ceiling when locked frame timing is active? Is there an API to query that effective limit? The format’s advertised exposure range includes 20 ms. Is there a supported way to retain exact fractional frame timing and a longer fully manual exposure? I can provide a minimal reproducer and device diagnostics.
Replies
0
Boosts
0
Views
51
Activity
1d
Possible iOS 27 Simulator bug with NowPlaying: metadata only appears after second lock cycle, playback buttons missing
Hi, I’m testing the new NowPlaying framework on iOS 27 using Xcode 27 and an iPhone 17 Simulator, and I’m seeing some behavior that I’m trying to determine is a Simulator issue or something incorrect in my implementation. I have a local AVPlayer playing an MP3 and a MediaSession conforming to MediaSessionRepresentable. The session publishes MusicContent with static artwork, AnimatedArtwork using the .tall aspect ratio, a MediaPlaybackSnapshot, and commands for play, pause, previous, next, and seek. The strange behavior is: I start playback and call requestToBecomeApplicationPrimary(). It succeeds and isApplicationPrimary becomes true, but isSystemPrimary remains false. I lock the Simulator for the first time. There is no Now Playing UI or metadata. I unlock it and return to the app. After recreating/re-registering the MediaSession and calling requestToBecomeApplicationPrimary() again, isSystemPrimary sometimes becomes true. When I lock the Simulator a second time, the Now Playing UI finally appears and the animated 3:4 artwork works correctly. However, the Lock Screen card still does not show the expected play/pause/previous/next buttons. It only shows the title, artist, progress bar, and volume control, despite those commands being included in commands. When the second session succeeds, the framework also starts requesting the artwork as expected: applicationPrimary: true systemPrimary: true Animated preview requested: (1206, 1608) ratio: tall Static artwork requested Animated video requested: (1206, 1608) ratio: tall Calling requestToBecomeSystemPrimary() directly has also consistently returned: internalFailure Playback itself continues correctly throughout all of this. So I’m trying to understand whether anyone else is seeing similar behavior with the iOS 27 Simulator: MediaSession becoming application-primary but not system-primary on the initial playback Now Playing metadata only appearing after a lock/unlock/re-registration cycle media commands being supplied but playback buttons missing from the Lock Screen requestToBecomeSystemPrimary() returning internalFailure I have not yet tested the same build on a physical iOS 27 device, so I’m particularly interested in whether this is currently a known Simulator limitation/bug or whether there is another step required when publishing a local playback MediaSession. Thanks!
Replies
0
Boosts
0
Views
273
Activity
3d
CMLogItem.timestamp: which clock is it on, and how to convert to AVCaptureSession.synchronizationClock / host time?
We receive live relative-altitude updates from CMAltimeter.startRelativeAltitudeUpdates(to:withHandler:) on iPhone. CMAltitudeData inherits CMLogItem.timestamp, which the documentation describes as the time when the item is valid and as seconds since device boot. We need to determine whether a pressure measurement's event time falls inside an application operation bounded by a supported monotonic clock. Could Apple clarify the supported contract for CMAltitudeData.timestamp on iOS? Does CMAltitudeData.timestamp use the same epoch and rate as mach_absolute_time(), DispatchTime.uptimeNanoseconds, CACurrentMediaTime(), or ProcessInfo.systemUptime? If only some are compatible, which ones? Does it advance or pause during device sleep, device lock, application suspension, and background execution? In particular, are its sleep/suspension semantics guaranteed to match any of the clocks above? Is there a supported API for converting CMLogItem.timestamp to a host CMClock/Mach time, or for sampling “now” in the exact same clock domain used by CMAltitudeData.timestamp? Are the answers contractual across supported iPhone hardware and iOS releases, or are they implementation details that applications should not rely on? If direct comparison is not supported, what Apple-supported clock or conversion mechanism should an application use to compare a CMAltitudeData event time with two application-side monotonic operation boundaries? The question concerns clock semantics only. A minimal reproducer can be supplied if requested, but no application identifier, production data, sensor values, or user data is required to answer it.
Replies
6
Boosts
0
Views
434
Activity
3d
iPhone Duo: Does CameraCaptureAccessory accept touch, what counts as an active camera session, and can both front cameras run at once?
After watching "Leverage multiple displays and scenes on iPhone Duo" and "Build a great camera experience for iPhone Duo", we have four questions about the outer display while the device is open: Does a view hosted in CameraCaptureAccessory receive touch input on the outer display, or is it display-only? The accessory is described as available when the app "has an active camera session." Does that require a running preview layer or an attached output (photo, movie or data), or is a running AVCaptureSession with an input enough? Is a session whose main purpose is to host the accessory a supported use? Can AVCaptureMultiCamSession run the inner (under-display) front camera and the outer front camera at the same time on iPhone Duo? If so, which device-type combinations are supported, and are there thermal or power limits? Is CameraCaptureAccessory the only way for an app to show content on the outer display while the inner display is live, or are scene accessories that don't depend on the camera planned? Our use case is not photography. We're exploring a companion view for a second person sitting across from the device owner, so we want to understand whether the camera accessory is the right tool before designing around it.
Replies
4
Boosts
0
Views
91
Activity
5d
Camera direction and rotation in the half-folded pose on iPhone Duo
"Build a great camera experience for iPhone Duo" covers the device open and the device closed, including reconfiguring the session from AVCaptureDeviceDirectionCoordinator's change handler when the forward-facing camera changes. It does not cover the partially open pose, which "Leverage multiple displays and scenes on iPhone Duo" treats as a first-class hinge state with a continuous angle. Consider a camera app that keeps a live preview on screen and adopts AVCaptureDevice.RotationCoordinator — the documented remedy for cases like the square front sensor on iPhone 17, where a hard-coded 90° leaves the preview tilted — applying videoRotationAngleForHorizonLevelPreview to both the preview and the capture connection. In the partially open pose the two halves are at different physical attitudes: one stands up, the other lies flat. Which camera does AVCaptureDeviceDirectionCoordinator report as forward-facing while the hinge status is partially open? Does the hinge angle affect that decision? RotationCoordinator derives its angle from device attitude. With the two halves at different attitudes, which one does videoRotationAngleForHorizonLevelPreview reflect — the half carrying the active camera, the half hosting the preview layer, or a single device-level reference? If an app deliberately keeps the rear camera on the scene while its preview is drawn on the half facing the user, is that a supported configuration, or is the coordinator expected to override that choice? Is any of this modelled in the iPhone Duo simulator, or does the partially open pose require real hardware to test? Question 4 matters for planning: building against a rotation value that only appears on device would be worth knowing about before the hardware ships.
Replies
4
Boosts
0
Views
124
Activity
5d
Simultaneous capture and fold transitions in a live-streaming app on iPhone Duo
I build Lumecast, a live-streaming app (custom AVCapture pipeline feeding WebRTC). Three questions:   1. Does iPhone Duo support AVCaptureMultiCamSession — specifically front + rear simultaneously? Our co-casting feature would let a solo host fill two panes with their face and what they're looking at. How should a continuous capture session (live broadcast, can't pause) respond to camera direction changes as the device opens and closes? Is there a recommended pattern for swapping to the Virtual Front Camera mid-session without a visible gap for viewers?  3. Which cameras are available to third-party apps in each posture (folded / half-open / fully open), and does availability change at a specific hinge state? Goal: a host folds or unfolds mid-broadcast and their audience never sees a dropped frame.
Replies
3
Boosts
0
Views
133
Activity
5d
Is AVCaptureMultiCamSession supported on iPhone Duo
is AVCaptureMultiCamSession supported on iPhone Duo — specifically streaming two cameras that face opposite directions simultaneously (e.g. .builtInInnerUltraWideCamera + .builtInOuterUltraWideCamera, or the Virtual Front Camera plus a backward-facing camera)? I noticed the article recommends single-input reconfiguration over multi-cam; is that a performance recommendation, or is opposite-direction multi-cam unavailable on this device? Our use case is a live co-cast where one host fills two video panes: their face and what they're looking at.
Replies
0
Boosts
0
Views
44
Activity
5d
Where is the API for variable aperture of the iPhone 18 Pro?
According to Apple's press release (https://www.apple.com/pt/newsroom/2026/09/apple-debuts-iphone-18-pro-and-iphone-18-pro-max/): Variable aperture also provides more control for creative pros by giving them the ability to manually adjust any of the four aperture settings in the Camera app, and an API is available to developers for even more control across the aperture range in their apps. Can a DTS engineer specify the quoted API please? I can't see it anywhere in the documentation yet. What does it mean "even more control across the aperture range"? Will it be possible to set the aperture in a 0.0-1.0 range?
Replies
3
Boosts
3
Views
995
Activity
5d
Picture-In-Picture not starting with Custom Player
Team, I am trying to start PictureInPicture for my video from custom player as mentioned in the following link, but I am getting an error Steps: Created a player using AVPlayer. Initiated AVPictureInPictureController with AVPlayerLayer Added observer on PIP button enable status When PIP button is tapped executed pipController.startPictureInPicture() statement. PIP is not getting started & getting following error Project created in Xcode11 & currently using Xcode13.2 Error Domain=AVKitErrorDomain Code=-1001 "Failed to start picture in picture." UserInfo={NSLocalizedDescription=Failed to start picture in picture., NSLocalizedFailureReason=The UIScene for the content source has an activation state other than UISceneActivationStateForegroundActive, which is not allowed.} I didn't get any solutions online, plz help me on this issue. FYI: PIP button/mode is working if I use AVPlayerViewController(with default playerItem buttons) instead of AVPlayer & AVPictureInPictureController with custom controls Thanks in advance. Adopting Picture in Picture in a Custom Player
Replies
3
Boosts
2
Views
1.7k
Activity
6d
`LockedCameraCaptureManager` practically unusable since iOS 26
Somewhere since iOS 26, the LockedCameraCapture framework gets in an unpredictable state after opening the main app from the LockedCamera extension using LockedCameraCaptureSession.openApplication(for userActivity:). (Feedback with sample code to reproduce: FB21966835) Opening the extension from the lock screen again doesn’t open the extension but puts the lock screen in a state as if it has. Content updated from LockedCameraCaptureManager.shared.sessionContentUpdates comes in inconsistently, usually needs the app to be opened again or the extension to be opened. This makes using this extension impossible for me as I use it to record video files that manually need to be imported when the app is launched (so not through PhotoKit). Does anybody have a suggestion to circumvent this issue or how to get this fixed?
Replies
2
Boosts
3
Views
1.2k
Activity
6d
AVPlayerItemSampleBufferOutput
I am using AVPlayer, but I am unable to retrieve PCM data from .m3u8 streams via the AVPlayerItemSampleBufferOutput API in iOS 27 beta. I have tried both the Objective-C and Swift APIs: the Objective-C delegate methods are not being called, and the Swift methods nextAvailableSampleBuffer() and nextSampleBuffer() are returning no data.
Replies
2
Boosts
1
Views
672
Activity
1w
fail to get HLS realtime stream via AVPlayerItemSampleBufferOutputDelegate
Hi, I'm trying to do some realtime audio processing on audio served from an HLS stream (i.e. an AVPlayer created using an M3U HTTP URL). And I find that new APIs(AVPlayerItemSampleBufferOutput) are available in iOS 27.0 to achieve it. However, I failed to get the available data via the AVPlayerItemSampleBufferOutputDelegate. And I found some error in the system log: I create AVPlayerItemSampleBufferOutput and set the delegate after receiving the AVPlayerItemStatusReadyToPlay event. And here's my code: @interface OCAudioSamplebuffer () <AVPlayerItemSampleBufferOutputDelegate> @property (nonatomic, strong) AVPlayerItemSampleBufferOutput *bufferOutput; @property (nonatomic, strong) dispatch_queue_t bufferOutputQueue; @property (nonatomic, strong) AVPlayerItem *playerItem; @end - (void)playItem:(AVPlayerItem *)item { if (@available(iOS 27, *)) { AVAudioSession *audioSession = [AVAudioSession sharedInstance]; if([NSThread mainThread]){ [audioSession setCategory:AVAudioSessionCategoryPlayback withOptions:AVAudioSessionCategoryPlayback error:nil]; [audioSession setActive:YES error:nil]; }else{ dispatch_async(dispatch_get_main_queue(), ^{ [audioSession setCategory:AVAudioSessionCategoryPlayback withOptions:AVAudioSessionCategoryPlayback error:nil]; [audioSession setActive:YES error:nil]; }); } AVPlayerItemSampleBufferOutputAudioConfiguration *cfg = [[AVPlayerItemSampleBufferOutputAudioConfiguration alloc] init]; CMFormatDescriptionRef formatDescription = [self createPCMFormatDescriptionWithSampleRate:44100.0 channels:2 isFloat:YES]; cfg.requestedAudioFormat = formatDescription; if (formatDescription) { NSLog(@"create PCM Format Description success"); CFRelease(formatDescription); } else { NSLog(@"fail to create PCM Format Description"); } self.bufferOutput = [[AVPlayerItemSampleBufferOutput alloc] initWithConfiguration:cfg]; NSLog(@"create buffer success, PCM Format Description%@---,%@", cfg.requestedAudioFormat, formatDescription); self.bufferOutputQueue = dispatch_queue_create("audioSamplebufferQueue", DISPATCH_QUEUE_CONCURRENT); [self.bufferOutput setDelegate:self queue:self.bufferOutputQueue]; [item addOutput:self.bufferOutput]; self.playerItem = item; } else { // Fallback on earlier versions } } Any help. Thank you
Replies
2
Boosts
0
Views
937
Activity
1w
iOS 27 beta: Opening Notification Center pauses AVPlayer playback
Since iOS 27 beta we are seeing a behavior change in our video streaming app and I would like to know whether others can reproduce it. Behavior on iOS 27 beta: Fully opening the Notification Center pauses playback. Audio continues for about 5 more seconds, then the app is suspended. Closing the Notification Center leaves the player paused. On iOS 26 the same build keeps playing in this situation. Setup: AVQueuePlayer playing video with audio, not muted audiovisualBackgroundPlaybackPolicy = .automatic (default) AVAudioSession category .playback, UIBackgroundModes: audio entitlement AVPictureInPictureController attached with canStartPictureInPictureAutomaticallyFromInline = true Filed as FB23893965 Questions: Can anyone reproduce this on iOS 27 beta? Is this intentional or a regression?
Replies
1
Boosts
3
Views
843
Activity
1w
How can an app detect a TrueDepth pipeline failure when no depth output or drop callback is delivered?
How can an app detect a TrueDepth pipeline failure when no depth output or drop callback is delivered? We are using AVFoundation to stream synchronized RGB and depth data from the front-facing TrueDepth camera. On a small number of physical devices, the capture session and RGB stream remain healthy, but the depth pipeline becomes silent or returns unusable data. We would like to understand the intended public-API behavior and the supported way to detect this state without relying on private system logs or an application-defined timeout. We need an AVFoundation-only solution rather than ARKit. Capture configuration Our setup follows Apple's TrueDepth streaming sample: Discover .builtInTrueDepthCamera. Select an activeFormat with a nonempty supportedDepthDataFormats list. Set a supported activeDepthDataFormat, typically DepthFloat16. Add AVCaptureVideoDataOutput and AVCaptureDepthDataOutput. Enable the .depthData connection. Test both direct depth delegate delivery and AVCaptureDataOutputSynchronizer. Observe capture-session runtime errors, interruptions, connection state, and system pressure. The same code and configuration continuously deliver valid RGB and depth frames on healthy control devices. Observed failure modes 1. No depth callback of any kind The video delegate continues to receive RGB frames at the expected rate. The depth connection reports enabled and active. The capture session remains running. No AVCaptureSessionRuntimeErrorNotification or interruption is reported. System pressure is nominal. depthDataOutput(_:didOutput:timestamp:connection:) is never called. depthDataOutput(_:didDrop:timestamp:connection:reason:) is also never called. With AVCaptureDataOutputSynchronizer, RGB synchronization points may continue, but no AVCaptureSynchronizedDepthData object is present for the depth output. 2. One depth frame, followed by permanent silence On other affected devices, one depth frame is delivered shortly after startup, while hundreds of later RGB frames continue with no further depth output or drop callback. 3. A depth object is delivered, but the map is unusable We have also observed an AVDepthData object whose map is empty or has no usable spatial information, for example: a zero-sized/empty depth representation; an unfiltered map containing only NaN; or a filtered map containing a single constant value across nearly all pixels. Metadata such as depth quality, accuracy, calibration availability, supported formats, and connection state does not reliably distinguish these frames from healthy ones. Relevant documented behavior The documentation for AVCaptureSynchronizedDepthData.depthDataWasDropped says that a dropped depth object is different from a synchronization timestamp for which no depth capture occurred. In the latter case, no AVCaptureSynchronizedDepthData object is present in the collection. The documentation for depthDataOutput(_:didDrop:timestamp:connection:reason:) says that the callback reports captured depth data that wasn't processed, with an empty-shell AVDepthData object. This appears to leave a diagnostic gap: if the TrueDepth pipeline fails before a depth object is produced, the application may receive neither an output callback nor a drop callback. System-log correlation We do not want to depend on these private and undocumented strings, but sysdiagnose/device logs from affected devices correlate the public-API silence with messages such as: Can't activate Pearl projector: no projector token found PROJECTOR ON DENIED Object too close (no depth) GMC status: -2 Projector GMC hasn't completed yet - dropping depth/dx buffers Pulses exceed limit. Projector Off In the first group, depth delivery never starts. In the second group, depth may never start or may stop after one frame. RGB capture can remain active in both cases. These logs suggest that depth or disparity buffers can be rejected before they reach AVCaptureDepthDataOutput, which would explain why the application sees neither didOutput nor didDrop. However, none of these projector, IR, calibration, protection, or token states appears to be exposed by a public API. Questions Is it expected that AVCaptureDepthDataOutput delivers neither didOutput nor didDrop when no depth object is produced upstream? Is the absence of AVCaptureSynchronizedDepthData from a synchronization collection the only public signal for this condition? Is there a supported API, notification, KVO property, or error that reports that the TrueDepth IR/projector/depth provider is unavailable, disabled, protected, stalled, or unable to produce data? Can an app distinguish a healthy but temporarily delayed depth stream from a pipeline that will never deliver depth, other than by implementing its own timeout and frame-count watchdog? Is there a documented validity rule for an empty, all-NaN, or near-constant AVDepthData.depthDataMap? Should applications treat these as unavailable depth even when the AVDepthData metadata reports a normal quality or accuracy? What is the recommended recovery procedure: rebuild AVCaptureDepthDataOutput, restart the capture session, switch cameras and switch back, wait for a system notification, or stop retrying until the device state changes? ARKit can report ARError.sensorFailed when a required sensor does not deliver input. Is there an AVFoundation equivalent that identifies which required input failed, without adopting ARKit? If no public signal currently exists, would Apple consider exposing a depth-provider status/error callback that distinguishes at least: captured but dropped; no depth capture for the timestamp; provider temporarily unavailable; and provider disabled or failed until recovery/service? We can provide a minimal sample project, affected-device sysdiagnose, and synchronized timestamp traces in Feedback Assistant if that would help. Please let us know which diagnostics and logging profiles would be most useful.
Replies
0
Boosts
0
Views
99
Activity
1w
AVCaptureVideoDataOutput stops zooming while AVCaptureVideoPreviewLayer continues — physical wide / ultra-wide / telephoto only
We use a single AVCaptureSession with AVCaptureVideoPreviewLayer and AVCaptureVideoDataOutput (preview-sized buffers, BGRA). When we increase videoZoomFactor, beyond a certain zoom level the image from AVCaptureVideoDataOutput no longer zooms further, while AVCaptureVideoPreviewLayer continues to zoom with the same zoom control. The preview and the video-data output therefore diverge. This behavior appears when the active camera is a physical lens device — wide, ultra-wide, or telephoto (e.g. builtInWideAngleCamera, builtInUltraWideCamera, builtInTelephotoCamera, or similar). It does not appear when the active input is a virtual / multi-camera (e.g. triple camera, dual-wide, or other system multi-camera). Are there known conditions under which this mismatch between preview and video-data output is expected? Thank you.
Replies
2
Boosts
0
Views
896
Activity
2w
AVAssetDownloadConfiguration does not download supplemental tracks on iOS 16
We're downloading HLS content for offline playback using AVAssetDownloadConfiguration. It's critical for our app that all subtitle tracks are downloaded together with the video, so users can watch any subtitle and audio option while offline. On iOS 16, not all tracks are downloaded, and when the user goes offline, playback fails with NSURLErrorNotConnectedToInternet (error -1009) because AVFoundation tries to fetch the missing subtitle segments over the network. On iOS 17 and later, everything works as expected — all subtitle and audio tracks are downloaded. However, on iOS 16 we observe two issues: First issue: Supplemental tracks are not downloaded on iOS 16 (device) Only the default audio track is downloaded. All non-default supplemental tracks — both subtitles and additional audio tracks — exhibit the same problem: Some supplemental tracks listed in the manifest don't even get a directory created in the .movpkg package Some supplemental tracks do get a directory created, but it contains only StreamInfoBoot.xml and StreamInfoRoot.xml — no actual segment data (.frag files) is downloaded. The StreamInfoBoot.xml for these directories shows NO, 0, and 0, with Supplemental Additionally, a Manifest.m3u8 stream directory is created with Complete=NO and no segment data. On iOS 17+ devices, all supplemental track directories are created and all contain segment data. Second issue: Downloads fail entirely on iOS 16 Simulator We receive the following error: Error Domain=NSURLErrorDomain Code=-16090 UserInfo={ NSLocalizedDescription=The operation could not be completed, _NSURLErrorFailingURLSessionTaskErrorKey=BackgroundAVAssetDownloadTask <DEC0D555-6C05-416C-91B8-F490F5A29046>.<3>, _NSURLErrorRelatedURLSessionTaskErrorKey=( "BackgroundAVAssetDownloadTask <DEC0D555-6C05-416C-91B8-F490F5A29046>.<3>" ), NSLocalizedFailureReason=An unknown error occurred (-16090) } On iOS 17+ Simulators, downloads work without issues. Our setup: let urlAsset = AVURLAsset(url: manifestURL) let preferredMediaSelection = try await urlAsset.load(.allMediaSelections) let config = AVAssetDownloadConfiguration(asset: urlAsset, title: title) config.primaryContentConfiguration.mediaSelections = preferredMediaSelection let task = assetDownloadURLSession.makeAssetDownloadTask(downloadConfiguration: config) task.resume() Example HLS manifest: #EXTM3U #EXT-X-VERSION:7 #EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="French",LANGUAGE="fr",DEFAULT=NO,AUTOSELECT=YES,FORCED=NO,URI="s_25/stream.m3u8" #EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="French (forced)",LANGUAGE="fr-x-cc",DEFAULT=NO,AUTOSELECT=YES,FORCED=YES,URI="s_31/stream.m3u8" #EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="English",LANGUAGE="en",DEFAULT=NO,AUTOSELECT=YES,FORCED=NO,URI="s_37/stream.m3u8" #EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="Kazakh",LANGUAGE="kk",DEFAULT=NO,AUTOSELECT=YES,FORCED=NO,URI="s_43/stream.m3u8" #EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="Uzbek",LANGUAGE="uz",DEFAULT=NO,AUTOSELECT=YES,FORCED=NO,URI="s_49/stream.m3u8" #EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="aud_ec3",NAME="French 5.1",LANGUAGE="fr",DEFAULT=YES,AUTOSELECT=YES,CHANNELS="6",URI="a_18/stream.m3u8" #EXT-X-STREAM-INF:AUDIO="aud_ec3",SUBTITLES="subs",BANDWIDTH=1931408,VIDEO-RANGE=SDR,CODECS="hvc1.1.6.L90.90,ec-3",RESOLUTION=1024x428,FRAME-RATE=25,HDCP-LEVEL=NONE v_6/stream.m3u8 #EXT-X-SESSION-KEY:METHOD=SAMPLE-AES,URI="skd://XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX",KEYFORMAT="com.apple.streamingkeydelivery",KEYFORMATVERSIONS="1" Questions: What is the correct way to make sure all supplemental tracks (subtitles, non-default audio) are actually downloaded on an iOS 16 real device? We pass asset.load(.allMediaSelections) to primaryContentConfiguration.mediaSelections — is there a different configuration, or should we use auxiliaryContentConfigurations, or something else entirely? Is there any way to make asset downloads work on the iOS 16 Simulator at all? We always get NSURLErrorDomain error -16090, while iOS 17+ Simulators work fine. If the supplemental tracks download bug on iOS 16 cannot be fixed or worked around from the app side, do you have any recommended approach for handling incomplete .movpkg packages at offline playback time?
Replies
1
Boosts
0
Views
393
Activity
2w