Today we are on-boarded to using CMCD data to allow us to get more diagnostic data of our streaming. We would like to have the ability to upload our own custom headers, this is supported on other players, but not supported on AVPlayer. The workaround today that works is adding in custom query parameters then querying based on that, but we would like support explicitly in the headers.

I'm batch-transcoding a library of clips into downscaled editing proxies. If I kick off two hardware transcodes at once, the encoder reliably falls over, so right now I run everything serially: each clip gets fully decoded, encoded, and muxed before the next one starts. It works, but it's slow, and the decode and encode hardware are mostly idle waiting on each other. A few things I can't pin down from the docs: Is there an actual ceiling on how many VTCompressionSession / VTDecompressionSession instances can be live at once, and does it depend on the device, the codec, or the resolution? Can I query that ceiling at runtime? I'd rather size my concurrency up front than find it by crashing. Decode and encode are separate hardware blocks, so can I safely run a decode session for one clip while the previous clip is still encoding, or does VideoToolbox serialize them anyway? When I do go over the limit, what should I be checking so I can back off cleanly? Right now I just get a crash instead of an error I can catch. Anything that gets me off the fully-serial pipeline would help. Thank you

Hi, I downloaded and ran https://developer.apple.com/documentation/realitykit/rendering-stereoscopic-video-with-realitykit and noticed that memory usage grows linearly. I replaced the sample video with a different 8k side by side video, and the app crashed almost immediately due to memory leak. it looks like the culprit is from makeMutablePixelBuffer() function and the allocated pixelBuffers are not recycled after being used. screenshot is from a physical device.

Is there limits on the supported dimension for VTLowLatencyFrameInterpolationConfiguration. Querying VTLowLatencyFrameInterpolationConfiguration.maximumDimensions and VTLowLatencyFrameInterpolationConfiguration.minimumDimensions returns nil. When I try the WWDC sample project EnhancingYourAppWithMachineLearningBasedVideoEffects with a 4k video this statement try frameProcessor.startSession(configuration: configuration) executes but try await frameProcessor.process(parameters: parameters) throws error Error Domain=VTFrameProcessorErrorDomain Code=-19730 "Processor is not initialized" UserInfo={NSLocalizedDescription=Processor is not initialized}. Also, why is VTLowLatencyFrameInterpolationConfiguration able to run while app is backgrounded but VTFrameRateConversionParameters can't (due to gpu usage)?

When I try to send a DRM-protected video via Airplay to an Apple TV, the license request is made twice instead of once as it normally does on iOS. We only allow one request per session for security reasons, this causes the second request to fail and the video won't play. We've tested DRM-protected videos without token usage limits and it works, but this creates a security hole in our system. Why does it request the license twice in function: func contentKeySession(_ session: AVContentKeySession, didProvide keyRequest: AVContentKeyRequest)? Is there a way to prevent this?

Good day. A video I created via iOS AVAssetWriter with the following settings: let videoWriterInput = AVAssetWriterInput( mediaType: .video, outputSettings: [ AVVideoCodecKey: AVVideoCodecType.hevc, AVVideoWidthKey: 1080, AVVideoHeightKey: 1920, AVVideoCompressionPropertiesKey: [ AVVideoAverageBitRateKey: 2_000_000, AVVideoMaxKeyFrameIntervalKey: 30 ], ] ) let audioWriterInput = AVAssetWriterInput( mediaType: .audio, outputSettings: [ AVFormatIDKey: kAudioFormatMPEG4AAC, AVNumberOfChannelsKey: 2, AVSampleRateKey: 44100, AVEncoderBitRateKey: 128000 ] ) When It is split into fMP4 HLS format using ffmpeg, the video is unable to be played in iOS with the following error: CoreMediaErrorDomain error -12848 However, the video is played normally in Android, Browser HLS players, and also VLC Media Player. Please assist. Thank you.

SBS ViewPacking add a half a frame to the opposite eye. Meaning if you look all the way right you can see an extra half frame with left eye and vice versa. OU doesn't work at all, the preview just doesn't show a thumbnail and the video doesn't play. Any hints on how to fix this? I submitted a bug report but haven't heard anything.

Background: For iOS, I have built a custom video record app that records video at the highest possible FPS (supposed to be around 240 FPS but more like ~200 in practice). When I save this video to the user's camera roll, I notice this dynamic playback speed bar. This bar will speed up or slow down the video. My question: How can I save my video such that this playback speed bar is constantly slow or plays at real time speed? For reference I have include the playback speed bar that I am talking about in the screenshot, you can find this in the photo app when you record slow motion video.

In our app, users select many videos (10–100+) that we upload to our server in one batch. Two problems compound each other: Backgrounding: Users almost always leave the app while the batch is in flight. Uploads started with a regular URLSession get suspended. Pre-upload transcoding: To reduce upload duration and cellular data usage, we downscale/re-encode each video (via AVAssetExportSession) before upload. This is resource-intensive and unlike a background URLSession upload task can't continue once the app is suspended, so the pipeline stalls even if the uploads themselves could continue. What's Apple's recommended design here? Is the intended pattern a background URLSession with file-based upload tasks, fed by a BGProcessingTask that drains the transcode queue opportunistically? Are there APIs we're missing, e.g. AVAssetExportSession's background-friendly options, HEVC/preferredTranscoding export presets that are cheaper, or any way to get extended runtime for the encode step (beginBackgroundTask budgets seem too short for dozens of videos)? And is there guidance on the trade-off of just uploading originals and transcoding server-side instead?

Avplayer encapsulates a player. After connecting to an Apple Bluetooth headset, it immediately reports an error. Non-Apple Bluetooth headsets can be played, but currently the issue is that the player works normally in one app but not in another. We are an educational app.

I’m building a macOS video editor that uses AVComposition and AVVideoComposition. Initially, my renderer creates a composition with some default video/audio tracks: @Published var composition: AVComposition? @Published var videoComposition: AVVideoComposition? @Published var playerItem: AVPlayerItem? Then I call a buildComposition() function that inserts all the default video segments. Later in the editing workflow, the user may choose to add their own custom video clip. For this I have a function like: private func handlePickedVideo(_ url: URL) { guard url.startAccessingSecurityScopedResource() else { print("Failed to access security-scoped resource") return } let asset = AVURLAsset(url: url) let videoTracks = asset.tracks(withMediaType: .video) guard let firstVideoTrack = videoTracks.first else { print("No video track found") url.stopAccessingSecurityScopedResource() return } renderer.insertUserVideoTrack(from: asset, track: firstVideoTrack) url.stopAccessingSecurityScopedResource() } What I want to achieve is the same behavior professional video editors provide, after the composition has already been initialized and built, the user should be able to add a new video track and the composition should update live, meaning the preview player should immediately reflect the changes without rebuilding everything from scratch manually. How can I structure my AVComposition / AVMutableComposition and my rendering pipeline so that adding a new clip later updates the existing composition in real time (similar to Final Cut/Adobe Premiere), instead of needing to rebuild everything from zero? You can find a playable version of this entire setup at :- https://github.com/zaidbren/SimpleEditor

I have an application which records video along with some custom metadata and a chapter track. The resultant video is stored in the Camera Roll. When sharing the video via the Share Sheet or AirDrop, the metadata track is stripped entirely (the chapter markers are preserved) Sharing via AirDrop with the "All Photos Data" option does include the metadata track, as does copying from the device with Image Capture but this is a bad user experience as the user must remember to explicitly select this option, and the filename is lost when sending this way. I have also tried various other approaches (such as encoding my metadata in a subtitle track, which I didn't expect to be stripped as it's an accessibility concern) but it's also removed. Essentially I am looking for a definitive list of things that are not stripped or if there's a way to encode a track in some way to indicate it should be preserved. The metadata is added via AVTimedMetadataGroup containing one AVMutableMetadataItem which has its value as a JSON string. I took a different approach with the Chapter Marker track (mainly because I did it first in a completely different way and didn't rework it when I added the other track). I post-process these after the video is recorded, and add them with addMutableTrack and then addTrackAssociation(to: chapterTrack, type: .chapterList) but I don't think that's the reason the chapter track persists where the custom metadata does not as other tests with video files from other sources containing subtitles etc also had their subtitle data stripped. tl;dr I record videos with metadata that I want to be able to share via Share Sheet and AirDrop, what am I doing wrong?

Starting with iOS 26.4 and the iOS 26.4 SDK, the .pauses audiovisual background playback policy is not correctly applied anymore to an AVPlayer having an attached video layer displayed on screen. This means that, when backgrounding a video-playing app (without Picture in Picture support) or locking the device, playback is not paused automatically by the system anymore. This issue affects the Apple TV application as well. We have filed FB22488151 with more information.

I read somewhere that the frames are returned in decode order instead of presentation order when using AVAssetReader. The documentation seems sparse on the subject. I have so far failed to find a video file where the frames are not returned in presentation order. Can anyone confirm the frames are actually returned in decode order?

These are follow up questions for: https://feedbackassistant.apple.com/feedback/22779473 For a large AVQueuePlayer queue (50-100+ HLS items), does AVFoundation guarantee it only allocates video decode sessions for the active and next item? Our profiling suggests this is the case, but we want to confirm whether we can rely on this behavior or if we should implement an application-side sliding window to limit concurrent decoders.

I saw that the Broadcast extension is deprecated in iOS 28, as well as the ReplayKit with nothing mentioned on what is going to be used going forward. I found out that the Screen Capture Kit is mentioned as replacement but is available for macOS only, even though some methods now are marked with iOS 28+ (https://developer.apple.com/documentation/screencapturekit). Will the ScreenCaptureKit be used for screen sharing going forward and replace the Broadcast Extension?

I have created code for iOS that allows me to start and stop video acquisition from a proprietary USB camera using AVFoundation's AVCaptureSession and AVCaptureDevice APIs. There is a start and stop method. The start method takes an argument to specify one of two formats that I use for my custom camera application. I can start the session and switch between formats all day without any errors. However, if I start and then stop the camera three times in a row, on the third invocation of start, I get errors in the console output and the CMSampleBuffers stop flowing to my callback. Additionally, once I get AVFoundation into this state, stoping the camera doesn't help. I have to kill the app and start over. Here are the errors. And below these, the code. I'm hoping someone who has experience with these errors or an engineer from Apple who knows the AVFoundation image capture pipeline code, can respond and tell me what I'm doing wrong. Thanks. <<<< FigCaptureSourceRemote >>>> Fig assert: "! storage->connectionDied" at bail (FigCaptureSourceRemote.m:235) - (err=0) <<<< FigCaptureSourceRemote >>>> Fig assert: "err == 0 " at bail (FigCaptureSourceRemote.m:558) - (err=-16453) <<<< FigCaptureSourceRemote >>>> Fig assert: "! storage->connectionDied" at bail (FigCaptureSourceRemote.m:235) - (err=0) <<<< FigCaptureSourceRemote >>>> Fig assert: "err == 0 " at bail (FigCaptureSourceRemote.m:253) - (err=-16453) <<<< FigCaptureSourceRemote >>>> Fig assert: "err == 0 " at bail (FigCaptureSourceRemote.m:269) - (err=-16453) <<<< FigCaptureSourceRemote >>>> Fig assert: "err == 0 " at bail (FigCaptureSourceRemote.m:511) - (err=-16453) Capture session error: The operation could not be completed Capture session error: The operation could not be completed func start(for deviceFormat: String) async throws -> AnyPublisher<CMSampleBuffer, Swift.Error> { func configureCaptureDevice(with deviceFormat: String) throws { guard let format = formatDict[deviceFormat] else { throw Error.captureFormatNotFound } captureSession.beginConfiguration() defer { captureSession.commitConfiguration() } try captureDevice.lockForConfiguration() captureDeviceFormat = deviceFormat captureDevice.activeFormat = format captureDevice.unlockForConfiguration() } return try await withCheckedThrowingContinuation { continuation in sessionQueue.async { [unowned self] in logger.debug("Start capture session for \(deviceFormat): \(String(describing: captureSession))") // If we were already steaming camera images from a different mode, terminate that stream. bufferPublisher?.send(completion: .finished) bufferPublisher = nil captureDeviceFormat = "" do { // Re-configure with the new format; should be harmless if called with the currently configured format. try configureCaptureDevice(with: deviceFormat) // Return a new stream publisher for this invocation. bufferPublisher = PassthroughSubject<CMSampleBuffer, Swift.Error>() // If we are not currently running, start the image capture pipeline. if captureSession.isRunning == false { captureSession.startRunning() } continuation.resume(returning: bufferPublisher!.eraseToAnyPublisher()) } catch { logger.fault("Failed to start camera: \(error.localizedDescription)") continuation.resume(throwing: error) } } } } func stop() async throws { try await withCheckedThrowingContinuation { continuation in sessionQueue.async { [unowned self] in logger.debug("Stop capture session: \(String(describing: captureSession))") // The following invocation is synchronous and takes time to execute; // looks like a stall but you can ignore it as the MainActor is not blocked. captureSession.stopRunning() // Terminate the stream and reset our state. bufferPublisher?.send(completion: .finished) bufferPublisher = nil captureDeviceFormat = "" // Signal the caller that we are done here. continuation.resume() } } }

What is the recommended way of showing a processed image from AVCaptureVideoDataOutput in a swiftUI view? Currently my chain is AVCaptureVideoDataOutput(SampleBuffer) -> CIImage -> CIFilters -> createCGImage from processed CIImage -> Create swiftUI 'Image' from CGIImage, that is in a view Is there a better way to go from AVCaptureVideoDataOutput to a swiftUI view, with image processing?

We develop software for video broadcasting, including for sporting events, where animated scoreboards play a key role. macOS offers excellent features for such animations—such as CAAnimations—but this holds true only as long as the animations run on visible screens. Distributing the video signal (e.g., via NDI) requires access to the CVPixelBuffers of the individual frames. Currently, we generate the animations on an external screen and create the pixel buffers using ScreenCaptureKit. We are unaware of any way to obtain these pixel buffers without relying on such—strictly speaking, unnecessary—external screens (or multiple screens). If the window is created offscreen, it updates only once per second, which is unusable. Are there alternatives to external screens? If not, why can’t we create a virtual offscreen device for such use cases—one where we specify the required frame rate and rendering frequency—to generate the necessary video frames? This would also be helpful for HTML-based overlays, which are becoming increasingly popular; currently, these are also rendered only once per second when offscreen.

Hi Everyone! I am writing a media extension to playback old "Amiga" ANIM files, as a test project. I have build the following objects: MEFormatReader METrackReader MESampleCursor The subclassed objects seem fairly straightforward. Rather than create a Video Decoder, I followed the instructions in the SampleCursor object header, and use the function: loadSampleBufferContainingSamplesToEndCursor:completionHandler: to deliver an BGRA 8-bit image. This works great AVPlayer object in my Swift based test app, but QuickTime Player will not actually play the movie. If I scrub on QuickTime Player's timeline, I can watch the movie just fine. But if I click on the "Play" button. Nothing happens. For fun, I created a custom pixel type, and implemented a MEVideoDecoder object. This also works with my AVPlayer test app, but again, same problem with QuickTime Player. I have even generated a JPEG image in the SampleCursor, and that fails too. I am stumped on this. I do not see any way to have QuickTime Player work properly with my MediaExtension, nor is there any documentation as to what to do. Suggestions? bob