Explore the integration of media technologies within your app. Discuss working with audio, video, camera, and other media functionalities.

All subtopics
Posts under Media Technologies topic

Post

Replies

Boosts

Views

Activity

photokit AI metadata
Is there a way to get AI metadata from photokit without having to run the algorithms again? I'd like to be able to get captions, featureprint, and optimal crops. I can use Vision and CoreML to generate these, but it'd be easier/better to read the info directly from Photokit.
1
0
28
5d
Setting up video and image capture pipeline creates internal errors in AVFoundation.
I have created code for iOS that allows me to start and stop video acquisition from a proprietary USB camera using AVFoundation's AVCaptureSession and AVCaptureDevice APIs. There is a start and stop method. The start method takes an argument to specify one of two formats that I use for my custom camera application. I can start the session and switch between formats all day without any errors. However, if I start and then stop the camera three times in a row, on the third invocation of start, I get errors in the console output and the CMSampleBuffers stop flowing to my callback. Additionally, once I get AVFoundation into this state, stoping the camera doesn't help. I have to kill the app and start over. Here are the errors. And below these, the code. I'm hoping someone who has experience with these errors or an engineer from Apple who knows the AVFoundation image capture pipeline code, can respond and tell me what I'm doing wrong. Thanks. <<<< FigCaptureSourceRemote >>>> Fig assert: "! storage->connectionDied" at bail (FigCaptureSourceRemote.m:235) - (err=0) <<<< FigCaptureSourceRemote >>>> Fig assert: "err == 0 " at bail (FigCaptureSourceRemote.m:558) - (err=-16453) <<<< FigCaptureSourceRemote >>>> Fig assert: "! storage->connectionDied" at bail (FigCaptureSourceRemote.m:235) - (err=0) <<<< FigCaptureSourceRemote >>>> Fig assert: "err == 0 " at bail (FigCaptureSourceRemote.m:253) - (err=-16453) <<<< FigCaptureSourceRemote >>>> Fig assert: "err == 0 " at bail (FigCaptureSourceRemote.m:269) - (err=-16453) <<<< FigCaptureSourceRemote >>>> Fig assert: "err == 0 " at bail (FigCaptureSourceRemote.m:511) - (err=-16453) Capture session error: The operation could not be completed Capture session error: The operation could not be completed func start(for deviceFormat: String) async throws -> AnyPublisher<CMSampleBuffer, Swift.Error> { func configureCaptureDevice(with deviceFormat: String) throws { guard let format = formatDict[deviceFormat] else { throw Error.captureFormatNotFound } captureSession.beginConfiguration() defer { captureSession.commitConfiguration() } try captureDevice.lockForConfiguration() captureDeviceFormat = deviceFormat captureDevice.activeFormat = format captureDevice.unlockForConfiguration() } return try await withCheckedThrowingContinuation { continuation in sessionQueue.async { [unowned self] in logger.debug("Start capture session for \(deviceFormat): \(String(describing: captureSession))") // If we were already steaming camera images from a different mode, terminate that stream. bufferPublisher?.send(completion: .finished) bufferPublisher = nil captureDeviceFormat = "" do { // Re-configure with the new format; should be harmless if called with the currently configured format. try configureCaptureDevice(with: deviceFormat) // Return a new stream publisher for this invocation. bufferPublisher = PassthroughSubject<CMSampleBuffer, Swift.Error>() // If we are not currently running, start the image capture pipeline. if captureSession.isRunning == false { captureSession.startRunning() } continuation.resume(returning: bufferPublisher!.eraseToAnyPublisher()) } catch { logger.fault("Failed to start camera: \(error.localizedDescription)") continuation.resume(throwing: error) } } } } func stop() async throws { try await withCheckedThrowingContinuation { continuation in sessionQueue.async { [unowned self] in logger.debug("Stop capture session: \(String(describing: captureSession))") // The following invocation is synchronous and takes time to execute; // looks like a stall but you can ignore it as the MainActor is not blocked. captureSession.stopRunning() // Terminate the stream and reset our state. bufferPublisher?.send(completion: .finished) bufferPublisher = nil captureDeviceFormat = "" // Signal the caller that we are done here. continuation.resume() } } }
1
0
386
5d
Feedback on Apple Spatial Audio re-render behavior for Dolby Atmos music delivery — perspective from a working mix engineer
Hey everyone, quick disclaimer before jumping in - I used my LLM to structure this around notes/observations I've been taking the last several months. I apologize for the length but felt this was the best distillation of an important challenge my peers and I are facing in mixing music for the largest device/service segment of the listening community - Airpods Pro/Max via Apple Music. Thanks in advance for reading and any feedback you can offer! -Kyle I'm a professional mix engineer working primarily in contemporary pop, indie, and country. After 20+ years of working in stereo, I've started delivering Dolby Atmos ADM masters for Apple Music distribution. I want to share some specific observations about the Apple Spatial Audio re-render in the hope that it's useful to the team that owns this rendering pipeline — and to ask a few questions I haven't been able to find answered in public documentation. I recognize this sits at an unusual intersection of the developer platform and the Apple Music delivery side of the house, but since the rendering behavior is ultimately a platform-level decision, this felt like the right place to start. Background: the three-format problem When delivering an Atmos ADM master, a mixer effectively has to satisfy three distinct listening contexts simultaneously: Speaker playback (7.1.4 or similar) via the Dolby renderer Dolby binaural re-render (AC-4), as heard on TIDAL and Amazon — which respects the OFF/NEAR/MID/FAR binaural mode settings on beds and objects Apple Spatial Audio headphone re-render on Apple Music The first two have reasonably predictable translation. The third is where I'm running into consistent issues — and where I'd value any guidance Apple is able to share. The core issue: Apple's re-render discards binaural mode metadata As best I can tell from testing and from community documentation, Apple's pipeline ingests the ADM, creates an internal 7.1.4 render, and then applies its own proprietary binaural spatialization — one that does not reference the OFF/NEAR/MID/FAR binaural mode parameters embedded by the mixer. This is distinct from the Dolby AC-4 path, which does honor those settings. In practice, this means: Apple's re-render applies a consistent room character regardless of what the mixer has specified for individual elements Elements like lead vocals and kick/snare — which I'm routing through beds or objects with OFF or NEAR binaural settings specifically to preserve intimacy and punch — receive the same ambient room treatment as wider, more spacious elements The result on Apple Music has noticeably more perceived distance and "room" on transient-heavy and close-mic'd elements than either the speaker mix or the Dolby binaural render To be specific about the perceptual effect: the Apple re-render's virtual room introduces early reflections and a sense of speaker-to-listener distance that significantly undercuts the intimacy and impact of close elements. On a pop or country vocal, this is the difference between a performance that feels present and direct versus one that feels recessed in a listening space. On drums, transient attack is softened in a way that doesn't happen in any other delivery context for the same master. Questions for the team I'd be grateful for any clarity on the following: Is the behavior of ignoring OFF/NEAR/MID/FAR metadata intentional and permanent, or is it something that may change as the rendering pipeline evolves? Is there any mechanism — existing or planned — by which a mixer can influence the room character or "closeness" of elements in Apple's re-render, outside of object positioning metadata? Is there any documentation of how Apple's binaural spatialization layer translates object distance metadata (as opposed to binaural mode) — i.e., does Z-axis positioning in the Atmos object space affect perceived distance in the re-render? Is there a recommended workflow or set of delivery parameters that Apple's audio team considers optimal for music content specifically, as opposed to film/TV? Notes on the Audiomovers Binaural Renderer for Apple Music I'm aware of and have used the Audiomovers plugin, which I understand was developed in collaboration with Apple and accurately reflects the Apple Spatial re-render during session monitoring. It's a genuinely useful tool and has improved my ability to anticipate Apple's output. My questions above are about the underlying rendering behavior — not the monitoring workflow, which is solved. Why this matters for music specifically Film and TV post content has different expectations around spatialization — a consistent room or "cinema" quality to the binaural render is arguably appropriate for that material. For music, particularly in contemporary genres where the stereo mix is already highly produced and intimate, an added room layer competes with the mix's own space design and consistently pushes elements further from the listener than intended. I'd argue music content would benefit from a rendering mode with a more "dry" or near-field room character — and I suspect I'm not alone in this among working Atmos music mixers. I'm happy to provide specific A/B examples or additional technical detail if that's useful to anyone on the platform team. Thanks for reading.
0
0
107
5d
MediaDeviceExtension is not launched; audiomxd logs “missing required entitlement” (iPadOS 27)
I'm trying to implement a MediaDeviceExtension, but it is not being launched. I followed the documentation, but I see "missing required entitlement" in log, even though I've added the entitlement. My Setup com.apple.developer.media-device-extension on both the app and the extension: <key>com.apple.developer.media-device-extension</key> <array> <string>media-device-protocol.myradio</string> </array> The same id is in the extension’s UTExportedTypeDeclarations → UTTypeIdentifier (conforming to public.media-sharing-protocol) and in the Swift protocolType property. The app’s Info.plist has MDESupportedProtocols with the same id. EXExtensionPointIdentifier = com.apple.media-device-extension in EXAppExtensionAttributes. The capability is enabled on both App IDs in Certificates, Identifiers & Profiles. Problem Opening an AVRoutePickerView does not call startDeviceDiscovery(), and no device appears in the picker. Device log: -FigCustomEndpointManager- manager_shouldAllowDiscoveryForProtocol: Protocol media-device-protocol.myradio has never been activated - allowing discovery for first-time use -FigCustomEndpointManager- manager_SetDiscoveryMode: Failed to launch system casting instance for protocol media-device-protocol.myradio -MXAppExtensionMonitor- -[MXAppExtensionInstance launchExtensionRequiring:]: Extension with identifier: <_EXExtensionIdentity: 0x…> is missing required entitlement, dropping! -MXSystemMediaCasting- -[MXSystemCastingExtensionInstance launch]: Failed to launch <SystemMediaCastingExtension<MyRadio:media-device-protocol.myradio>: …, <EXExtensionProcess: (null) PID: 0>, 0> Question What entitlement is MXAppExtensionMonitor checking for when launching a SystemMediaCastingExtension? It is not named in the log. If an entitlement beyond com.apple.developer.media-device-extension is required, how is it obtained? FB23043277
1
1
54
5d
Support for webm playback and processing
Hi, Safari supports playing back webm files (such as those recorded by Chrome/Firefox browsers with the MediaRecorder API), but our native app is unable to play these back with AVPlayer. Is there a supported way to do this? Right now we have to use a WebView for this! Would also love to ExtAudioFileURLOpen or process these files with AVAudioEngine. Have I missed something obvious here?
4
0
86
6d
Auv3 Synth Zippering
I am running into zippering and zipper storms in making my Auv3 synth. Do you have advice? It is a big synthesizer, big cinematic sounds, big presets, dual mode, I am mostly hitting zipper storms when trying to play fast on big preset recipes. Is there something I need to know to avoid zippering?
1
0
69
6d
RemoteMediaSession device add/remove
Are there plans to extend the session or MediaDevice capabilities to be able to add and remove speakers via the Lock Screen and Control Center? For example, user may have multiple speakers, they start playback on a subset of them and would like to add other speakers or remove some from the session through Control Center.
1
0
101
6d
My biggest priorities after 9 months of shipping Albums for macOS
Hi everyone! Adam here, the developer of Albums. Bummed not to get to have our yearly WebEx reunion where I beg y’all to add an endpoint to the Apple Music API to allow deleting items from a user’s library. But that’s not what this thread is about. Back in October, thanks to your team’s multi-year efforts, I was finally able to ship Albums for macOS. It’s been a genuine dream come true to use it on the platform I always envisioned it on, and I hear from users all the time who feel the same way. I know it was a significant engineering effort, and I’m genuinely very grateful. I’ll be judicious with my time here (I’d love to chat in more detail about my adventures in MusicKit on the Mac somehow, sometime). These are the main things on my wishlist for MusicKit on the Mac. There are three main things I hear from users that my app “can’t do” that can be dealbreakers for them. The first two are AirPlay support and a volume slider. Users are only able to AirPlay using the control center utility, because AVRoutePickerView does not work with ApplicationMusicPlayer (FB13934910). Relatedly, the MPVolumeView does not work with ApplicationMusicPlayer (FB21042385), so I can’t allow users app-specific volume control for my app. The lack of those two things is a real detriment to my app being able to be taken seriously alongside all of the other music player apps on the platform for which those features are table stakes. I know there’s a challenge here given the playback actually happens in the subprocess, but hoping some progress can be made here. The other issue is that library tracks played in ApplicationMusicPlayer do not update the last played date or play count in Music.app or in the user’s iCloud Music Library (FB17675148). Some people refuse to use the app for that reason, and I can’t say I blame them. I’ve only been able to test this briefly in Golden Gate, but it seems like this is still the case. Are you able to share anything about your work on the music library in macOS this year? Thank you again for all your work on MusicKit! I’m planning to get the last of the load-bearing MediaPlayer code out of my codebase later this year. Hooray!
1
0
92
6d
Slow MusicKit library performance in Golden Gate beta 1
Hello friends! Happy WWDC. Thanks very much for all your work on MusicKit this year! I figure I’ll start things off with a bug report (sorry!). I filed a Feedback earlier today that music library operations in MusicKit are significantly slower in macOS Golden Gate beta 1 than in Tahoe. For example, a .with([.tracks]) operation on an Album takes 4-5 seconds rather than the 95ms it did in Tahoe. Sample project, traces, and sysdiagnoses in FB23037115.
1
0
68
6d
Audio Prioritization
Is there anyway that we as users can set apps to not be able to play audio? There have been times that I want to listen to music and then scroll specific social media apps, and the social media apps keep interrupt the audio. I made a feedback related to this last year: FB18494861
0
0
73
6d
microphone level monitoring
My macOS app provides real-time microphone level monitoring, and I also have a macOS widget that shows recent audio level status. Since WidgetKit widgets are snapshot/timeline based rather than continuously live SwiftUI views, is there any new capability in macOS 27 that allows a widget to show more responsive or animated audio-level changes? If true real-time animation is still not supported in widgets, what is Apple’s recommended approach for displaying frequently changing signal levels, such as audio meters, in a macOS widget?
2
0
117
6d
CATaps
I’ve noticed that CATap appears to scale the stereo/mono mixdown based on the number of channels in the output audio device. For example, if the selected output device has 4 channels, the tapped audio is reduced by half. This behavior does not seem to be documented anywhere, but I’m assuming it may have been done to reduce the chance of clipping when downmixing. The issue is that users are complaining that the audio level is too low. Is there any workaround for this that does not involve tapping every audio device separately and doing my own mixdown? Ideally, I’d like to keep using the system tap/mix behavior but avoid the automatic level reduction, or at least compensate for it in a reliable way.
1
0
61
6d
What's the recommended way for a Mac app to receive audio from other apps in 2026?
My app allows users to transcribe and translate audio from other apps running on your Mac. I currently use ScreenCaptureKit to do this, and just ignore the video stream and only use the audio stream. This works, but it's awkward because it requires I ask users for screen recording permission, even though all I want is the audio, not the screen. Is there a better way to do this in modern macOS releases? Thanks!
1
0
65
6d
AudioServerPlugin AddDeviceClient/RemoveDeviceClient calls.
I added logging in an AudioServerPlugin to trace the arrival and removal of a particular client in the AddDeviceClient() and RemoveDeviceClient() methods. To my surprise the app was removed and added as a client often, even when it was in a steady state, for example when using the Audio MIDI Setup to change sample rates. Is this expected behavior, and if so, are there any explanations about this.
0
0
37
6d
Optimization of multiple audio track synchronization.
I have an app where multiple audio files of the same length are played and they need to be in sync to produce the full song. My current setup syncs each track to the same position every time the user pauses or plays the song. However it is super laggy when the Audio is played either for the first time or when unpausing/pausing. What's the best way to do this?
2
0
94
6d
How to Seamlessly Handle FIDO QR Codes in Your iOS App
When scanning a FIDO QR code within an iOS app—whether using a custom AVFoundation (AVCaptureSession) implementation or DataScannerViewController—the system displays a native OS confirmation prompt. However, scanning the same QR code using the native system Code Scanner bypasses this prompt entirely. As a developer: Is there a way to suppress or avoid this native prompt when using custom in-app scanners? Alternatively, can I programmatically invoke the system Code Scanner directly from my app and have it deep-link back to the app once the scan is complete?
0
0
54
6d
photokit AI metadata
Is there a way to get AI metadata from photokit without having to run the algorithms again? I'd like to be able to get captions, featureprint, and optimal crops. I can use Vision and CoreML to generate these, but it'd be easier/better to read the info directly from Photokit.
Replies
1
Boosts
0
Views
28
Activity
5d
Setting up video and image capture pipeline creates internal errors in AVFoundation.
I have created code for iOS that allows me to start and stop video acquisition from a proprietary USB camera using AVFoundation's AVCaptureSession and AVCaptureDevice APIs. There is a start and stop method. The start method takes an argument to specify one of two formats that I use for my custom camera application. I can start the session and switch between formats all day without any errors. However, if I start and then stop the camera three times in a row, on the third invocation of start, I get errors in the console output and the CMSampleBuffers stop flowing to my callback. Additionally, once I get AVFoundation into this state, stoping the camera doesn't help. I have to kill the app and start over. Here are the errors. And below these, the code. I'm hoping someone who has experience with these errors or an engineer from Apple who knows the AVFoundation image capture pipeline code, can respond and tell me what I'm doing wrong. Thanks. <<<< FigCaptureSourceRemote >>>> Fig assert: "! storage->connectionDied" at bail (FigCaptureSourceRemote.m:235) - (err=0) <<<< FigCaptureSourceRemote >>>> Fig assert: "err == 0 " at bail (FigCaptureSourceRemote.m:558) - (err=-16453) <<<< FigCaptureSourceRemote >>>> Fig assert: "! storage->connectionDied" at bail (FigCaptureSourceRemote.m:235) - (err=0) <<<< FigCaptureSourceRemote >>>> Fig assert: "err == 0 " at bail (FigCaptureSourceRemote.m:253) - (err=-16453) <<<< FigCaptureSourceRemote >>>> Fig assert: "err == 0 " at bail (FigCaptureSourceRemote.m:269) - (err=-16453) <<<< FigCaptureSourceRemote >>>> Fig assert: "err == 0 " at bail (FigCaptureSourceRemote.m:511) - (err=-16453) Capture session error: The operation could not be completed Capture session error: The operation could not be completed func start(for deviceFormat: String) async throws -> AnyPublisher<CMSampleBuffer, Swift.Error> { func configureCaptureDevice(with deviceFormat: String) throws { guard let format = formatDict[deviceFormat] else { throw Error.captureFormatNotFound } captureSession.beginConfiguration() defer { captureSession.commitConfiguration() } try captureDevice.lockForConfiguration() captureDeviceFormat = deviceFormat captureDevice.activeFormat = format captureDevice.unlockForConfiguration() } return try await withCheckedThrowingContinuation { continuation in sessionQueue.async { [unowned self] in logger.debug("Start capture session for \(deviceFormat): \(String(describing: captureSession))") // If we were already steaming camera images from a different mode, terminate that stream. bufferPublisher?.send(completion: .finished) bufferPublisher = nil captureDeviceFormat = "" do { // Re-configure with the new format; should be harmless if called with the currently configured format. try configureCaptureDevice(with: deviceFormat) // Return a new stream publisher for this invocation. bufferPublisher = PassthroughSubject<CMSampleBuffer, Swift.Error>() // If we are not currently running, start the image capture pipeline. if captureSession.isRunning == false { captureSession.startRunning() } continuation.resume(returning: bufferPublisher!.eraseToAnyPublisher()) } catch { logger.fault("Failed to start camera: \(error.localizedDescription)") continuation.resume(throwing: error) } } } } func stop() async throws { try await withCheckedThrowingContinuation { continuation in sessionQueue.async { [unowned self] in logger.debug("Stop capture session: \(String(describing: captureSession))") // The following invocation is synchronous and takes time to execute; // looks like a stall but you can ignore it as the MainActor is not blocked. captureSession.stopRunning() // Terminate the stream and reset our state. bufferPublisher?.send(completion: .finished) bufferPublisher = nil captureDeviceFormat = "" // Signal the caller that we are done here. continuation.resume() } } }
Replies
1
Boosts
0
Views
386
Activity
5d
Feedback on Apple Spatial Audio re-render behavior for Dolby Atmos music delivery — perspective from a working mix engineer
Hey everyone, quick disclaimer before jumping in - I used my LLM to structure this around notes/observations I've been taking the last several months. I apologize for the length but felt this was the best distillation of an important challenge my peers and I are facing in mixing music for the largest device/service segment of the listening community - Airpods Pro/Max via Apple Music. Thanks in advance for reading and any feedback you can offer! -Kyle I'm a professional mix engineer working primarily in contemporary pop, indie, and country. After 20+ years of working in stereo, I've started delivering Dolby Atmos ADM masters for Apple Music distribution. I want to share some specific observations about the Apple Spatial Audio re-render in the hope that it's useful to the team that owns this rendering pipeline — and to ask a few questions I haven't been able to find answered in public documentation. I recognize this sits at an unusual intersection of the developer platform and the Apple Music delivery side of the house, but since the rendering behavior is ultimately a platform-level decision, this felt like the right place to start. Background: the three-format problem When delivering an Atmos ADM master, a mixer effectively has to satisfy three distinct listening contexts simultaneously: Speaker playback (7.1.4 or similar) via the Dolby renderer Dolby binaural re-render (AC-4), as heard on TIDAL and Amazon — which respects the OFF/NEAR/MID/FAR binaural mode settings on beds and objects Apple Spatial Audio headphone re-render on Apple Music The first two have reasonably predictable translation. The third is where I'm running into consistent issues — and where I'd value any guidance Apple is able to share. The core issue: Apple's re-render discards binaural mode metadata As best I can tell from testing and from community documentation, Apple's pipeline ingests the ADM, creates an internal 7.1.4 render, and then applies its own proprietary binaural spatialization — one that does not reference the OFF/NEAR/MID/FAR binaural mode parameters embedded by the mixer. This is distinct from the Dolby AC-4 path, which does honor those settings. In practice, this means: Apple's re-render applies a consistent room character regardless of what the mixer has specified for individual elements Elements like lead vocals and kick/snare — which I'm routing through beds or objects with OFF or NEAR binaural settings specifically to preserve intimacy and punch — receive the same ambient room treatment as wider, more spacious elements The result on Apple Music has noticeably more perceived distance and "room" on transient-heavy and close-mic'd elements than either the speaker mix or the Dolby binaural render To be specific about the perceptual effect: the Apple re-render's virtual room introduces early reflections and a sense of speaker-to-listener distance that significantly undercuts the intimacy and impact of close elements. On a pop or country vocal, this is the difference between a performance that feels present and direct versus one that feels recessed in a listening space. On drums, transient attack is softened in a way that doesn't happen in any other delivery context for the same master. Questions for the team I'd be grateful for any clarity on the following: Is the behavior of ignoring OFF/NEAR/MID/FAR metadata intentional and permanent, or is it something that may change as the rendering pipeline evolves? Is there any mechanism — existing or planned — by which a mixer can influence the room character or "closeness" of elements in Apple's re-render, outside of object positioning metadata? Is there any documentation of how Apple's binaural spatialization layer translates object distance metadata (as opposed to binaural mode) — i.e., does Z-axis positioning in the Atmos object space affect perceived distance in the re-render? Is there a recommended workflow or set of delivery parameters that Apple's audio team considers optimal for music content specifically, as opposed to film/TV? Notes on the Audiomovers Binaural Renderer for Apple Music I'm aware of and have used the Audiomovers plugin, which I understand was developed in collaboration with Apple and accurately reflects the Apple Spatial re-render during session monitoring. It's a genuinely useful tool and has improved my ability to anticipate Apple's output. My questions above are about the underlying rendering behavior — not the monitoring workflow, which is solved. Why this matters for music specifically Film and TV post content has different expectations around spatialization — a consistent room or "cinema" quality to the binaural render is arguably appropriate for that material. For music, particularly in contemporary genres where the stereo mix is already highly produced and intimate, an added room layer competes with the mix's own space design and consistently pushes elements further from the listener than intended. I'd argue music content would benefit from a rendering mode with a more "dry" or near-field room character — and I suspect I'm not alone in this among working Atmos music mixers. I'm happy to provide specific A/B examples or additional technical detail if that's useful to anyone on the platform team. Thanks for reading.
Replies
0
Boosts
0
Views
107
Activity
5d
Microphone Gain on Safari/MacOS
The microphone gain for our app running on Safari/MacOS is substantially lower than with other browsers. What might account for that and what is the preferred method for a web app to control microphone gain.
Replies
4
Boosts
1
Views
105
Activity
5d
MediaDeviceExtension is not launched; audiomxd logs “missing required entitlement” (iPadOS 27)
I'm trying to implement a MediaDeviceExtension, but it is not being launched. I followed the documentation, but I see "missing required entitlement" in log, even though I've added the entitlement. My Setup com.apple.developer.media-device-extension on both the app and the extension: <key>com.apple.developer.media-device-extension</key> <array> <string>media-device-protocol.myradio</string> </array> The same id is in the extension’s UTExportedTypeDeclarations → UTTypeIdentifier (conforming to public.media-sharing-protocol) and in the Swift protocolType property. The app’s Info.plist has MDESupportedProtocols with the same id. EXExtensionPointIdentifier = com.apple.media-device-extension in EXAppExtensionAttributes. The capability is enabled on both App IDs in Certificates, Identifiers & Profiles. Problem Opening an AVRoutePickerView does not call startDeviceDiscovery(), and no device appears in the picker. Device log: -FigCustomEndpointManager- manager_shouldAllowDiscoveryForProtocol: Protocol media-device-protocol.myradio has never been activated - allowing discovery for first-time use -FigCustomEndpointManager- manager_SetDiscoveryMode: Failed to launch system casting instance for protocol media-device-protocol.myradio -MXAppExtensionMonitor- -[MXAppExtensionInstance launchExtensionRequiring:]: Extension with identifier: <_EXExtensionIdentity: 0x…> is missing required entitlement, dropping! -MXSystemMediaCasting- -[MXSystemCastingExtensionInstance launch]: Failed to launch <SystemMediaCastingExtension<MyRadio:media-device-protocol.myradio>: …, <EXExtensionProcess: (null) PID: 0>, 0> Question What entitlement is MXAppExtensionMonitor checking for when launching a SystemMediaCastingExtension? It is not named in the log. If an entitlement beyond com.apple.developer.media-device-extension is required, how is it obtained? FB23043277
Replies
1
Boosts
1
Views
54
Activity
5d
Support for webm playback and processing
Hi, Safari supports playing back webm files (such as those recorded by Chrome/Firefox browsers with the MediaRecorder API), but our native app is unable to play these back with AVPlayer. Is there a supported way to do this? Right now we have to use a WebView for this! Would also love to ExtAudioFileURLOpen or process these files with AVAudioEngine. Have I missed something obvious here?
Replies
4
Boosts
0
Views
86
Activity
6d
Auv3 Synth Zippering
I am running into zippering and zipper storms in making my Auv3 synth. Do you have advice? It is a big synthesizer, big cinematic sounds, big presets, dual mode, I am mostly hitting zipper storms when trying to play fast on big preset recipes. Is there something I need to know to avoid zippering?
Replies
1
Boosts
0
Views
69
Activity
6d
RemoteMediaSession device add/remove
Are there plans to extend the session or MediaDevice capabilities to be able to add and remove speakers via the Lock Screen and Control Center? For example, user may have multiple speakers, they start playback on a subset of them and would like to add other speakers or remove some from the session through Control Center.
Replies
1
Boosts
0
Views
101
Activity
6d
My biggest priorities after 9 months of shipping Albums for macOS
Hi everyone! Adam here, the developer of Albums. Bummed not to get to have our yearly WebEx reunion where I beg y’all to add an endpoint to the Apple Music API to allow deleting items from a user’s library. But that’s not what this thread is about. Back in October, thanks to your team’s multi-year efforts, I was finally able to ship Albums for macOS. It’s been a genuine dream come true to use it on the platform I always envisioned it on, and I hear from users all the time who feel the same way. I know it was a significant engineering effort, and I’m genuinely very grateful. I’ll be judicious with my time here (I’d love to chat in more detail about my adventures in MusicKit on the Mac somehow, sometime). These are the main things on my wishlist for MusicKit on the Mac. There are three main things I hear from users that my app “can’t do” that can be dealbreakers for them. The first two are AirPlay support and a volume slider. Users are only able to AirPlay using the control center utility, because AVRoutePickerView does not work with ApplicationMusicPlayer (FB13934910). Relatedly, the MPVolumeView does not work with ApplicationMusicPlayer (FB21042385), so I can’t allow users app-specific volume control for my app. The lack of those two things is a real detriment to my app being able to be taken seriously alongside all of the other music player apps on the platform for which those features are table stakes. I know there’s a challenge here given the playback actually happens in the subprocess, but hoping some progress can be made here. The other issue is that library tracks played in ApplicationMusicPlayer do not update the last played date or play count in Music.app or in the user’s iCloud Music Library (FB17675148). Some people refuse to use the app for that reason, and I can’t say I blame them. I’ve only been able to test this briefly in Golden Gate, but it seems like this is still the case. Are you able to share anything about your work on the music library in macOS this year? Thank you again for all your work on MusicKit! I’m planning to get the last of the load-bearing MediaPlayer code out of my codebase later this year. Hooray!
Replies
1
Boosts
0
Views
92
Activity
6d
Slow MusicKit library performance in Golden Gate beta 1
Hello friends! Happy WWDC. Thanks very much for all your work on MusicKit this year! I figure I’ll start things off with a bug report (sorry!). I filed a Feedback earlier today that music library operations in MusicKit are significantly slower in macOS Golden Gate beta 1 than in Tahoe. For example, a .with([.tracks]) operation on an Album takes 4-5 seconds rather than the 95ms it did in Tahoe. Sample project, traces, and sysdiagnoses in FB23037115.
Replies
1
Boosts
0
Views
68
Activity
6d
Audio Prioritization
Is there anyway that we as users can set apps to not be able to play audio? There have been times that I want to listen to music and then scroll specific social media apps, and the social media apps keep interrupt the audio. I made a feedback related to this last year: FB18494861
Replies
0
Boosts
0
Views
73
Activity
6d
microphone level monitoring
My macOS app provides real-time microphone level monitoring, and I also have a macOS widget that shows recent audio level status. Since WidgetKit widgets are snapshot/timeline based rather than continuously live SwiftUI views, is there any new capability in macOS 27 that allows a widget to show more responsive or animated audio-level changes? If true real-time animation is still not supported in widgets, what is Apple’s recommended approach for displaying frequently changing signal levels, such as audio meters, in a macOS widget?
Replies
2
Boosts
0
Views
117
Activity
6d
CATaps
I’ve noticed that CATap appears to scale the stereo/mono mixdown based on the number of channels in the output audio device. For example, if the selected output device has 4 channels, the tapped audio is reduced by half. This behavior does not seem to be documented anywhere, but I’m assuming it may have been done to reduce the chance of clipping when downmixing. The issue is that users are complaining that the audio level is too low. Is there any workaround for this that does not involve tapping every audio device separately and doing my own mixdown? Ideally, I’d like to keep using the system tap/mix behavior but avoid the automatic level reduction, or at least compensate for it in a reliable way.
Replies
1
Boosts
0
Views
61
Activity
6d
What's the recommended way for a Mac app to receive audio from other apps in 2026?
My app allows users to transcribe and translate audio from other apps running on your Mac. I currently use ScreenCaptureKit to do this, and just ignore the video stream and only use the audio stream. This works, but it's awkward because it requires I ask users for screen recording permission, even though all I want is the audio, not the screen. Is there a better way to do this in modern macOS releases? Thanks!
Replies
1
Boosts
0
Views
65
Activity
6d
New text dictation system available to 3rd party apps?
In the keynote (or maybe SOTU) they demo'd a new text dictation system. I'm wondering if this is being added to the Speech framework as a new transcription mechanism available to 3rd party apps. I couldn't find any videos referring to updates to the Speech framework.
Replies
0
Boosts
0
Views
34
Activity
6d
AudioServerPlugin AddDeviceClient/RemoveDeviceClient calls.
I added logging in an AudioServerPlugin to trace the arrival and removal of a particular client in the AddDeviceClient() and RemoveDeviceClient() methods. To my surprise the app was removed and added as a client often, even when it was in a steady state, for example when using the Audio MIDI Setup to change sample rates. Is this expected behavior, and if so, are there any explanations about this.
Replies
0
Boosts
0
Views
37
Activity
6d
New Siri text-to-speech available to 3rd party apps?
In the keynote (or maybe it was SOTU) they demo'd a new more expressive Siri voice. Is that voice going to be available to 3rd party apps to use for text-to-speech in our apps?
Replies
0
Boosts
0
Views
39
Activity
6d
Optimization of multiple audio track synchronization.
I have an app where multiple audio files of the same length are played and they need to be in sync to produce the full song. My current setup syncs each track to the same position every time the user pauses or plays the song. However it is super laggy when the Audio is played either for the first time or when unpausing/pausing. What's the best way to do this?
Replies
2
Boosts
0
Views
94
Activity
6d
Is there a supported way for apps on iOS to receive audio from other apps?
I make a Mac app where users can transcribe and translate audio from other apps. I want to bring this to iOS, and I'm wondering what APIs are available (if any) to do this. On MacOS I use ScreenCaptureKit currently, but that's not available on iOS.
Replies
0
Boosts
0
Views
61
Activity
6d
How to Seamlessly Handle FIDO QR Codes in Your iOS App
When scanning a FIDO QR code within an iOS app—whether using a custom AVFoundation (AVCaptureSession) implementation or DataScannerViewController—the system displays a native OS confirmation prompt. However, scanning the same QR code using the native system Code Scanner bypasses this prompt entirely. As a developer: Is there a way to suppress or avoid this native prompt when using custom in-app scanners? Alternatively, can I programmatically invoke the system Code Scanner directly from my app and have it deep-link back to the app once the scan is complete?
Replies
0
Boosts
0
Views
54
Activity
6d