Explore the integration of media technologies within your app. Discuss working with audio, video, camera, and other media functionalities.

All subtopics
Posts under Media Technologies topic

Post

Replies

Boosts

Views

Activity

RemoteMediaSession started while the app is in the background no longer appears in Control Center (worked in earlier betas)
My app publishes a NowPlaying.RemoteMediaSession for music playing on speakers on the local network. While the app is in the background and someone taps play on my widget, that intent runs in my app process and wakes the app in the background — it can fetch the speaker state and start and update the remote session without any trouble. As of Beta 6, the session is created successfully but never shows up in Control Center or on the Lock Screen. It only appears the moment I bring my app to the foreground. In earlier betas the same code showed the session in control center, without needing to open the app. I don't call requestToBecomeSystemPrimary() and I don’t expect it to be set as primary, only when my app is in the foreground but at least the session should appear in control center or the lock screen as an option for the user to set it to system primary. FB24439526 has been submitted with Sysdiagnose attached.
1
2
171
2w
On iOS 26.6.1 and 26.6.2, Exif (XResolution / YResolution) values become 0 when an image attached in the Gmail app is saved to the Photos app
My app is built with Xcode 26.1 and uses a customized photo picker implemented with the PhotoKit Framework. After saving an image attached in the Gmail app to the Photos app on iOS 26.6.1 or iOS 26.6.2, when my app selects and loads that image, the Exif XResolution and YResolution values are both reported as 0. Is this an iOS bug? The following code retrieves the XResolution and YResolution values from the PHAsset of the selected image. private func selectAsset(_ asset: PHAsset) { let options = PHImageRequestOptions() options.isNetworkAccessAllowed = true options.deliveryMode = .highQualityFormat PHImageManager.default().requestImageDataAndOrientation(for: asset, options: options) { data, _, _, _ in DispatchQueue.main.async { if let data { let options = PHImageRequestOptions() options.isNetworkAccessAllowed = true options.deliveryMode = .highQualityFormat PHImageManager.default().requestImageDataAndOrientation(for: asset, options: options) { data, _, _, _ in guard let data, let source = CGImageSourceCreateWithData(data as CFData, nil), let properties = CGImageSourceCopyPropertiesAtIndex(source, 0, nil) as? [CFString: Any], let tiff = properties[kCGImagePropertyTIFFDictionary] as? [CFString: Any] else { print("XResolution: NaN, YResolution: NaN") return } let xResolution = tiff[kCGImagePropertyTIFFXResolution] as? Double let yResolution = tiff[kCGImagePropertyTIFFYResolution] as? Double print("XResolution: \(xResolution.map { String($0) } ?? "NaN"), YResolution: \(yResolution.map { String($0) } ?? "NaN")") } } } } } Result on iOS 26.6 XResolution: 72, YResolution: 72 Result on iOS 26.6.1 and 26.6.2 XResolution: 0, YResolution: 0
1
0
144
2w
AVAssetDownloadConfiguration does not download supplemental tracks on iOS 16
We're downloading HLS content for offline playback using AVAssetDownloadConfiguration. It's critical for our app that all subtitle tracks are downloaded together with the video, so users can watch any subtitle and audio option while offline. On iOS 16, not all tracks are downloaded, and when the user goes offline, playback fails with NSURLErrorNotConnectedToInternet (error -1009) because AVFoundation tries to fetch the missing subtitle segments over the network. On iOS 17 and later, everything works as expected — all subtitle and audio tracks are downloaded. However, on iOS 16 we observe two issues: First issue: Supplemental tracks are not downloaded on iOS 16 (device) Only the default audio track is downloaded. All non-default supplemental tracks — both subtitles and additional audio tracks — exhibit the same problem: Some supplemental tracks listed in the manifest don't even get a directory created in the .movpkg package Some supplemental tracks do get a directory created, but it contains only StreamInfoBoot.xml and StreamInfoRoot.xml — no actual segment data (.frag files) is downloaded. The StreamInfoBoot.xml for these directories shows NO, 0, and 0, with Supplemental Additionally, a Manifest.m3u8 stream directory is created with Complete=NO and no segment data. On iOS 17+ devices, all supplemental track directories are created and all contain segment data. Second issue: Downloads fail entirely on iOS 16 Simulator We receive the following error: Error Domain=NSURLErrorDomain Code=-16090 UserInfo={ NSLocalizedDescription=The operation could not be completed, _NSURLErrorFailingURLSessionTaskErrorKey=BackgroundAVAssetDownloadTask <DEC0D555-6C05-416C-91B8-F490F5A29046>.<3>, _NSURLErrorRelatedURLSessionTaskErrorKey=( "BackgroundAVAssetDownloadTask <DEC0D555-6C05-416C-91B8-F490F5A29046>.<3>" ), NSLocalizedFailureReason=An unknown error occurred (-16090) } On iOS 17+ Simulators, downloads work without issues. Our setup: let urlAsset = AVURLAsset(url: manifestURL) let preferredMediaSelection = try await urlAsset.load(.allMediaSelections) let config = AVAssetDownloadConfiguration(asset: urlAsset, title: title) config.primaryContentConfiguration.mediaSelections = preferredMediaSelection let task = assetDownloadURLSession.makeAssetDownloadTask(downloadConfiguration: config) task.resume() Example HLS manifest: #EXTM3U #EXT-X-VERSION:7 #EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="French",LANGUAGE="fr",DEFAULT=NO,AUTOSELECT=YES,FORCED=NO,URI="s_25/stream.m3u8" #EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="French (forced)",LANGUAGE="fr-x-cc",DEFAULT=NO,AUTOSELECT=YES,FORCED=YES,URI="s_31/stream.m3u8" #EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="English",LANGUAGE="en",DEFAULT=NO,AUTOSELECT=YES,FORCED=NO,URI="s_37/stream.m3u8" #EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="Kazakh",LANGUAGE="kk",DEFAULT=NO,AUTOSELECT=YES,FORCED=NO,URI="s_43/stream.m3u8" #EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="Uzbek",LANGUAGE="uz",DEFAULT=NO,AUTOSELECT=YES,FORCED=NO,URI="s_49/stream.m3u8" #EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="aud_ec3",NAME="French 5.1",LANGUAGE="fr",DEFAULT=YES,AUTOSELECT=YES,CHANNELS="6",URI="a_18/stream.m3u8" #EXT-X-STREAM-INF:AUDIO="aud_ec3",SUBTITLES="subs",BANDWIDTH=1931408,VIDEO-RANGE=SDR,CODECS="hvc1.1.6.L90.90,ec-3",RESOLUTION=1024x428,FRAME-RATE=25,HDCP-LEVEL=NONE v_6/stream.m3u8 #EXT-X-SESSION-KEY:METHOD=SAMPLE-AES,URI="skd://XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX",KEYFORMAT="com.apple.streamingkeydelivery",KEYFORMATVERSIONS="1" Questions: What is the correct way to make sure all supplemental tracks (subtitles, non-default audio) are actually downloaded on an iOS 16 real device? We pass asset.load(.allMediaSelections) to primaryContentConfiguration.mediaSelections — is there a different configuration, or should we use auxiliaryContentConfigurations, or something else entirely? Is there any way to make asset downloads work on the iOS 16 Simulator at all? We always get NSURLErrorDomain error -16090, while iOS 17+ Simulators work fine. If the supplemental tracks download bug on iOS 16 cannot be fixed or worked around from the app side, do you have any recommended approach for handling incomplete .movpkg packages at offline playback time?
1
0
394
2w
Variable aperture on the iPhone 18 Pro
Does iOS 27 provide an API which allows developers to set the iPhone 18 Pro camera's aperture? If it does, is the aperture going to be limited to just 4 discrete settings (like in the Camera app and like mentioned in the Tech Specs for the 18 Pro) or does the hypothetical iOS API allow us to set the aperture to a continuous value in the 0.0–1.0 range (similar to how a camera's focus can be set)?
2
1
657
2w
Capturing 48mp photos with .builtInTripleCamera
I am able to capture 48mp photos using .builtInWideAngleCamera, but it seems like .builtInTripleCamera is capped at 12mp? Is there a way to capture 48mp photos using .builtInTripleCamera? Because .builtInTripleCamera provides smooth transition between cameras during zooming, and I'd like to keep this behavior. New iPhone 17 Pro have all their cameras at 48mp. Is there a chance that their .builtInTripleCamera is capable of capturing 48mp? Or is this an API limitation?
4
0
975
2w
Working with kCVPixelFormatType_96VersatileBayerPacked12
Whilst AVCaptureSession is setup to capture ProRes RAW video, is it possible to get video pixel data which can read and processed, such as using CIImage(cvPixelBuffer: ) AVCaptureVideoDataOutput outputs ProRes RAW in kCVPixelFormatType_96VersatileBayerPacked12 pixel format. Is there a provided way to debayer this pixel format into something more usable?
1
1
880
2w
Supported MusicKit APIs for vinyl-style speed, pitch and reverse playback of subscription songs on iOS?
I am building a personal iPhone prototype with an on-screen record platter. Before enrolling in the Apple Developer Program or developing the Apple Music integration further, I would like to establish whether the core interaction is supported by public APIs. The intended interaction is: Tap the platter to play or pause an Apple Music subscription song. Hold the platter to stop playback at its current position. Move the platter forward or backward to control the direction and speed of audible playback continuously. Change pitch together with speed, like a physical record, rather than preserving the original pitch. Release the platter to smoothly return to normal playback. Hold a separate control to listen continuously at a faster speed. A prototype using an AVAudioEngine-based renderer works with unprotected local audio. I have NOT verified these behaviors with Apple Music subscription content. I understand that this local renderer does not establish feasibility for MusicKit, and I am not seeking to extract protected audio or bypass content protection. MusicPlayer.State.playbackRate is documented as settable, with positive values for forward playback and negative values for reverse playback: https://developer.apple.com/documentation/musickit/musicplayer/state-swift.class/playbackrate For ApplicationMusicPlayer on iOS, is this vinyl-style interaction supported for full Apple Music subscription songs, rather than previews or user-owned unprotected files? In particular: What playback-rate range is supported for subscription songs? Do negative rates produce audible reverse playback, and how should zero-speed holding be implemented? Is there a documented public API to make pitch follow playback speed, or to disable pitch preservation? Are frequent rate and direction changes supported for interactive scratching? Are there documented limitations on update frequency, buffering, seeking precision or responsiveness? If this interaction is outside public MusicKit capabilities, is there an official channel where an individual developer can ask about eligibility for an approved Apple Music DJ integration? I am asking for the supported API path and its limitations, not implementation details of another app. If only ordinary variable-speed playback or seeking is supported, a clear distinction from continuous vinyl-style scratching would be very helpful. The initial goal is personal use; possible App Store distribution is undecided. I have an active Apple Music subscription but am not currently enrolled in the paid Apple Developer Program. Thank you.
0
0
321
3w
iOS27, CIRAWFilter RAW9 broken/missing in latest dev beta
In the WWDC talk: https://developer.apple.com/videos/play/wwdc2026/305/?time=350 "Enhance RAW image processing with Core Image" there is a section on the new RAW9 processing APIs and how to enable them. First off, the sample code compiles on iOS18 but causes a runtime exception because the .version9 symbol cannot be found, looks like the header definition is missing a version check for this value, so I'm not sure how to use this code if you need to support iOS18. Also on later dev betas (I tried the latest beta7) now trying to check if version9 is available the value is no longer present in the supported list: guard filter.supportedDecoderVersions.contains(.version9) else { ... There is another value .version9DNG that resolves to true but if I try to use that decoder the colors on the RAW image are wrong and the image becomes corrupted. How to use the RAW9 decoder in the latest dev betas, what should the final code before production because the WWDC sample now seems out of date and was always wrong wrt. iOS18.
3
0
1.5k
3w
Building a library playlist on macOS that mixes catalog songs and the user's own uploads in a fixed order – supported route, and how to know when an upload is registered?
Our sandboxed macOS app digitises audio cassettes and builds one playlist per cassette in the user's Apple Music library that follows the tape's order: catalog songs where a title was recognised, and the user's own recordings (AAC files the app exported from the tape) where it was not. The user has an Apple Music subscription and Sync Library on. On macOS every write on MusicLibrary is marked @available(macOS, unavailable) in the 26.5 SDK – add, add(_:to:), createPlaylist and edit (see thread 844114, which got no answer). So we build the playlist through Music.app's scripting interface: make new user playlist, duplicate <library track> to <playlist> for catalog songs, add <file> to <playlist> for own recordings. That works, with one exception that leads to our questions. What we observe Reproducible on macOS 26.6.2 / Music 26; a standalone AppleScript is at the end. A playlist built in one go from 14 subscription tracks keeps all 14. The same playlist with one local file added after the second track: all 15 entries are there right after the build and one second later. About 15 s later the playlist has 3 entries – the two catalog entries before the file, the file, and nothing that was added after it. No error anywhere. If the file has been in the library for several minutes before the playlist is built, everything stays. Adding it to the library and waiting 45 s is not enough. Meanwhile the track's cloud status stays unknown, and GET /v1/me/library/search?types=library-songs does not list it (checked for 10 minutes). Emptying the playlist and building it again ~30 s after the entries were removed keeps everything, every time. That is what we do today; it costs 30–40 s per import, and we have to tell the user that entries were removed and put back. Our reading: while the freshly added file is not yet registered in iCloud Music Library, the playlist as pushed to the server is cut at the first item the server cannot reference, and the next sync adopts the shorter list for cloud items while keeping the local-only item in place. Questions Is there a supported way for a macOS app to create a library playlist and add tracks to it? Specifically: is the Apple Music API (POST /v1/me/library/playlists with relationships.tracks, and POST /v1/me/library/playlists/{id}/tracks) the intended route from a macOS app holding a MusicKit user token, and can such a playlist reference the user's own uploaded songs by their library id (i.…)? After a local file has been added to the library (via Music.app's add, or any other supported way), how can an app learn that iCloud Music Library has registered it, and what its library song id is? A notification, a MusicKit property, an Apple Music API endpoint? We would wait on that signal instead of rebuilding. Is the removal described above expected behaviour? Catalog ids are re-resolved at import time via id, ISRC and search as recommended in thread 122110, so stale catalog ids are not the cause. Reproduction Needs Sync Library on, at least 14 subscription tracks in the library, and a local audio file the library does not know yet. Prints the counts after 1 s, 21 s and 41 s, then cleans up. on run argv set localFile to POSIX file (item 1 of argv) tell application "Music" set cloudTracks to (every track of library playlist 1 whose cloud status is subscription) set idsBefore to persistent ID of every track of library playlist 1 set pl to make new user playlist with properties {name:"Reconciliation repro"} repeat with i from 1 to 2 duplicate (item i of cloudTracks) to pl end repeat set fileTrack to add localFile to pl set fileID to persistent ID of fileTrack repeat with i from 3 to 14 duplicate (item i of cloudTracks) to pl end repeat set n0 to count of tracks of pl delay 1 set n1 to count of tracks of pl delay 20 set n2 to count of tracks of pl delay 20 set n3 to count of tracks of pl delete pl if fileID is not in idsBefore then delete (first track of library playlist 1 whose persistent ID is fileID) return "after build: " & n0 & ", after 1 s: " & n1 & ", after 21 s: " & n2 & ", after 41 s: " & n3 end tell end run Output here: after build: 15, after 1 s: 15, after 21 s: 3, after 41 s: 3.
0
0
327
3w
visionOS Developer Strap: UVC video works, but how can I access the capture device’s USB Audio Class (UAC) audio?
I’m developing a native visionOS application that uses an external HDMI capture device connected to Apple Vision Pro through the Developer Strap. The goal is to use Apple Vision Pro as a low-latency display for external HDMI sources such as an Xbox Series X, including both the HDMI video and the original digital game audio. My current hardware path is: Xbox Series X → HDMI → Elgato 4K X → USB-C → Apple Vision Pro Developer Strap The Elgato 4K X receives both video and audio from the Xbox over HDMI. What is working UVC video capture is working successfully on visionOS. The external Elgato capture device can be used as the video source and the Xbox video is displayed correctly inside the Vision Pro application. I have also verified the Elgato 4K X independently on macOS, where the device is recognized correctly. The problem I cannot find a supported way on visionOS to access the USB Audio Class (UAC) audio interface associated with the capture device. I specifically need the digital game audio originating from: Xbox HDMI → Elgato 4K X → USB/UAC → visionOS I do not want to use the Apple Vision Pro’s built-in microphone. Selecting/capturing the Vision Pro microphone would only record environmental sound and is not a substitute for the digital HDMI audio stream. Apple’s visionOS documentation for external devices through the Developer Strap appears to focus on UVC video capture. I haven’t been able to determine whether the accompanying UAC interface of a UVC/UAC composite capture device is exposed to visionOS applications. What I’ve tried / verified Connected an Xbox Series X to the HDMI input of an Elgato 4K X. Connected the 4K X to Apple Vision Pro through the Developer Strap. Successfully discovered and received the external UVC video stream. Successfully displayed the Xbox video feed in visionOS. Confirmed that the 4K X itself is recognized and functional on macOS. Attempted to obtain audio corresponding to the external capture source. The Vision Pro microphone is available as an audio source, but this is not the audio source required. I have not found a documented visionOS API that exposes the 4K X’s USB/UAC audio endpoint through the Developer Strap. Questions Does visionOS currently support USB Audio Class input devices connected through the Apple Vision Pro Developer Strap? If a device exposes both UVC video and UAC audio interfaces, is there a supported way to associate/access both interfaces from a visionOS application? If UAC is supported, which framework/API should be used to discover the external audio endpoint — AVFoundation, Core Audio, or another API? Does accessing the UAC interface require an entitlement in addition to UVC device access? If generic UAC access through the Developer Strap is currently unavailable to third-party visionOS applications, is there a restricted/managed entitlement that developers can request? If this isn’t currently supported, what is Apple’s recommended architecture for receiving synchronized video and audio from an HDMI capture device on Apple Vision Pro? Ultimately, I’m trying to achieve: HDMI source → UVC video + UAC audio → visionOS application → video displayed in Vision Pro + audio played through Vision Pro speakers/AirPods Since UVC video from this exact hardware is already working, I’m primarily trying to determine whether access to the accompanying USB audio interface is possible with the current visionOS SDK, requires another entitlement/API, or is currently unsupported. Any guidance from Apple or developers who have worked with composite UVC/UAC devices on Vision Pro would be greatly appreciated.
0
1
622
3w
Request path for com.apple.coremedia.allow-mpeg4streaming on tvOS
We maintain an existing iOS/tvOS App Store application that uses AVFoundation for authenticated streaming playback. One upstream provider supplies some account-entitled playback sessions as fragmented MP4 using ISO Common Encryption scheme cenc (AES-CTR). The provider controls the rendition, and no corresponding cbcs rendition is available to our client. We understand that Apple’s documented FairPlay Streaming path supports cbcs. We identified com.apple.coremedia.allow-mpeg4streaming as relevant to this playback case, but the entitlement is undocumented and does not appear under Capability Requests for our App ID. Apple Developer Support was unable to identify the request process and recommended posting this question in the forums and filing Feedback Assistant report FB24649355. Could an Apple engineer clarify: Is this entitlement available to third-party iOS/tvOS applications? If so, what is the official request process? We can provide additional technical information privately if an Apple engineer needs it.
1
0
1.1k
3w
Volumes issue
My volume will randomly go all the way down to mute or all the way up to full blast while I’m listen to music, watching a video, playing a game. I have all the volume safety features turned off. Does anyone else have this issue?
1
0
424
3w
iOS 27 beta: setExposureModeCustom silently ignored on the front camera
On every iOS 27.0 beta, setExposureModeCustom has no effect on the front camera (.builtInTrueDepthCamera). exposureMode does change to .custom, but the sensor keeps auto-exposing: device.iso and device.exposureDuration report auto-exposure's values, not the requested ones, and frame brightness does not change. No error is thrown. The same code works on iOS 18.x and 26.x on the same devices. Device: iPhone 15 Pro, every iOS 27.0 beta. Also seen on iPhone 14 Pro Max, 16, 16 Pro Max and 17 Pro Max. Session: front camera, AVCaptureVideoDataOutput (BGRA) + AVCaptureDepthDataOutput. Format advertises ISO 20...1920, duration 0.00004...0.06667 s. isExposureModeSupported(.custom) is true. Repro: let auto-exposure settle, then try device.lockForConfiguration() device.setExposureModeCustom(duration: CMTime(value: 1, timescale: 15), iso: 1920, completionHandler: nil) device.unlockForConfiguration() then poll device.iso / device.exposureDuration (the completion handler never fires on this device) and measure frame luma. Readback, polled 0.8 s after the call: requested ISO 1920, 66.7 ms -> reported ISO 349, 60.0 ms requested ISO 20, 0.04 ms -> reported ISO 188, 66.5 ms Requesting the shortest exposure returns the longest, and the same ISO 1920 request returned 249, 349 and 450 on three consecutive runs, so these are auto-exposure's values. Frame luma stayed within 1% between the two phases. setExposureTargetBias, however, is honored on the same device: at +4 EV auto-exposure drives the sensor to ISO 1920 / 66.5 ms and at -4 EV to ISO 24 / 30.0 ms, moving mean luma from 49 to 232. So the sensor does reach the values setExposureModeCustom cannot set. Question: is this intentional or a regression, and how should an app detect that a custom exposure request was refused?
2
0
655
3w
Does the TN3135 audio-session networking exception have a defined lifetime? Seeing a ~38.5 s revoke/re-grant cycle
TN3135 describes the exception that lets a watchOS app use low-level networking while it holds an active audio session. I have that working, and the app functions — but the network path is withdrawn and restored on a strikingly regular cycle, and I would like to know whether that is expected behaviour rather than something I am doing wrong. Setup Apple Watch Series 10 (Watch7,9), watchOS 26.5. Reproduced on a Series 6 (Watch6,2). UIBackgroundModes: [audio]; AVAudioSession category .playAndRecord, mode .spokenAudio; activated with the async activate(options:completionHandler:). NWConnection with NWProtocolWebSocket to a WebSocket relay over TLS. The app streams 16 kHz mono PCM continuously while transmitting and holds the socket open otherwise. Symptom NWPathMonitor reports .unsatisfied, then .satisfied about two seconds later, over and over. Measured with the iPhone powered off, so the watch was on its own Wi-Fi: Uptime between drops Outage 36.4 s 2.1 s 36.7 s 1.9 s 36.9 s 2.1 s The regularity is what prompts the question — uptime varies by ±0.3 s and the outage is consistently 2.0 s. That reads as a timeout expiring rather than radio behaviour. What I have ruled out Not the network or the server. A browser client on the same relay, same TLS, same wire protocol, holds a WebSocket indefinitely. Not the interface. Identical cadence over the companion ipsec1 tunnel with the iPhone present, and over the watch's own en0 with the iPhone powered off. Pinning requiredInterfaceType = .wifi while the iPhone is reachable fails outright — the path offers only ipsec1. Not audio-session interruption. I observe interruptionNotification, routeChangeNotification, mediaServicesWereResetNotification and silenceSecondaryAudioHintNotification. None fire at a drop. At the moment the path goes .unsatisfied, the engine is running and the player node is actively playing. Not session idleness. Playing continuous silence for the whole session, rather than only while reconnecting, made no difference — still 36.4 s. The control that surprised me To test whether this affects any long-lived watch socket or only audio-unlocked ones, I built a second app with no AVAudioSession at all, no audio background mode, holding a URLSessionWebSocketTask and kept alive by a WKExtendedRuntimeSession so screen sleep was not a factor. It never connected. NWPathMonitor reported .unsatisfied once and never changed, across a 30 s run, and every request failed with "The Internet connection appears to be offline." I had expected URLSession to be permitted regardless. Questions Does the audio-session networking exception in TN3135 have a defined lifetime, and is a periodic revoke/re-grant cycle expected? If so, is there a supported way to hold it continuously — or is the correct design simply to expect the interruption and reconnect through it? Is it expected that an app with no audio session gets no network path at all on watchOS, including via URLSession, even in the foreground with an extended runtime session?
16
0
3.6k
3w
AirPods Sleep Detection pauses meditation playback with no API/callback to identify the reason
Good morning! I work on a meditation app, and since Sleep Detection was introduced for AirPods in iOS 26, we've been receiving complaints from users reporting that their meditation sessions unexpectedly stop after approximately 15–20 minutes when Sleep Detection is enabled. This seems to happen particularly during deep meditation sessions, where the user may remain still and exhibit behavior that could potentially be interpreted as falling asleep. I've also tested this on iOS 27 with the latest beta firmware available for AirPods Pro, and the same behavior still occurs. While investigating and debugging the issue, I found that there doesn't appear to be any public API that allows an app to determine that playback was paused specifically because of AirPods Sleep Detection. From the application's perspective, the AirPods appear to simply send a standard pause command to the AVPlayer, without providing any additional context or reason. There also doesn't seem to be an API that allows us to determine whether the user has Sleep Detection enabled. Even having access to that information would allow us to warn users that their meditation session could potentially be interrupted. AVAudioSession.InterruptionReason does not seem to help in this case either: https://developer.apple.com/documentation/avfaudio/avaudiosession/interruptionreason Is there currently any supported way for an app to: Detect that playback was paused specifically because of AirPods Sleep Detection? Determine whether Sleep Detection is enabled for the connected AirPods? Prevent Sleep Detection from pausing playback for specific apps or specific types of audio sessions? If none of these are currently possible, could Apple consider providing either a notification/callback indicating that Sleep Detection triggered the pause, or an option for users to exclude specific apps from Sleep Detection? This behavior is particularly problematic for meditation apps, since a user in a deep meditation session can easily be mistaken for someone who has fallen asleep. Other meditation apps appear to be affected by the same behavior as well. Best regards, Carlos Antunes
1
0
1.1k
3w
MusicKit on HarmonyOS — playback approach for a platform without an official SDK, licensing question
Hello Apple Developer community, I'm building an Apple Music client for HarmonyOS (Huawei's operating system) and would like to clarify the licensing and technical approach before submitting to a third-party app store (Huawei AppGallery). I've read the Apple Developer Program License Agreement and App Store Review Guideline 4.5.2, but there are platform-specific constraints I'd like to discuss. Background Apple currently does not provide a MusicKit SDK for HarmonyOS. The available options are: MusicKit for iOS/macOS — not applicable (different OS) MusicKit for Android — ships as proprietary .aar binaries that cannot be redistributed and are not compatible with HarmonyOS's runtime MusicKit JS — available, but HarmonyOS WebView does not support FairPlay or Widevine EME, so full-song DRM playback through MusicKit JS does not work on this platform The only Apple Music API access path available to a HarmonyOS app is the REST API (api.music.apple.com) with a Developer Token (JWT, ES256) and a Music User Token obtained through the MusicKit JS authorization flow running inside a WebView. The technical constraint To deliver full-song playback for subscribed Apple Music users on HarmonyOS, our app retrieves the HLS playback URL via the Apple Music API, downloads the encrypted media segments, and decrypts them using HarmonyOS's native DRM system (WisePlay, the only CDM available on the platform). MusicKit JS's FairPlay/Widevine CDM is not compatible with HarmonyOS — WisePlay is the sole DRM implementation the OS exposes. We are aware that Section 4.5.2 states: "You may not, and You may not permit Your end users to, download, upload, or modify any MusicKit Content" and "You may play MusicKit Content only as rendered by the MusicKit APIs or MusicKit JS" What we want to clarify Our playback implementation works as follows, and we want to confirm whether it is acceptable under the license terms: No download service is offered to users. The download is purely an internal playback mechanism — the app fetches encrypted segments, decrypts them in memory / a protected sandbox, and plays them via the system AVPlayer. There is no "download button," no offline library, no user-facing download feature. Decrypted files are ephemeral and protected. Decrypted media is written to the app's private sandbox (not accessible to users or other apps), played immediately, and deleted as soon as playback ends or the song is skipped. The decrypted file never persists beyond the current playback session. No file leakage. The decrypted content is never exposed via any sharing mechanism, file picker, or external storage. It exists only within the app's protected data directory for the duration of playback. Standard playback controls are preserved. Users initiate playback and can use play / pause / skip / seek — the same controls required by 4.5.2(i). No monetization of Apple Music access. The app does not charge for Apple Music playback, does not show ads around it, and does not collect or share user data. Our question Given that: Apple has no SDK presence on HarmonyOS, MusicKit JS DRM is technically non-functional on HarmonyOS WebView, the only way to deliver full-song playback to Apple Music subscribers on this platform is to decrypt the stream using the OS's native DRM (WisePlay) with strict ephemeral handling, is this playback approach acceptable under the Apple Developer Program License Agreement, provided that: decrypted content is never persisted, shared, or exposed to users, the app only serves subscribed Apple Music users with valid Music User Tokens, all usage remains within the scope of facilitating access to the user's own Apple Music subscription? We fully respect Apple's content protection requirements and are happy to adjust the implementation (e.g., stricter deletion timing, additional sandboxing, attestation) to comply. We just want to understand whether the fundamental approach — decrypting via the host OS's DRM because Apple's own CDMs are unavailable on this platform — is permissible, or whether Apple considers HarmonyOS out of scope for full-song playback entirely. Any guidance from the MusicKit team or community members who have dealt with non-Apple-platform constraints would be greatly appreciated. Thank you.
0
0
360
3w
Is there a supported way to read on-screen text with the user's consent? (worker-safety / income-transparency app)
Hello, I am an independent developer in Argentina building an app for rideshare drivers, and I would like to know what the supported approach is on iOS before I build the wrong thing. The problem When a trip offer appears, the driver sees a fare, a distance and a duration. What they do not see is what is left after the cost of running their own car — fuel, maintenance, and very often the weekly rent they pay for the vehicle. Here those costs are high and change constantly, and drivers routinely accept trips that lose them money without realising it. They have about three seconds to decide, while driving. Our app reads the numbers already visible on the driver's own screen, subtracts that particular driver's cost per kilometre, and shows one figure: what this trip actually leaves them per hour. In our own field measurements with two drivers over several weeks, being able to see that number raised their hourly earnings and — just as important — kept their eyes off mental arithmetic at the wheel. What we will not do, on purpose I want to be explicit about this, because I know other tools in this category cut corners: We never ask for the driver's Uber or DiDi username or password, and we never connect to or automate their account. Some tools do, and that puts the driver's account — their livelihood — at risk of deactivation. We will not build that. We never accept or decline a trip for the driver. The decision and the tap are always theirs. We never store passenger data: no name, no rating, no photo, no exact address. Text recognition runs entirely on device, offline. The captured image is discarded immediately and never leaves the phone or reaches any server. No advertising, no data sales, no third-party trackers. The app does not read anything the driver is not already looking at, it only reads while the driver has explicitly turned it on for their shift, and it exists for one purpose: so that someone working long hours knows what an hour of their work is actually worth. My question On Android this is done with MediaProjection, with explicit user consent each session and a persistent system indicator. On iOS I understand a Broadcast Upload Extension is not intended for background processing, and that there is no overlay across apps. Is there any supported path on iOS for an app to read, with the user's explicit and repeated consent, text the user is already looking at on their own screen, in order to give that same user an immediate assessment of their own work? Accessibility APIs, a system extension point, anything. If there is no supported path, I would rather know now and design the iOS version around what is allowed, instead of shipping something that gets rejected or that quietly breaks the rules. Thank you.
1
0
382
3w
Iphone Duo
Does the iPhone Duo support spatial video recoding for playback on Apple Vision Pro?
Replies
0
Boosts
0
Views
119
Activity
2w
RemoteMediaSession started while the app is in the background no longer appears in Control Center (worked in earlier betas)
My app publishes a NowPlaying.RemoteMediaSession for music playing on speakers on the local network. While the app is in the background and someone taps play on my widget, that intent runs in my app process and wakes the app in the background — it can fetch the speaker state and start and update the remote session without any trouble. As of Beta 6, the session is created successfully but never shows up in Control Center or on the Lock Screen. It only appears the moment I bring my app to the foreground. In earlier betas the same code showed the session in control center, without needing to open the app. I don't call requestToBecomeSystemPrimary() and I don’t expect it to be set as primary, only when my app is in the foreground but at least the session should appear in control center or the lock screen as an option for the user to set it to system primary. FB24439526 has been submitted with Sysdiagnose attached.
Replies
1
Boosts
2
Views
171
Activity
2w
On iOS 26.6.1 and 26.6.2, Exif (XResolution / YResolution) values become 0 when an image attached in the Gmail app is saved to the Photos app
My app is built with Xcode 26.1 and uses a customized photo picker implemented with the PhotoKit Framework. After saving an image attached in the Gmail app to the Photos app on iOS 26.6.1 or iOS 26.6.2, when my app selects and loads that image, the Exif XResolution and YResolution values are both reported as 0. Is this an iOS bug? The following code retrieves the XResolution and YResolution values from the PHAsset of the selected image. private func selectAsset(_ asset: PHAsset) { let options = PHImageRequestOptions() options.isNetworkAccessAllowed = true options.deliveryMode = .highQualityFormat PHImageManager.default().requestImageDataAndOrientation(for: asset, options: options) { data, _, _, _ in DispatchQueue.main.async { if let data { let options = PHImageRequestOptions() options.isNetworkAccessAllowed = true options.deliveryMode = .highQualityFormat PHImageManager.default().requestImageDataAndOrientation(for: asset, options: options) { data, _, _, _ in guard let data, let source = CGImageSourceCreateWithData(data as CFData, nil), let properties = CGImageSourceCopyPropertiesAtIndex(source, 0, nil) as? [CFString: Any], let tiff = properties[kCGImagePropertyTIFFDictionary] as? [CFString: Any] else { print("XResolution: NaN, YResolution: NaN") return } let xResolution = tiff[kCGImagePropertyTIFFXResolution] as? Double let yResolution = tiff[kCGImagePropertyTIFFYResolution] as? Double print("XResolution: \(xResolution.map { String($0) } ?? "NaN"), YResolution: \(yResolution.map { String($0) } ?? "NaN")") } } } } } Result on iOS 26.6 XResolution: 72, YResolution: 72 Result on iOS 26.6.1 and 26.6.2 XResolution: 0, YResolution: 0
Replies
1
Boosts
0
Views
144
Activity
2w
AVAssetDownloadConfiguration does not download supplemental tracks on iOS 16
We're downloading HLS content for offline playback using AVAssetDownloadConfiguration. It's critical for our app that all subtitle tracks are downloaded together with the video, so users can watch any subtitle and audio option while offline. On iOS 16, not all tracks are downloaded, and when the user goes offline, playback fails with NSURLErrorNotConnectedToInternet (error -1009) because AVFoundation tries to fetch the missing subtitle segments over the network. On iOS 17 and later, everything works as expected — all subtitle and audio tracks are downloaded. However, on iOS 16 we observe two issues: First issue: Supplemental tracks are not downloaded on iOS 16 (device) Only the default audio track is downloaded. All non-default supplemental tracks — both subtitles and additional audio tracks — exhibit the same problem: Some supplemental tracks listed in the manifest don't even get a directory created in the .movpkg package Some supplemental tracks do get a directory created, but it contains only StreamInfoBoot.xml and StreamInfoRoot.xml — no actual segment data (.frag files) is downloaded. The StreamInfoBoot.xml for these directories shows NO, 0, and 0, with Supplemental Additionally, a Manifest.m3u8 stream directory is created with Complete=NO and no segment data. On iOS 17+ devices, all supplemental track directories are created and all contain segment data. Second issue: Downloads fail entirely on iOS 16 Simulator We receive the following error: Error Domain=NSURLErrorDomain Code=-16090 UserInfo={ NSLocalizedDescription=The operation could not be completed, _NSURLErrorFailingURLSessionTaskErrorKey=BackgroundAVAssetDownloadTask <DEC0D555-6C05-416C-91B8-F490F5A29046>.<3>, _NSURLErrorRelatedURLSessionTaskErrorKey=( "BackgroundAVAssetDownloadTask <DEC0D555-6C05-416C-91B8-F490F5A29046>.<3>" ), NSLocalizedFailureReason=An unknown error occurred (-16090) } On iOS 17+ Simulators, downloads work without issues. Our setup: let urlAsset = AVURLAsset(url: manifestURL) let preferredMediaSelection = try await urlAsset.load(.allMediaSelections) let config = AVAssetDownloadConfiguration(asset: urlAsset, title: title) config.primaryContentConfiguration.mediaSelections = preferredMediaSelection let task = assetDownloadURLSession.makeAssetDownloadTask(downloadConfiguration: config) task.resume() Example HLS manifest: #EXTM3U #EXT-X-VERSION:7 #EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="French",LANGUAGE="fr",DEFAULT=NO,AUTOSELECT=YES,FORCED=NO,URI="s_25/stream.m3u8" #EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="French (forced)",LANGUAGE="fr-x-cc",DEFAULT=NO,AUTOSELECT=YES,FORCED=YES,URI="s_31/stream.m3u8" #EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="English",LANGUAGE="en",DEFAULT=NO,AUTOSELECT=YES,FORCED=NO,URI="s_37/stream.m3u8" #EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="Kazakh",LANGUAGE="kk",DEFAULT=NO,AUTOSELECT=YES,FORCED=NO,URI="s_43/stream.m3u8" #EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="Uzbek",LANGUAGE="uz",DEFAULT=NO,AUTOSELECT=YES,FORCED=NO,URI="s_49/stream.m3u8" #EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="aud_ec3",NAME="French 5.1",LANGUAGE="fr",DEFAULT=YES,AUTOSELECT=YES,CHANNELS="6",URI="a_18/stream.m3u8" #EXT-X-STREAM-INF:AUDIO="aud_ec3",SUBTITLES="subs",BANDWIDTH=1931408,VIDEO-RANGE=SDR,CODECS="hvc1.1.6.L90.90,ec-3",RESOLUTION=1024x428,FRAME-RATE=25,HDCP-LEVEL=NONE v_6/stream.m3u8 #EXT-X-SESSION-KEY:METHOD=SAMPLE-AES,URI="skd://XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX",KEYFORMAT="com.apple.streamingkeydelivery",KEYFORMATVERSIONS="1" Questions: What is the correct way to make sure all supplemental tracks (subtitles, non-default audio) are actually downloaded on an iOS 16 real device? We pass asset.load(.allMediaSelections) to primaryContentConfiguration.mediaSelections — is there a different configuration, or should we use auxiliaryContentConfigurations, or something else entirely? Is there any way to make asset downloads work on the iOS 16 Simulator at all? We always get NSURLErrorDomain error -16090, while iOS 17+ Simulators work fine. If the supplemental tracks download bug on iOS 16 cannot be fixed or worked around from the app side, do you have any recommended approach for handling incomplete .movpkg packages at offline playback time?
Replies
1
Boosts
0
Views
394
Activity
2w
Variable aperture on the iPhone 18 Pro
Does iOS 27 provide an API which allows developers to set the iPhone 18 Pro camera's aperture? If it does, is the aperture going to be limited to just 4 discrete settings (like in the Camera app and like mentioned in the Tech Specs for the 18 Pro) or does the hypothetical iOS API allow us to set the aperture to a continuous value in the 0.0–1.0 range (similar to how a camera's focus can be set)?
Replies
2
Boosts
1
Views
657
Activity
2w
Capturing 48mp photos with .builtInTripleCamera
I am able to capture 48mp photos using .builtInWideAngleCamera, but it seems like .builtInTripleCamera is capped at 12mp? Is there a way to capture 48mp photos using .builtInTripleCamera? Because .builtInTripleCamera provides smooth transition between cameras during zooming, and I'd like to keep this behavior. New iPhone 17 Pro have all their cameras at 48mp. Is there a chance that their .builtInTripleCamera is capable of capturing 48mp? Or is this an API limitation?
Replies
4
Boosts
0
Views
975
Activity
2w
Working with kCVPixelFormatType_96VersatileBayerPacked12
Whilst AVCaptureSession is setup to capture ProRes RAW video, is it possible to get video pixel data which can read and processed, such as using CIImage(cvPixelBuffer: ) AVCaptureVideoDataOutput outputs ProRes RAW in kCVPixelFormatType_96VersatileBayerPacked12 pixel format. Is there a provided way to debayer this pixel format into something more usable?
Replies
1
Boosts
1
Views
880
Activity
2w
Supported MusicKit APIs for vinyl-style speed, pitch and reverse playback of subscription songs on iOS?
I am building a personal iPhone prototype with an on-screen record platter. Before enrolling in the Apple Developer Program or developing the Apple Music integration further, I would like to establish whether the core interaction is supported by public APIs. The intended interaction is: Tap the platter to play or pause an Apple Music subscription song. Hold the platter to stop playback at its current position. Move the platter forward or backward to control the direction and speed of audible playback continuously. Change pitch together with speed, like a physical record, rather than preserving the original pitch. Release the platter to smoothly return to normal playback. Hold a separate control to listen continuously at a faster speed. A prototype using an AVAudioEngine-based renderer works with unprotected local audio. I have NOT verified these behaviors with Apple Music subscription content. I understand that this local renderer does not establish feasibility for MusicKit, and I am not seeking to extract protected audio or bypass content protection. MusicPlayer.State.playbackRate is documented as settable, with positive values for forward playback and negative values for reverse playback: https://developer.apple.com/documentation/musickit/musicplayer/state-swift.class/playbackrate For ApplicationMusicPlayer on iOS, is this vinyl-style interaction supported for full Apple Music subscription songs, rather than previews or user-owned unprotected files? In particular: What playback-rate range is supported for subscription songs? Do negative rates produce audible reverse playback, and how should zero-speed holding be implemented? Is there a documented public API to make pitch follow playback speed, or to disable pitch preservation? Are frequent rate and direction changes supported for interactive scratching? Are there documented limitations on update frequency, buffering, seeking precision or responsiveness? If this interaction is outside public MusicKit capabilities, is there an official channel where an individual developer can ask about eligibility for an approved Apple Music DJ integration? I am asking for the supported API path and its limitations, not implementation details of another app. If only ordinary variable-speed playback or seeking is supported, a clear distinction from continuous vinyl-style scratching would be very helpful. The initial goal is personal use; possible App Store distribution is undecided. I have an active Apple Music subscription but am not currently enrolled in the paid Apple Developer Program. Thank you.
Replies
0
Boosts
0
Views
321
Activity
3w
iOS27, CIRAWFilter RAW9 broken/missing in latest dev beta
In the WWDC talk: https://developer.apple.com/videos/play/wwdc2026/305/?time=350 "Enhance RAW image processing with Core Image" there is a section on the new RAW9 processing APIs and how to enable them. First off, the sample code compiles on iOS18 but causes a runtime exception because the .version9 symbol cannot be found, looks like the header definition is missing a version check for this value, so I'm not sure how to use this code if you need to support iOS18. Also on later dev betas (I tried the latest beta7) now trying to check if version9 is available the value is no longer present in the supported list: guard filter.supportedDecoderVersions.contains(.version9) else { ... There is another value .version9DNG that resolves to true but if I try to use that decoder the colors on the RAW image are wrong and the image becomes corrupted. How to use the RAW9 decoder in the latest dev betas, what should the final code before production because the WWDC sample now seems out of date and was always wrong wrt. iOS18.
Replies
3
Boosts
0
Views
1.5k
Activity
3w
AppleTV ContinuityCamera Scan QR Code issue
I found two issues with scan code to connect continuity cameras in tvOS QR code scanning connection timeout Scan the QR code to connect successfully, but the Camera has no data and the Capture Session does not report an exception. And these are not 100% reproduce.
Replies
0
Boosts
0
Views
500
Activity
3w
Building a library playlist on macOS that mixes catalog songs and the user's own uploads in a fixed order – supported route, and how to know when an upload is registered?
Our sandboxed macOS app digitises audio cassettes and builds one playlist per cassette in the user's Apple Music library that follows the tape's order: catalog songs where a title was recognised, and the user's own recordings (AAC files the app exported from the tape) where it was not. The user has an Apple Music subscription and Sync Library on. On macOS every write on MusicLibrary is marked @available(macOS, unavailable) in the 26.5 SDK – add, add(_:to:), createPlaylist and edit (see thread 844114, which got no answer). So we build the playlist through Music.app's scripting interface: make new user playlist, duplicate <library track> to <playlist> for catalog songs, add <file> to <playlist> for own recordings. That works, with one exception that leads to our questions. What we observe Reproducible on macOS 26.6.2 / Music 26; a standalone AppleScript is at the end. A playlist built in one go from 14 subscription tracks keeps all 14. The same playlist with one local file added after the second track: all 15 entries are there right after the build and one second later. About 15 s later the playlist has 3 entries – the two catalog entries before the file, the file, and nothing that was added after it. No error anywhere. If the file has been in the library for several minutes before the playlist is built, everything stays. Adding it to the library and waiting 45 s is not enough. Meanwhile the track's cloud status stays unknown, and GET /v1/me/library/search?types=library-songs does not list it (checked for 10 minutes). Emptying the playlist and building it again ~30 s after the entries were removed keeps everything, every time. That is what we do today; it costs 30–40 s per import, and we have to tell the user that entries were removed and put back. Our reading: while the freshly added file is not yet registered in iCloud Music Library, the playlist as pushed to the server is cut at the first item the server cannot reference, and the next sync adopts the shorter list for cloud items while keeping the local-only item in place. Questions Is there a supported way for a macOS app to create a library playlist and add tracks to it? Specifically: is the Apple Music API (POST /v1/me/library/playlists with relationships.tracks, and POST /v1/me/library/playlists/{id}/tracks) the intended route from a macOS app holding a MusicKit user token, and can such a playlist reference the user's own uploaded songs by their library id (i.…)? After a local file has been added to the library (via Music.app's add, or any other supported way), how can an app learn that iCloud Music Library has registered it, and what its library song id is? A notification, a MusicKit property, an Apple Music API endpoint? We would wait on that signal instead of rebuilding. Is the removal described above expected behaviour? Catalog ids are re-resolved at import time via id, ISRC and search as recommended in thread 122110, so stale catalog ids are not the cause. Reproduction Needs Sync Library on, at least 14 subscription tracks in the library, and a local audio file the library does not know yet. Prints the counts after 1 s, 21 s and 41 s, then cleans up. on run argv set localFile to POSIX file (item 1 of argv) tell application "Music" set cloudTracks to (every track of library playlist 1 whose cloud status is subscription) set idsBefore to persistent ID of every track of library playlist 1 set pl to make new user playlist with properties {name:"Reconciliation repro"} repeat with i from 1 to 2 duplicate (item i of cloudTracks) to pl end repeat set fileTrack to add localFile to pl set fileID to persistent ID of fileTrack repeat with i from 3 to 14 duplicate (item i of cloudTracks) to pl end repeat set n0 to count of tracks of pl delay 1 set n1 to count of tracks of pl delay 20 set n2 to count of tracks of pl delay 20 set n3 to count of tracks of pl delete pl if fileID is not in idsBefore then delete (first track of library playlist 1 whose persistent ID is fileID) return "after build: " & n0 & ", after 1 s: " & n1 & ", after 21 s: " & n2 & ", after 41 s: " & n3 end tell end run Output here: after build: 15, after 1 s: 15, after 21 s: 3, after 41 s: 3.
Replies
0
Boosts
0
Views
327
Activity
3w
RAW 9: Color Differences between 9 and 8
Are color differences expected for any cameras? I have heard some reports, but have not seen it myself. If it's not expected, then I can get those people to file bugs.
Replies
5
Boosts
1
Views
874
Activity
3w
visionOS Developer Strap: UVC video works, but how can I access the capture device’s USB Audio Class (UAC) audio?
I’m developing a native visionOS application that uses an external HDMI capture device connected to Apple Vision Pro through the Developer Strap. The goal is to use Apple Vision Pro as a low-latency display for external HDMI sources such as an Xbox Series X, including both the HDMI video and the original digital game audio. My current hardware path is: Xbox Series X → HDMI → Elgato 4K X → USB-C → Apple Vision Pro Developer Strap The Elgato 4K X receives both video and audio from the Xbox over HDMI. What is working UVC video capture is working successfully on visionOS. The external Elgato capture device can be used as the video source and the Xbox video is displayed correctly inside the Vision Pro application. I have also verified the Elgato 4K X independently on macOS, where the device is recognized correctly. The problem I cannot find a supported way on visionOS to access the USB Audio Class (UAC) audio interface associated with the capture device. I specifically need the digital game audio originating from: Xbox HDMI → Elgato 4K X → USB/UAC → visionOS I do not want to use the Apple Vision Pro’s built-in microphone. Selecting/capturing the Vision Pro microphone would only record environmental sound and is not a substitute for the digital HDMI audio stream. Apple’s visionOS documentation for external devices through the Developer Strap appears to focus on UVC video capture. I haven’t been able to determine whether the accompanying UAC interface of a UVC/UAC composite capture device is exposed to visionOS applications. What I’ve tried / verified Connected an Xbox Series X to the HDMI input of an Elgato 4K X. Connected the 4K X to Apple Vision Pro through the Developer Strap. Successfully discovered and received the external UVC video stream. Successfully displayed the Xbox video feed in visionOS. Confirmed that the 4K X itself is recognized and functional on macOS. Attempted to obtain audio corresponding to the external capture source. The Vision Pro microphone is available as an audio source, but this is not the audio source required. I have not found a documented visionOS API that exposes the 4K X’s USB/UAC audio endpoint through the Developer Strap. Questions Does visionOS currently support USB Audio Class input devices connected through the Apple Vision Pro Developer Strap? If a device exposes both UVC video and UAC audio interfaces, is there a supported way to associate/access both interfaces from a visionOS application? If UAC is supported, which framework/API should be used to discover the external audio endpoint — AVFoundation, Core Audio, or another API? Does accessing the UAC interface require an entitlement in addition to UVC device access? If generic UAC access through the Developer Strap is currently unavailable to third-party visionOS applications, is there a restricted/managed entitlement that developers can request? If this isn’t currently supported, what is Apple’s recommended architecture for receiving synchronized video and audio from an HDMI capture device on Apple Vision Pro? Ultimately, I’m trying to achieve: HDMI source → UVC video + UAC audio → visionOS application → video displayed in Vision Pro + audio played through Vision Pro speakers/AirPods Since UVC video from this exact hardware is already working, I’m primarily trying to determine whether access to the accompanying USB audio interface is possible with the current visionOS SDK, requires another entitlement/API, or is currently unsupported. Any guidance from Apple or developers who have worked with composite UVC/UAC devices on Vision Pro would be greatly appreciated.
Replies
0
Boosts
1
Views
622
Activity
3w
Request path for com.apple.coremedia.allow-mpeg4streaming on tvOS
We maintain an existing iOS/tvOS App Store application that uses AVFoundation for authenticated streaming playback. One upstream provider supplies some account-entitled playback sessions as fragmented MP4 using ISO Common Encryption scheme cenc (AES-CTR). The provider controls the rendition, and no corresponding cbcs rendition is available to our client. We understand that Apple’s documented FairPlay Streaming path supports cbcs. We identified com.apple.coremedia.allow-mpeg4streaming as relevant to this playback case, but the entitlement is undocumented and does not appear under Capability Requests for our App ID. Apple Developer Support was unable to identify the request process and recommended posting this question in the forums and filing Feedback Assistant report FB24649355. Could an Apple engineer clarify: Is this entitlement available to third-party iOS/tvOS applications? If so, what is the official request process? We can provide additional technical information privately if an Apple engineer needs it.
Replies
1
Boosts
0
Views
1.1k
Activity
3w
Volumes issue
My volume will randomly go all the way down to mute or all the way up to full blast while I’m listen to music, watching a video, playing a game. I have all the volume safety features turned off. Does anyone else have this issue?
Replies
1
Boosts
0
Views
424
Activity
3w
iOS 27 beta: setExposureModeCustom silently ignored on the front camera
On every iOS 27.0 beta, setExposureModeCustom has no effect on the front camera (.builtInTrueDepthCamera). exposureMode does change to .custom, but the sensor keeps auto-exposing: device.iso and device.exposureDuration report auto-exposure's values, not the requested ones, and frame brightness does not change. No error is thrown. The same code works on iOS 18.x and 26.x on the same devices. Device: iPhone 15 Pro, every iOS 27.0 beta. Also seen on iPhone 14 Pro Max, 16, 16 Pro Max and 17 Pro Max. Session: front camera, AVCaptureVideoDataOutput (BGRA) + AVCaptureDepthDataOutput. Format advertises ISO 20...1920, duration 0.00004...0.06667 s. isExposureModeSupported(.custom) is true. Repro: let auto-exposure settle, then try device.lockForConfiguration() device.setExposureModeCustom(duration: CMTime(value: 1, timescale: 15), iso: 1920, completionHandler: nil) device.unlockForConfiguration() then poll device.iso / device.exposureDuration (the completion handler never fires on this device) and measure frame luma. Readback, polled 0.8 s after the call: requested ISO 1920, 66.7 ms -> reported ISO 349, 60.0 ms requested ISO 20, 0.04 ms -> reported ISO 188, 66.5 ms Requesting the shortest exposure returns the longest, and the same ISO 1920 request returned 249, 349 and 450 on three consecutive runs, so these are auto-exposure's values. Frame luma stayed within 1% between the two phases. setExposureTargetBias, however, is honored on the same device: at +4 EV auto-exposure drives the sensor to ISO 1920 / 66.5 ms and at -4 EV to ISO 24 / 30.0 ms, moving mean luma from 49 to 232. So the sensor does reach the values setExposureModeCustom cannot set. Question: is this intentional or a regression, and how should an app detect that a custom exposure request was refused?
Replies
2
Boosts
0
Views
655
Activity
3w
Does the TN3135 audio-session networking exception have a defined lifetime? Seeing a ~38.5 s revoke/re-grant cycle
TN3135 describes the exception that lets a watchOS app use low-level networking while it holds an active audio session. I have that working, and the app functions — but the network path is withdrawn and restored on a strikingly regular cycle, and I would like to know whether that is expected behaviour rather than something I am doing wrong. Setup Apple Watch Series 10 (Watch7,9), watchOS 26.5. Reproduced on a Series 6 (Watch6,2). UIBackgroundModes: [audio]; AVAudioSession category .playAndRecord, mode .spokenAudio; activated with the async activate(options:completionHandler:). NWConnection with NWProtocolWebSocket to a WebSocket relay over TLS. The app streams 16 kHz mono PCM continuously while transmitting and holds the socket open otherwise. Symptom NWPathMonitor reports .unsatisfied, then .satisfied about two seconds later, over and over. Measured with the iPhone powered off, so the watch was on its own Wi-Fi: Uptime between drops Outage 36.4 s 2.1 s 36.7 s 1.9 s 36.9 s 2.1 s The regularity is what prompts the question — uptime varies by ±0.3 s and the outage is consistently 2.0 s. That reads as a timeout expiring rather than radio behaviour. What I have ruled out Not the network or the server. A browser client on the same relay, same TLS, same wire protocol, holds a WebSocket indefinitely. Not the interface. Identical cadence over the companion ipsec1 tunnel with the iPhone present, and over the watch's own en0 with the iPhone powered off. Pinning requiredInterfaceType = .wifi while the iPhone is reachable fails outright — the path offers only ipsec1. Not audio-session interruption. I observe interruptionNotification, routeChangeNotification, mediaServicesWereResetNotification and silenceSecondaryAudioHintNotification. None fire at a drop. At the moment the path goes .unsatisfied, the engine is running and the player node is actively playing. Not session idleness. Playing continuous silence for the whole session, rather than only while reconnecting, made no difference — still 36.4 s. The control that surprised me To test whether this affects any long-lived watch socket or only audio-unlocked ones, I built a second app with no AVAudioSession at all, no audio background mode, holding a URLSessionWebSocketTask and kept alive by a WKExtendedRuntimeSession so screen sleep was not a factor. It never connected. NWPathMonitor reported .unsatisfied once and never changed, across a 30 s run, and every request failed with "The Internet connection appears to be offline." I had expected URLSession to be permitted regardless. Questions Does the audio-session networking exception in TN3135 have a defined lifetime, and is a periodic revoke/re-grant cycle expected? If so, is there a supported way to hold it continuously — or is the correct design simply to expect the interruption and reconnect through it? Is it expected that an app with no audio session gets no network path at all on watchOS, including via URLSession, even in the foreground with an extended runtime session?
Replies
16
Boosts
0
Views
3.6k
Activity
3w
AirPods Sleep Detection pauses meditation playback with no API/callback to identify the reason
Good morning! I work on a meditation app, and since Sleep Detection was introduced for AirPods in iOS 26, we've been receiving complaints from users reporting that their meditation sessions unexpectedly stop after approximately 15–20 minutes when Sleep Detection is enabled. This seems to happen particularly during deep meditation sessions, where the user may remain still and exhibit behavior that could potentially be interpreted as falling asleep. I've also tested this on iOS 27 with the latest beta firmware available for AirPods Pro, and the same behavior still occurs. While investigating and debugging the issue, I found that there doesn't appear to be any public API that allows an app to determine that playback was paused specifically because of AirPods Sleep Detection. From the application's perspective, the AirPods appear to simply send a standard pause command to the AVPlayer, without providing any additional context or reason. There also doesn't seem to be an API that allows us to determine whether the user has Sleep Detection enabled. Even having access to that information would allow us to warn users that their meditation session could potentially be interrupted. AVAudioSession.InterruptionReason does not seem to help in this case either: https://developer.apple.com/documentation/avfaudio/avaudiosession/interruptionreason Is there currently any supported way for an app to: Detect that playback was paused specifically because of AirPods Sleep Detection? Determine whether Sleep Detection is enabled for the connected AirPods? Prevent Sleep Detection from pausing playback for specific apps or specific types of audio sessions? If none of these are currently possible, could Apple consider providing either a notification/callback indicating that Sleep Detection triggered the pause, or an option for users to exclude specific apps from Sleep Detection? This behavior is particularly problematic for meditation apps, since a user in a deep meditation session can easily be mistaken for someone who has fallen asleep. Other meditation apps appear to be affected by the same behavior as well. Best regards, Carlos Antunes
Replies
1
Boosts
0
Views
1.1k
Activity
3w
MusicKit on HarmonyOS — playback approach for a platform without an official SDK, licensing question
Hello Apple Developer community, I'm building an Apple Music client for HarmonyOS (Huawei's operating system) and would like to clarify the licensing and technical approach before submitting to a third-party app store (Huawei AppGallery). I've read the Apple Developer Program License Agreement and App Store Review Guideline 4.5.2, but there are platform-specific constraints I'd like to discuss. Background Apple currently does not provide a MusicKit SDK for HarmonyOS. The available options are: MusicKit for iOS/macOS — not applicable (different OS) MusicKit for Android — ships as proprietary .aar binaries that cannot be redistributed and are not compatible with HarmonyOS's runtime MusicKit JS — available, but HarmonyOS WebView does not support FairPlay or Widevine EME, so full-song DRM playback through MusicKit JS does not work on this platform The only Apple Music API access path available to a HarmonyOS app is the REST API (api.music.apple.com) with a Developer Token (JWT, ES256) and a Music User Token obtained through the MusicKit JS authorization flow running inside a WebView. The technical constraint To deliver full-song playback for subscribed Apple Music users on HarmonyOS, our app retrieves the HLS playback URL via the Apple Music API, downloads the encrypted media segments, and decrypts them using HarmonyOS's native DRM system (WisePlay, the only CDM available on the platform). MusicKit JS's FairPlay/Widevine CDM is not compatible with HarmonyOS — WisePlay is the sole DRM implementation the OS exposes. We are aware that Section 4.5.2 states: "You may not, and You may not permit Your end users to, download, upload, or modify any MusicKit Content" and "You may play MusicKit Content only as rendered by the MusicKit APIs or MusicKit JS" What we want to clarify Our playback implementation works as follows, and we want to confirm whether it is acceptable under the license terms: No download service is offered to users. The download is purely an internal playback mechanism — the app fetches encrypted segments, decrypts them in memory / a protected sandbox, and plays them via the system AVPlayer. There is no "download button," no offline library, no user-facing download feature. Decrypted files are ephemeral and protected. Decrypted media is written to the app's private sandbox (not accessible to users or other apps), played immediately, and deleted as soon as playback ends or the song is skipped. The decrypted file never persists beyond the current playback session. No file leakage. The decrypted content is never exposed via any sharing mechanism, file picker, or external storage. It exists only within the app's protected data directory for the duration of playback. Standard playback controls are preserved. Users initiate playback and can use play / pause / skip / seek — the same controls required by 4.5.2(i). No monetization of Apple Music access. The app does not charge for Apple Music playback, does not show ads around it, and does not collect or share user data. Our question Given that: Apple has no SDK presence on HarmonyOS, MusicKit JS DRM is technically non-functional on HarmonyOS WebView, the only way to deliver full-song playback to Apple Music subscribers on this platform is to decrypt the stream using the OS's native DRM (WisePlay) with strict ephemeral handling, is this playback approach acceptable under the Apple Developer Program License Agreement, provided that: decrypted content is never persisted, shared, or exposed to users, the app only serves subscribed Apple Music users with valid Music User Tokens, all usage remains within the scope of facilitating access to the user's own Apple Music subscription? We fully respect Apple's content protection requirements and are happy to adjust the implementation (e.g., stricter deletion timing, additional sandboxing, attestation) to comply. We just want to understand whether the fundamental approach — decrypting via the host OS's DRM because Apple's own CDMs are unavailable on this platform — is permissible, or whether Apple considers HarmonyOS out of scope for full-song playback entirely. Any guidance from the MusicKit team or community members who have dealt with non-Apple-platform constraints would be greatly appreciated. Thank you.
Replies
0
Boosts
0
Views
360
Activity
3w
Is there a supported way to read on-screen text with the user's consent? (worker-safety / income-transparency app)
Hello, I am an independent developer in Argentina building an app for rideshare drivers, and I would like to know what the supported approach is on iOS before I build the wrong thing. The problem When a trip offer appears, the driver sees a fare, a distance and a duration. What they do not see is what is left after the cost of running their own car — fuel, maintenance, and very often the weekly rent they pay for the vehicle. Here those costs are high and change constantly, and drivers routinely accept trips that lose them money without realising it. They have about three seconds to decide, while driving. Our app reads the numbers already visible on the driver's own screen, subtracts that particular driver's cost per kilometre, and shows one figure: what this trip actually leaves them per hour. In our own field measurements with two drivers over several weeks, being able to see that number raised their hourly earnings and — just as important — kept their eyes off mental arithmetic at the wheel. What we will not do, on purpose I want to be explicit about this, because I know other tools in this category cut corners: We never ask for the driver's Uber or DiDi username or password, and we never connect to or automate their account. Some tools do, and that puts the driver's account — their livelihood — at risk of deactivation. We will not build that. We never accept or decline a trip for the driver. The decision and the tap are always theirs. We never store passenger data: no name, no rating, no photo, no exact address. Text recognition runs entirely on device, offline. The captured image is discarded immediately and never leaves the phone or reaches any server. No advertising, no data sales, no third-party trackers. The app does not read anything the driver is not already looking at, it only reads while the driver has explicitly turned it on for their shift, and it exists for one purpose: so that someone working long hours knows what an hour of their work is actually worth. My question On Android this is done with MediaProjection, with explicit user consent each session and a persistent system indicator. On iOS I understand a Broadcast Upload Extension is not intended for background processing, and that there is no overlay across apps. Is there any supported path on iOS for an app to read, with the user's explicit and repeated consent, text the user is already looking at on their own screen, in order to give that same user an immediate assessment of their own work? Accessibility APIs, a system extension point, anything. If there is no supported path, I would rather know now and design the iOS version around what is allowed, instead of shipping something that gets rejected or that quietly breaks the rules. Thank you.
Replies
1
Boosts
0
Views
382
Activity
3w