Which of the Vision framework‘s support for grounding/bounding boxes/segmentation carry over to the iOS 27 on-device Foundation Model image support?

I am trying to compress a CIImage below a certain file size threshold when creating a JPEG. Currently, I am handling this in, potentially, multiple passes. In the first pass, I am essentially calling:
encodeJPEG(CIImage, quality: 0.85, colorSpace: CIImage.colorSpace ?? CGColorSpace(name: CGColorSpace.sRGB)!)
I am then checking the image size. If the image is still too big, I try to estimate how much I need to reduce it and try again until the image size is small enough. My question: Is there a more efficient way of doing this?

The iPhone Lock Screen lets you display a small time and date at the top of the display that has a neat image processing effect where the text takes on the underlying colors with a blurred gradient while still maintaining legibility appearing largely white. In this screenshot you can see the white text features blue colors at the top left and transitions to a purple color on the right due to the underlying vibrant blue and purple blocks. Can you share how this text effect is achieved using Core Image?

I’m really excited about RAW 9, great work! Since RAW 9 is not enabled by default, are there any recommended use cases where it makes sense to select it by default, if supported? For compatibility handling, should I treat decoderVersion as covering all processing and adjustments performed by CIRAWFilter, or does it mostly affect the RAW decoding stage? When switching decoder versions, should I re-evaluate which CIRAWFilter adjustments are supported? Also, the macOS 26 headers appear to expose version9 and version9DNG. Is RAW 9 actually supported on macOS 26 runtimes? Thanks!

CIRAWFilter documentation lists despeckleAmount as available on macOS 12.0+, but I cannot find this property in the macOS 26 headers. Could you provide a detailed description of how despeckleAmount affects the image so I can explain this adjustment accurately to my users? Does it run before or after linearSpaceFilter, and what should it be set to when generating scene-referred linear output? Thanks!

What is the high water mark for rendering an image in RAW 9 vs. RAW 8? I realize it depends on the megapixels of the render, but what can you share memory wise, especially compared to RAW 8, perhaps as a multiple? I'm mostly concerned about iOS in case that's relevant.

What is the minimum iPhone system requirement for RAW 9? Anything that can run iOS 27 / iPadOS 27 / macOS 27? I was getting some ANE errors on an M1 MacBook Air this morning (FB23103074)

In our PhotoKit backup/export code, we need to read PHAssetResource.originalFilename for each resource. The Swift API imports this property as non-optional: var originalFilename: String { get } However, we have customer reports from real Photos libraries where reading this property appears to crash because the underlying Objective-C value is nil at runtime. This affects actual users of our shipping app, not just a synthetic test case. The relevant code path is essentially this: import Photos func inspectAsset(_ asset: PHAsset) { let resources = PHAssetResource.assetResources(for: asset) for resource in resources { // This property is imported as non-optional String. // For some affected customer assets, reading it appears to trap/crash // because the underlying value is nil. let filename: String = resource.originalFilename print(filename) } } This makes the API difficult to use safely from Swift: because the property is non-optional, there is no normal Swift optional-handling path before the crash. We filed this as FB22589474. We also filed a related Feedback, FB22519412, for empty-string / otherwise unusable filename values. That is a separate issue, but it points in the same direction: apps that export, back up, or sync Photos resources cannot treat PHAssetResource.originalFilename as a guaranteed usable filename. For PHAssetResource.originalFilename, what is the intended contract?

Reconstruction with Object Mask in PhotogrammetrySession Fails and doesn't complete because of alignment and object masks with thing segmentation even after increasing the mask region . Also the PhotogrammetrySample hangs with EXC_BREAKPOINTwhen I usePhotogrammetrySession(input: [PhotogrammetrySample])withobjectMask set and the Apple Developer Forums report crashes (“HelloPhotogrammetry” / sample+mask), and (EXC_BREAKPOINT` was being observed at ~4 GB) with freezing the execution and Xcode interface

Feedback: PHBackgroundResourceUploadExtension is never scheduled when iCloud Photos is enabled Summary PHBackgroundResourceUploadExtension's init() and process() methods are never called by the system when iCloud Photos is enabled on the device, even though setUploadJobExtensionEnabled(true) succeeds and uploadJobExtensionEnabled returns true. Environment iOS 26.4 (both devices) Xcode 26.x Tested on iPhone 17 Pro (primary device, 10,000+ photos) and an older iPhone (development device, 200+ photos) Same build deployed to both devices Full photo library access (.authorized) on both devices Steps to Reproduce Create an app with a PHBackgroundResourceUploadExtension (ExtensionKit, extension point com.apple.photos.background-upload) Enable iCloud Photos on the device (Settings > Photos > iCloud Photos) In the host app, request .readWrite photo library authorization and receive .authorized Call PHPhotoLibrary.shared().setUploadJobExtensionEnabled(true) — succeeds without error Verify PHPhotoLibrary.shared().uploadJobExtensionEnabled == true Wait for the system to schedule the extension (tested overnight, with device on charger + WiFi) Expected Behavior The system should call the extension's init() and process() methods to allow the app to create upload jobs, as documented in "Uploading asset resources in the background." Actual Behavior The extension's init() and process() are never called. The extension process is never launched. Investigation via Console.app Using Console.app to monitor system logs on both devices revealed the root cause: When iCloud Photos is DISABLED (extension works correctly): assetsd Checked all submitted library bundles. Result: YES dasd SUBMITTING: com.apple.assetsd.backgroundjobservice.assetresourceuploadextensionrunner:35E7B5 dasd Submitted: ...assetresourceuploadextensionrunner at priority 5 dasd submitTaskRequestWithIdentifier: Submitted BGSystemTask com.apple.assetsd.backgroundjobservice.assetresourceuploadextensionrunner The extension is subsequently launched and init() / process() are called as expected. When iCloud Photos is ENABLED (extension never works): assetsd Checked all submitted library bundles. Result: NO No assetresourceuploadextensionrunner background task is ever submitted to dasd. The extension is never launched. Reproducibility 100% reproducible across two different devices Toggling iCloud Photos off (and waiting for sync to complete) immediately resolves the issue Toggling iCloud Photos on immediately causes the issue to reappear Reinstalling the app does not help The same build works on the same device when iCloud Photos is disabled Impact This effectively makes PHBackgroundResourceUploadExtension unusable for the vast majority of users, as most iPhone users have iCloud Photos enabled. Third-party photo backup apps (Google Photos, Dropbox, OneDrive, etc.) would all be affected by this limitation. The documentation for "Uploading asset resources in the background" does not mention any incompatibility with iCloud Photos being enabled. Requested Resolution Please either: Allow PHBackgroundResourceUploadExtension to be scheduled regardless of iCloud Photos status, or Document this limitation clearly in the API documentation if it is intentional behavior

I'm developing a PHBackgroundResourceUploadExtension and finding it difficult to debug because the system controls when the extension launches. Current experience: The extension starts at unpredictable times (anywhere from 1 minute to several hours after photos are added) By the time I attach the debugger, the upload may have already completed or failed Breakpoints in init() or early lifecycle methods are often missed Questions: Is there a way to force-launch the extension during development (similar to how we can manually trigger Background App Refresh in Xcode)? Are there any launch arguments or environment variables that put the extension in a debug/eager mode? I tried taking photos/videos, but this doesn't trigged app extension in all cases. Any tips for improving the debug cycle would be greatly appreciated. Environment: iOS 26, Xcode 18

I am requesting technical clarification and a formal feature addition regarding the availability of per-frame calibration data for Standard RAW captures on iPhone 17 and 17 Pro. The Technical Gap: Currently, AVCaptureVideoDataOutput provides a direct path to the AVCameraCalibrationData object, allowing real-time access to the Intrinsic Matrix, Radial/Tangential Distortion coefficients, and Lens Shading Maps. However, this same level of geometric transparency is missing from the AVCapturePhoto RAW/ProRAW delegate. While standard RAW files contain some metadata, they lack the physics sidecar required for precision manual alignment. Because the lens assembly in the iPhone 17 Pro is dynamic shifted constantly by Optical Image Stabilization (OIS) and high-speed Voice Coil Motors (VCM) for focus a static or factor calibration is mathematically insufficient for high-precision workflows. The Problem: Without the 1:1 hardware state at the millisecond of exposure, we cannot perform accurate geometric reconstruction from RAW stills. We are forced to choose between the high dynamic range of a RAW sensor dump and the geometric precision of the video pipeline. Final Questions: Is there a documented, supported method to force the inclusion of the AVCameraCalibrationData object or a raw metadata sidecar in the AVCapturePhoto workflow? If not, can Apple provide parity between the Video and Photo APIs so that the "rawest" data (RAW) is accompanied by the "rawest" physics (Calibration Data)? Providing the pixels without the lens geometry limits the utility of the RAW format for any technical workflow requiring sub-pixel geometric integrity.

Hi AVFoundation team, I'm seeing a uniform 90° offset in AVCaptureDevice.RotationCoordinator's reported angles between iPhone 17 Pro and iPhone 14 Pro using the front-facing .builtInWideAngleCamera (Center Stage on 17 Pro), and I'd like to confirm whether this is by design and what the recommended consumption pattern is when the rendering surface is an AVSampleBufferDisplayLayer rather than an AVCaptureVideoPreviewLayer. Here is the github repo of sample project. Setup Devices: iPhone 14 Pro (iOS 26.5) and iPhone 17 Pro (iOS 26.4.2) Camera: front, AVCaptureDeviceTypeBuiltInWideAngleCamera Active format: 1920×1080 Three RotationCoordinator instances are created on the same AVCaptureDevice, varying only the previewLayer: argument: - previewLayer: nil - previewLayer: AVSampleBufferDisplayLayer (the surface receiving frames from AVCaptureVideoDataOutput) - previewLayer: AVCaptureVideoPreviewLayer (with .session = captureSession, not displayed) Each instance is KVO-observed for videoRotationAngleForHorizonLevelPreview and videoRotationAngleForHorizonLevelCapture. Observed angles Device / Orientation: 14 Pro · Portrait (interface=1) RC[nil] prev / cap: 0° / 90° RC[AVSampleBufferLayer] prev / cap: 90° / 90° RC[AVCaptureVideoPreviewLayer] prev / cap: 0° / 90° ──────────────────────────────────────── Device / Orientation: 14 Pro · LandscapeRight (interface=3) RC[nil] prev / cap: 0° / 180° RC[AVSampleBufferLayer] prev / cap: 180° / 180° RC[AVCaptureVideoPreviewLayer] prev / cap: 0° / 180° ──────────────────────────────────────── Device / Orientation: 17 Pro · Portrait (interface=1) RC[nil] prev / cap: 0° / 0° RC[AVSampleBufferLayer] prev / cap: 0° / 0° RC[AVCaptureVideoPreviewLayer] prev / cap: 0° / 0° ──────────────────────────────────────── Device / Orientation: 17 Pro · LandscapeRight (interface=3) RC[nil] prev / cap: 0° / 90° RC[AVSampleBufferLayer] prev / cap: 90° / 90° RC[AVCaptureVideoPreviewLayer] prev / cap: 0° / 90° The −90° offset on 17 Pro is uniform: it appears in every RC variant, in both the preview-angle and the capture-angle properties, at every orientation tested. It is not specific to the previewLayer: argument.

Thanks for your great questions. We'll be wrapping up our live feedback session in 15 minutes. Please get your questions in soon.

What is the intended purpose of the new PHPhotoLibraryPersistentChangesObserver? I currently use PHPhotoLibraryChangeObserver as a trigger, then call fetchPersistentChanges(since:). What behavior or guarantees differ when using PHPhotoLibraryPersistentChangesObserver instead? Thanks!

For a PHAsset in the same Photos library on the same device/Mac, what are the documented stability guarantees of PHObject.localIdentifier? Is it safe to persist and use for future PhotoKit operations in that same local library? Are there known cases where it can change or stop resolving? If a persisted localIdentifier no longer resolves but a persisted PHCloudIdentifier.archivalStringValue does resolve in the same library, is updating the stored local identifier from that cloud mapping the recommended recovery path? Thanks!

My app displays photos from user's photo library. I want to provide a way to deep link to that photo in the Photos app (open the photo in the Photos app). Is there any way to do this?

Is there a supported way for developers to purge locally downloaded Photos originals/resources in order to force an asset/resource into a non-downloaded iCloud-only state? Basically simulate "Optimize Mac Storage" behavior on demand for testing. Thanks!

Is there a way to determine whether a specific PHAssetResource original is locally available/downloaded without requesting it? If not, is the recommended approach to request the resource with PHAssetResourceRequestOptions.isNetworkAccessAllowed = false and handle networkAccessRequired error? Is that error the definitive signal that the requested resource is not locally available, or are there other errors that should be treated the same way? Thanks!

Can Photos.framework distinguish between assets that are stored in the user’s Personal Library and assets that are stored in the Shared Library? If not, is there a planned API that would allow developers to identify the library membership of a PHAsset?