I'm building a MR teleoperation app for the Apple Vision Pro. Instead of 2D camera feeds, I render the robot's remote workspace as 3D Gaussian splats, so the operator can perceive depth and inspect objects from any angle by moving their head. That means the operator needs to lean in close to an object, and sometimes move the viewpoint right up to or through it, without the object dropping out of view. Right now when I use the visionOS 27 splat sample out of the box, the splats disappear when the operator gets too close. As the viewpoint approaches a splatted object the entire splat will vanish. This can be solved partially with chunking the splats, but the operator still can't move inside the splat itself. The entire splat vanishing, and chunking helping, makes me think the splat entity is culled once the camera enters its bounds, rather than individual Gaussians being clipped at the near plane. I don't know the renderer's internals, so I may be wrong about the cause. I believe the sample renders in an ImmersiveSpace, not a bounded volume. Does GaussianSplatComponent expose any control over how splats are culled up close (near plane, clipping, or entity bounds), so the splat stays visible as the viewer moves close to or inside it? If there's no such control today, is there a recommended way to let the viewer move through a splat without it disappearing?

We're building animated and streamed Gaussian splat playback into Scenery, our native Apple AR/spatial authoring platform, and have a few questions about the intended usage of GaussianSplatComponent / GaussianSplatResource. Dynamic / animated splats (4D) Is mutating the underlying LowLevelBuffer behind a BufferResource at runtime a supported pattern, or is the component intended for static captures? BufferResource's init is @MainActor and the resource's bufferResource property is let, which suggests static intent. For animated (4D) splats, do we need to wrap a new BufferResource each frame and reassign it, or will in-place writes to the existing buffer be picked up by the renderer? If re-wrapping is required, is there a recommended cadence, and does it trigger a full re-sort/re-upload internally? LOD streaming / swapping Related: for streaming splat LOD, is swapping the BufferResource (or its buffers) at runtime supported, and is there a cost model we should design around — e.g. does a swap re-sort and re-upload everything, or can partial updates be done? Splat-count limit The BufferResource initializer throws past an internal limit (we're seeing ~200k). Is this limit fixed, memory-derived, or per-device? Is it per-component or per-scene — i.e. can we tile a large capture across multiple GaussianSplatComponents to exceed it? Any guidance on the intended ceiling and whether it's expected to change? Large captures as immersive environments We'd like to use a large splat capture as an environment the user walks inside (camera within the splat volume), rather than an object viewed from outside. Is this an intended use case? We're seeing splats appear to fade/break up as the camera gets very close — is that near-plane behavior, an opacity falloff, or the projection breaking down at close range, and can it be controlled? Does ProjectionMode.tangential help for inside-the-volume viewing? Thank you and appreciate the great updates!

I'm trying to create a hexagonal triplanar material in Reality Composer Pro 3 using Shader Graph. Could you provide guidance, or an example node setup, for creating a seamless hex-tile texture using triplanar projection? My goal is to apply a hex pattern to 3D objects without visible seams or texture stretching. Are there any sample projects, Shader Graph examples, or recommended workflows for achieving this in Reality Composer Pro?

This is a very general and high level question. I am working on an app that scans buildings using on device LiDAR. One of my biggest issues is drift over time. What is the recommended approach by Apple to successfully scan for a long period of time and reduce drift error? I am currently storing frames at 20 fps with pose data and then trying to correct for pose errors off device. Maybe my approach is incorrect and would like to understand how Apple recommends to do about this. I am being overly general on purpose. Thank you for your help.

Dear Apple Developer Team, I would like to ask a few questions related to RoomPlan, Room API, USDZ export, and possible future spatial scanning workflows: Can the RoomPlan or Room API load an existing USDZ room model and compare it in real time with a new live scan of the same space? Is Apple considering extending RoomPlan beyond indoor rooms, for example to scan outdoor areas, house exteriors, terrain, and simple building volumes with rough dimensions? Will RoomPlan support multi-floor continuous scanning, custom object detection elements, and reliable export of the scanned result to USDZ for further CAD/BIM workflows? Is it possible to create a real-time comparator between a reference object in USDZ or STEP format and a physical object being scanned live, so that deviations in geometry or dimensions can be detected during scanning? Best regards, Ivo Saina

We are building an ARKit application targeting iPhone Pro devices on iOS 26.4+. During testing, we consistently observe the following log message at a specific point in our AR session (approximately 90° into a walking arc around a vehicle): ARWorldTrackingTechnique: resource constraints [33] What specifically does resource constraint code [33] represent? Is it a VIO constraint type, a sensor fusion budget limit, or something else entirely? How does it differ from code [1] (which appears to be a general CPU starvation signal for the ARKit sensor fusion thread)? Is there a public API to detect or respond to this specific constraint condition, short of waiting for a full ARCamera.TrackingState downgrade? Was this constraint type introduced or changed in iOS 26.4?

I'm trying to create a hexagonal triplanar material in Reality Composer Pro 3 using Shader Graph. Could you provide guidance, or an example node setup, for creating a seamless hex-tile texture using triplanar projection? My goal is to apply a hex pattern to 3D objects without visible seams or texture stretching. Are there any sample projects, Shader Graph examples, or recommended workflows for achieving this in Reality Composer Pro? Thanks

For Vision Pro's tiled GPU, how does MeshInstancesComponent compare to manually merging meshes for draw-call reduction? Does it collapse draw calls similarly while still allowing per-instance transform and material variation? Deciding whether to re-architect a heavy interior scene around it instead of hand-merging in Maya.

RCP 3's Animation Graph covers state machines and blending of imported clips. As an animator, is there now any in-editor support for authoring or editing animation curves and keyframes directly — a graph editor or dope sheet — or must all keyframe animation still be authored in a DCC (Maya/Blender) and imported as clips?

RCP 3 adds a lot of artist-facing surface: lightmaps, Animation Graph, Script Graph. Are step-by-step, artist-oriented tutorials planned beyond the API reference docs? A guided onboarding path would help 3D artists adopt these features much faster.

May I ask if there is a complete source code project for this instructional video that needs to be learned. Work with Reality Composer Pro content in Xcode

I'm trying to track identified objects in realtime with bounding rects, with no 3D integration, but still has poor update performance. I'm trying to understand how to optimize frame updates. (I'm a new programmer) Using: Foundation, AVFoundation, ARKit, CoreVideo, Vision, CoreImage, CoreVideo with YOLOE-11s object detection currently throttled to 2fps. (target iOS, testing on 16pro) YOLOE-11S CoreML model detects objects with class labels + bounding boxes Labels are matched against ObjectCatalog.json for relevance Matched objects are promoted from blue (detected) to green (identified) Log warnings: ARSession <0x110afdb80>: The delegate of ARSession is retaining 13 ARFrames. The camera will stop delivering camera images if the delegate keeps holding on to too many ARFrames. This could be a threading or memory management issue in the delegate and should be fixed. Skipping integration due to poor slam at time: 619447.208339 vio_initialized(1) map_size(0) tracking_state_is_nominal(0) is_3dof(0) reinitialize_attempts(6) slam_mode(RegularSLAM)

I’m familiar with using ModelSortGroupComponent to control draw order, but how does RealityKit determine the draw order between model sort groups? How can I enforce a specific draw order between two model sort groups that require different depth pass modes and therefore can't be combined into a single model sort group? Are there model bounding box tricks I can safely rely on?

There is no "export" for a scene or project like in RCP2. I tried to select root|world but none of the 3 file formats export components, therefore no interaction is possible. It works inside RCP3 with PLAY. I tried the exported .reality asset in macOS27 and on iPadOS27. No interactions.

Film crews record "takes" in my app: camera feed plus virtual AR content with timecode. On SceneKit I rely on a third-party Metal-layer hook (SCNRecorder), because ReplayKit captures overlay UI and adds a permission prompt — and RealityKit appears to have no capture API at all, making recording a regression in any SceneKit→RealityKit migration. Is there a recommended first-party way to record an AR view to a movie file, or should I file feedback?

On visionOS the user's viewpoint isn't directly movable, so we currently invert world movement to simulate locomotion — translating the scene root rather than the camera. With RCP 3 / RealityKit this year, is there a supported way to define or animate a starting viewpoint — a "player start" transform — for an immersive scene, or is world-relative transform still the only sanctioned approach?

I’m building a visionOS app with an ImmersiveSpace, and I want to show a menu or control panel inside that immersive space. The menu would be created as part of the app’s immersive content, for example as a SwiftUI attachment in a RealityView, or as a custom RealityKit entity with UI-like content. What I would like is for this in-space menu to behave more like a regular visionOS 2D window: The user can move the menu naturally. While the menu is being moved, it automatically adjusts its orientation to face the user. It maintains a comfortable apparent size or distance while being repositioned. It avoids awkward angles or unreadable placement. It feels similar to the system-managed behavior of regular 2D windows. My question is: is there a supported way to give an in-ImmersiveSpace menu the same placement and movement behavior as a normal 2D window? More specifically: Is there a built-in component or API that provides window-like movement, billboard-facing behavior, comfortable distance handling, or automatic scaling for custom panels inside an immersive space? If not, is the recommended approach to implement this behavior manually in RealityKit, for example by tracking the user’s head position and updating the panel’s transform? If manual implementation is required, are there recommended comfort guidelines for menu distance, scale, rotation limits, and movement behavior in immersive spaces? Alternatively, is the recommended design to use a regular 2D window or utility panel outside the immersive content, rather than trying to recreate window behavior inside the ImmersiveSpace?

My users import their own 3D assets at runtime — OBJ, FBX, DAE, STL, PLY, often with skeletal animation — which we convert in-app to SceneKit scenes via Assimp. RealityKit only loads USD/.reality at runtime. Is the recommended path to hand-build MeshResource, skeletons, and animations from parsed geometry, or is there any supported on-device conversion to USD? This is one of my biggest blockers to leaving SceneKit.

My app is an 8-year-old film-previsualization tool built on ARSCNView, with SceneKit rendering over an ARKit world-tracking session. Given SceneKit's deprecation with critical-bug-only maintenance: does that commitment, and the "ample notice" promise, explicitly cover ARSCNView? Should I plan on existing SceneKit apps continuing to work on new hardware and OS releases for several more years? I have a few blockers stopping me from migrating to RealityKit and I've noticed the lack of big updates to ARKit over the recent years.

When I iterate geometry in Maya/Blender after assigning components, prototype overrides, and lightmaps in RCP 3, does re-importing the updated USD preserve those bindings, or do they need reauthoring on each import? What's the recommended round-trip for art that's still changing?