Discuss spatial computing on Apple platforms and how to design and build an entirely new universe of apps and games for Apple Vision Pro.

All subtopics
Posts under Spatial Computing topic

Post

Replies

Boosts

Views

Activity

With manipulation component, once you let go, how to prevent the entity from disappearing while animating it back into the volume
So with the new ManipulationComponent, we can choose "stay" and then if you drag it out of your volume, once you let go it will instantly disappear. We can "animate" it back to inside the volume, eg.: content.subscribe(to: ManipulationEvents.WillRelease.self) { event in Entity.animate( .easeInOut(duration: 1), body: { event.entity.position = [0, 0.2, 0] }, completion: {} ) }, Howeve,r for the duration that it travels outside of the volume it's invisible the whole time. In this apple video, it seems to be visible when dragging and when letting go, but perhaps that's not a volume they're dragging it out of? https://youtu.be/VtenPKrvPOU?si=y1zoZOs2IMyDzOm6&t=1748 Does anyone know how to keep the entity visible even when after letting the entity go while you animate it back towards inside of your volume?
1
1
986
Jan ’26
ManipulationComponent in both parent and child entities
Hello, In my project, I have attached a ManipulationComponent to Entity A and as expected, I'm able interact with it using the built-in gestures. I have another Entity B which is a child of A that I would like to interact with as well, so I attempted to add a ManipulationComponent to B. However, no gestures seem to be registered on B; I can still interact with A but B cannot be interacted with despite having ManipulationComponents on both entities. So I'm wondering if I'm just doing something wrong, if this is an issue with the ManipulationComponent, or if this is a limitation of the API. Attached is the code used to add the ManipulationComponent to an Entity and it was done on both A and B: let mc = ManipulationComponent() model.components.set(mc) var boxShape = ShapeResource.generateBox(width: 0.25, height: 0.05, depth: 0.25) boxShape = boxShape.offsetBy(translation: simd_float3(0, -0.05, -0.25)) ManipulationComponent.configureEntity(model, collisionShapes: [boxShape]) if var mc = model.components[ManipulationComponent.self] { mc.releaseBehavior = .stay mc.dynamics.inertia = .low model.components.set(mc) } I am using visionOS 26.0; let me know if there's any additional information needed.
1
0
376
Oct ’25
Current Apple Forum about ARKit and visionOS
Recently, questions about ARKit/visionOS seem to be being asked in the Apple forum by internal Apple engineers. Inexperienced and untested makeshift features are being offered, putting average but experienced developers in a difficult position. They are unable to react and get something useful from the posts. Apple needs to review the situation.
1
0
372
Sep ’25
The folding and unfolding effect of the NBA sand table
Seeing this magical sand table, the unfolding and folding effects are similar to spreading out cards, which is very interesting. But I don't know how to achieve it. I want to see if there are any ways to achieve this effect and give some ideas. May I ask if this effect can be achieved under the existing API
1
0
99
May ’25
Displaying multiple immersive movies in spheres in an immersive environment
In visionOS, I'm trying to create an immersive environment which would feature several spheres in which immersive movies are visible. I'm starting from a sample code which creates a sphere, sets an immersive movie as its material, and opens it as an immersive environment. This works fine. But if I create a sphere in an open immersive environment using Reality Composer Pro and sets its material to an immersive movie, I can see the movie on the sphere while I move outside of it but if I try to get inside the sphere, it disappears. What would be the right way of doing this ?
1
1
722
Oct ’25
ARKit camera transform orientation vector doesn't match physical device heading (despite `.gravityAndHeading`)
Hi all, I'm working on an ARKit-based iOS app where I need to accurately determine the direction the device is facing to localize objects in the real world. I'm using: let config = ARWorldTrackingConfiguration() config.worldAlignment = .gravityAndHeading Thus, I would expect the world alignment to behave as given in the gravityAndHeading page. The AR session is started after verifying that CLLocationManager.headingAccuracy <= 20, and the compass appears to be calibrated. However, I'm seeing a major inconsistency: When the rear camera is physically pointed toward true North, I would expect: cameraTransform.columns.2.z ≈ -1 // (i.e. ARKit's -Z pointing North) But instead, I'm consistently seeing: cameraTransform.columns.2.z ≈ +0.97 // Implies camera is facing South Meanwhile, the translation vector behaves as expected: As I physically move North, cameraTransform.columns.3.z becomes more negative, matching the world’s +Z = South assumption. For example, let's say I have the device in landscapeRight (or landscapeLeft for UIDeviceOrientation). Let's say the device rear camera is pointing towards True North, and I start moving towards True North. I get something like this: Camera Transform = simd_float4x4( [ [0.98446155, -0.030119859, 0.172998, 0.0], [0.023979114, 0.9990097, 0.037477385, 0.0], [-0.17395553, -0.032746706, 0.98420894, 0.0], [0.024039675, -0.037087332, -0.22780673, 0.99999994] ]) As you can see, the cameraTransform.columns.2.z is positive despite the rear camera pointing towards True North, while cameraTransform.columns.3.z is correctly positive as the device is moving towards True North. So here is my question: Why is cameraTransform.columns.2.z positive when the rear camera is physically facing North? Any clarity would be deeply appreciated. I've read the documentation and tested with different heading accuracies and AR session resets, but I keep running into this orientation mismatch. Thanks in advance!
1
0
144
Jun ’25
Sticky Horizontal AnchorEntity
Hi, I'm trying to use AnchorEntity for horizontal surfaces. It works when the entity is being created, but I'm looking for a way to snap this entity to the nearest surface, after translating it, for example with a DragGesture. What would be the best way to achieve this? Using raycast, creating a new anchor, trackingMode to continuous etc. Do I need to use ARKitSession as I want continuous tracking?
1
0
278
Feb ’26
Setting immerstionStyle while in immersive space breaks all entities.
I have my immersive space set up like: ImmersiveSpace(id: "Theater") { ImmersiveTeleopView() .environment(appModel) .onAppear() { appModel.immersiveSpaceState = .open } .onDisappear { appModel.immersiveSpaceState = .closed } } .immersionStyle(selection: .constant(appModel.immersionStyle.style), in: .mixed, .full) Which allows me to set the immersive style while in the space (from a Picker on a SwiftUI window). The scene responds correctly but a lot of the functionality of my immersive space is gone after the change in style; in that I am no longer able to enable/disable entities (which I also have a toggles for in the SwiftUI window). I have to exit and reenter the immersive space to regain the ability to change the enabled state of my entities. My appModel.immersionStyle is inspired by the Compositor-Services demo (although I am using a RealityView) listed in https://developer.apple.com/documentation/CompositorServices/interacting-with-virtual-content-blended-with-passthrough and looks like this: public enum IStyle: String, CaseIterable, Identifiable { case mixedStyle, fullStyle public var id: Self { self } var style: ImmersionStyle { switch self { case .mixedStyle: return .mixed case .fullStyle: return .full } } } /// Maintains app-wide state @MainActor @Observable class AppModel { // Immersion Style public var immersionStyle: IStyle = .mixedStyle
1
0
242
Oct ’25
Possible contradiction between ARKit's definition of UIDeviceOrientation.landscapeRight and the actual definition of UIDeviceOrientation.landscapeRight
So it seems to be that there is a contradiction between how ARKit defines UIDeviceOrientation.landscapeRight, and the actual definition of UIDeviceOrientation.landscapeRight in the UIKit documentation. In the ARKit documentation for ARCamera.transform, it says the following: This transform creates a local coordinate space for the camera that is constant with respect to device orientation. In camera space, the x-axis points to the right when the device is in UIDeviceOrientation.landscapeRight orientation—that is, the x-axis always points along the long axis of the device, from the front-facing camera toward the Home button. The y-axis points upward (with respect to UIDeviceOrientation.landscapeRight orientation), and the z-axis points away from the device on the screen side. Going through the same link, we see the definition of UIDeviceOrientation.landscapeRight given as: The device is in landscape mode, with the device held upright and the front-facing camera on the right side. There seems to be a conflict in the two definitions, that has already been asked and visualized in this StackOverflow thread The resolution of that answer says that ARKit landscapeRight, unlike what is given in UIDeviceOrientation.landscapeRight, has home button on the right, as stated in the ARCamera.transform documentation. It says that more details are given in this StackOverflow thread, but this thread talks about the discrepancy between the definitions of landscapeRight in UIDeviceOrientation and UIInterfaceOrientation, and not anything related to ARKit. So I am wondering, why does ARKit definition of landscapeRight contradict with that of UIDeviceOrientation despite explicitly mentioning it? Is it just a mistake by Apple developers that hasn't been resolved even after so long?
1
0
110
Jun ’25
VisionOs Development: Seeking Advice on Key Strategic Crossroads
I am a developer working on developing a space journal application. During the development process, I encountered several crucial strategic and technical decisions, and I would like to hear the experiences of those who have gone through similar situations. Here are the simplified versions of several questions I have. Resource allocation: Which problem should I address first? Design direction: In terms of interaction and UI design, how should I balance "immersion" and "usability"? Market selection: Was it easier for a business to survive in the early stages as a B2B or B2C entity? Cost estimation: How can I reasonably present to my investors the development costs of this project? In order to avoid relying solely on intuition in my decisions, I created a short questionnaire, hoping to gather more structured opinions from my colleagues. If you are also exploring VisionOS, I sincerely hope you can take a few minutes to fill it out. The results are extremely important to me, and I would be more than happy to share the final summary findings with you.
1
0
158
Feb ’26
AVPlayer stutters when using AVPlayerItemVideoOutput
We’re trying to build a custom player for Unity. For this, we’re using AVPlayer with AVPlayerItemVideoOutput to get textures. However, we noticed that playback is not smooth and the stream often freezes. For testing, we used this 8K video: https://deovr.com/nwfnq1 The video was played using the following code: @objc public func playVideo(urlString: String) { guard let url = URL(string: urlString) else { return } let pItem = AVPlayerItem(url: url) playerItem = pItem pItem.preferredForwardBufferDuration = 10.0 let pixelBufferAttributes: [String: Any] = [ kCVPixelBufferPixelFormatTypeKey as String: kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange, kCVPixelBufferMetalCompatibilityKey as String: true, ] let output = AVPlayerItemVideoOutput( pixelBufferAttributes: pixelBufferAttributes ) pItem.add(output) playerItemObserver = pItem.observe(\.status) { [weak self] pItem, _ in guard pItem.status == .readyToPlay else { return } self?.playerItemObserver = nil self?.player.play() } player = AVPlayer(playerItem: pItem) player.currentItem?.preferredPeakBitRate = 35_000_000 } When AVPlayerItemVideoOutput is attached, the video stutters and the log looks like this: 🟢 Playback likely to keep up 🟡 Buffer ahead: 4.08s | buffer: 4.08s 🟡 Buffer ahead: 4.08s | buffer: 4.08s 🟡 Buffer ahead: -0.07s | buffer: 0.00s 🟡 Buffer ahead: 2.94s | buffer: 3.49s 🟡 Buffer ahead: 2.50s | buffer: 4.06s 🟡 Buffer ahead: 1.74s | buffer: 4.30s 🟡 Buffer ahead: 0.74s | buffer: 4.30s 🟠 Playback may stall 🛑 Buffer empty 🟡 Buffer ahead: 0.09s | buffer: 4.30s 🟠 Playback may stall 🟠 Playback may stall 🛑 Buffer empty 🟠 Playback may stall 🟣 Buffer full 🟡 Buffer ahead: 1.41s | buffer: 1.43s 🟡 Buffer ahead: 1.41s | buffer: 1.43s 🟡 Buffer ahead: 1.07s | buffer: 1.43s 🟣 Buffer full 🟡 Buffer ahead: 0.47s | buffer: 1.65s 🟠 Playback may stall 🛑 Buffer empty 🟡 Buffer ahead: 0.10s | buffer: 1.65s 🟠 Playback may stall 🟡 Buffer ahead: 1.99s | buffer: 2.03s 🟡 Buffer ahead: 1.99s | buffer: 2.03s 🟣 Buffer full 🟣 Buffer full 🟡 Buffer ahead: 1.41s | buffer: 2.00s 🟡 Buffer ahead: 0.68s | buffer: 2.27s 🟡 Buffer ahead: 0.09s | buffer: 2.27s 🟠 Playback may stall 🛑 Buffer empty 🟠 Playback may stall When we remove AVPlayerItemVideoOutput from the player, the video plays smoothly, and the output looks like this: 🟢 Playback likely to keep up 🟡 Buffer ahead: 1.94s | buffer: 1.94s 🟡 Buffer ahead: 1.94s | buffer: 1.94s 🟡 Buffer ahead: 1.22s | buffer: 2.22s 🟡 Buffer ahead: 1.05s | buffer: 3.05s 🟡 Buffer ahead: 1.12s | buffer: 4.12s 🟡 Buffer ahead: 1.18s | buffer: 5.18s 🟡 Buffer ahead: 0.72s | buffer: 5.72s 🟡 Buffer ahead: 1.27s | buffer: 7.28s 🟡 Buffer ahead: 2.09s | buffer: 3.03s 🟡 Buffer ahead: 4.16s | buffer: 6.10s 🟡 Buffer ahead: 6.66s | buffer: 7.09s 🟡 Buffer ahead: 5.66s | buffer: 7.09s 🟡 Buffer ahead: 4.66s | buffer: 7.09s 🟡 Buffer ahead: 4.02s | buffer: 7.45s 🟡 Buffer ahead: 3.62s | buffer: 8.05s 🟡 Buffer ahead: 2.62s | buffer: 8.05s 🟡 Buffer ahead: 2.49s | buffer: 3.53s 🟡 Buffer ahead: 2.43s | buffer: 3.38s 🟡 Buffer ahead: 1.90s | buffer: 3.85s We’ve tried different attribute settings for AVPlayerItemVideoOutput. We also removed all logic related to reading frame data, but the choppy playback still remained. Can you advise whether this is a player issue or if we’re doing something wrong?
1
0
411
Oct ’25
RealityKit_DirectionalLight_Question
My application calculates three distinct Meesus Double [x, y, z] Radian values to light a sphere in RealityKit with DirectionalLight. It is my understanding that I must use (simd_quatf) for each radian value to properly light the sphere in the view. The code correctly [orientates] the sphere with the combined (simd_quatf) DirectionalLight in the view, but the illumination (Z-axis) fails to properly illuminate the sphere with the expected result, compared to associated Meesus web page images. For the moment, I do not know how to correct the (Z-axis). Curious for a suggestion ... :] // Location values. let theLatitude: Double = 51.13107260 let theLongitude: Double = -114.01127910 let currentDate: Date = Date() struct TheCalculatedMoonPhaseTest_ContentView: View { var body: some View { VStack { if #available(macOS 15.0, *) { RealityView { content in let moonSphere_Entity = Entity.createSphere(radius: 0.90, color: .black) moonSphere.Entity.name = "MoonSphere" moonSphere.Entity.position = SIMD3<Float>(x: 0, y: 0, z: 0) content.add(moonSphere.Entity) let sunLight_Entity = createDirectionalLight(latitude: theLatitude, longitude: theLongitude, date: currentDate) content.add(sunLight_Entity) } // End of [RealityView] } else { // Earlier version required. } // End of [if #available(macOS 15.0, *)] } // End of [VStack] .background(Color.black) } // End of [var body: some View] // MARK: - 🟠🟠🟠🟠 [SET THE BACKGROUND COLOUR] 🟠🟠🟠🟠 var backgroundColor: Color = Color.init(.black) // MARK: - 🟠🟠🟠🟠 [CREATE THE DIRECTIONAL LIGHT FOR THE SPHERE] 🟠🟠🟠🟠 func createDirectionalLight(latitude: Double, longitude: Double, date: Date) -> Entity { let directionalLight = DirectionalLight() directionalLight.light.color = .white directionalLight.light.intensity = 1000000 directionalLight.shadow = DirectionalLightComponent.Shadow() directionalLight.shadow?.maximumDistance = 5 directionalLight.shadow?.depthBias = 1 // MARK: 🟠🟠🟠🟠 Retrieve the [MEESUS MOON AGE VALUES] from the [CONSTANT FOLDER] 🟠🟠🟠🟠 let theMeesusMoonAge_LunarAgeDaysValue = 25.90567592898601 if theMeesusMoonAge_LunarAgeDaysValue >= 23.10 && theMeesusMoonAge_LunarAgeDaysValue < (29.530588853 - 1.00) { let someCalculatedX_WestEastRadian: Float = Float(1.00) // Identify the sphere’s DirectionalLight Tilt Angle (Y) radian value :: // Note :: The following Tilt Angle is corrected to [Zenith] with the [MeesusCalculatedTilt_Angle] minus the [MeesusCalculatedPar_Angle]. let someCalculatedY_TiltAngleRadian: Float = Float(1.3396086) // Identify the sphere’s DirectionalLight Illumination (Z) radian Value :: // Note :: The Meesus calculated illumination fraction is converted to degrees, then converted to a radian value. let someCalculatedZ_IlluminationAngleRadian: Float = Float(0.45176168630244457) // <=== 14.3800% Illumination. // Define rotation angles in radians for X, Y, and Z axes. let x_Radians = someCalculatedX_WestEastRadian let y_Radians = someCalculatedY_TiltAngleRadian let z_Radians = someCalculatedZ_IlluminationAngleRadian // Identify and separate the quaternion [simd_quatf] for each Radian. let q_X = simd_quatf(angle: x_Radians, axis: SIMD3<Float>(1, 0, 0)) let q_Y = simd_quatf(angle: y_Radians, axis: SIMD3<Float>(0, 1, 0)) let q_Z = simd_quatf(angle: z_Radians, axis: SIMD3<Float>(0, 0, 1)) // Apply and combine the rotations, where order matters. let combinedRotation = q_Z * q_Y * q_X // Identify the [Combined Rotation]. // The [MyMoonMeesus] :: [WANING CRESCENT] calculated [combinedRotation] :: simd_quatf(real: 0.73715997, imag: SIMD3<Float>(0.24427173, 0.61516714, -0.13599981)) ° Radians // Normalize the [combinedRotation]. let theNormalizesRotation = simd_normalize(combinedRotation) // Identify the [Normalized Combined Rotation]. // The [MyMoonMeesus] :: [WANING CRESCENT] calculated [normalizedRotation] :: simd_quatf(real: 0.73715997, imag: SIMD3<Float>(0.24427173, 0.61516714, -0.13599981)) ° Radians // Assume the [theNormalizesRotation] appears reversed. let theCorrectedRotation = theNormalizesRotation.inverse // Identify the [Reversed Combined Rotation]. // The [MyMoonMeesus] :: [WANING CRESCENT] calculated [correctedRotation] :: simd_quatf(real: 0.73715997, imag: SIMD3<Float>(-0.24427173, -0.61516714, 0.13599981)) ° Radians // Apply the [Corrected Rotation] to the entity. directionalLight.transform.rotation *= theCorrectedRotation // Add the [directionalLight] to the scene :: let anchor = AnchorEntity() anchor.addChild(directionalLight) } // End of [if theMeesusMoonAge_LunarAgeDaysValue >= 23.10 && theMeesusMoonAge_LunarAgeDaysValue < (29.530588853 - 1.00)] return directionalLight } // End of [func createDirectionalLight(latitude: Double, longitude: Double, date: Date) -> Entity] } // End of [struct TheCalculatedMoonPhaseTest_ContentView: View] // MARK: 🟠🟠🟠🟠 [ENTITY HELPER EXTENSION] 🟠🟠🟠🟠 extension Entity { static func createSphere(radius: Float, color: NSColor) -> Entity { let mesh = MeshResource.generateSphere(radius: radius) var material = PhysicallyBasedMaterial() material.baseColor = .init(tint: color) let modelComponent = ModelComponent(mesh: mesh, materials: [material]) let entity = Entity() entity.components.set(modelComponent) entity.components.set(Transform()) return entity } // End of [static func createSphere(radius: Float, color: NSColor) -> Entity] } // End of [extension Entity] // Application Image :: Calgary // Website Image :: timeanddate // mooncalc.org
1
0
176
Feb ’26
Please provide access to face tracking blendshapes on vision os
Apple, please provide access to face tracking blend shapes on vision os, just like you do on iOS. You have the best eye and face tracking implementation on the market, please let us use it. There is a sizable audience who will buy the headset just for it. I personally know multiple people who are not buying the headset simply because you locked those features out. No raw camera access is needed, just abstracted blendshapes values. You will make the headset so much more useful if you do this simple thing.
1
1
147
Jun ’25
visualBounds ignores TextComponents set for Entity. Workarounds?
After adding TextComponents to my Entities on visionOS, I have observed that visualBounds will ignore the TextComponents. Documentation states that it should render a rounded rectangle mesh. These mashes are visible on the device, but not visible in the debugger ("Capture Entity Hierarchy") and ignored by visualBounds. Am I missing something? static func makeDirection(_ direction: Direction) -> Entity { let text = Entity() text.name = direction.rawValue text.setScale(SIMD3(repeating: 5), relativeTo: nil) text.transform.rotation = direction.rotation text.components.set(direction.textComponent) return text } My workaround is to add a disabled ModelEntity and take its bounds 😬
1
0
294
Dec ’25
Collision Detection Fails After Anchoring ModelEntity to Hand in VisionOS
I'm starting my journey in developing an immersive app for VisionOS. I've been making steady progress, but I've encountered a specific challenge that I haven't been able to resolve. I created two ModelEntity objects — a sphere and a cube — and added a DragGesture to the cube. When I drag the cube over the sphere, the two collide correctly, and the collision is logged in the console. So far, everything works as expected. However, when I try to anchor the cube to my hand, the collision stops working. It's as if the cube loses its ability to detect collisions once it's anchored. Any guidance or clarification on this behavior would be greatly appreciated. //  ImmersiveView.swift //  estudos_vision // //  Created by Lailan Rogerio Rodrigues Matos on 15/05/25. // import SwiftUI import RealityKit import RealityKitContent struct ImmersiveView: View { @Environment(AppModel.self) var appModel @State private var session: SpatialTrackingSession? @State private var box = ModelEntity() @State private var subs: [EventSubscription] = [] @State private var ballEntity: Entity? var body: some View { RealityView { content in // Load initial content from the RealityKit scene. if let immersiveContentEntity = try? await Entity(named: "Immersive", in: realityKitContentBundle) { content.add(immersiveContentEntity) } // Create and run a spatial tracking session. let session = SpatialTrackingSession() let configuration = SpatialTrackingSession.Configuration(tracking: [.hand]) _ = await session.run(configuration) self.session = session // Create a red box. let boxMesh = MeshResource.generateBox(size: 0.2) let material = SimpleMaterial(color: .red, isMetallic: false) box = ModelEntity(mesh: boxMesh, materials: [material]) box.position.y += 0.15 // Position the box slightly above the origin. // Configure the box for user interaction and physics. box.components.set(InputTargetComponent(allowedInputTypes: .indirect)) // Make it interactive. box.generateCollisionShapes(recursive: false) // Generate collision shapes for physics. box.components.set(PhysicsBodyComponent( // Add physics behavior. massProperties: .default, material: .default, mode: .kinematic // Use kinematic mode so it can be moved by user interaction. )) box.components.set(GroundingShadowComponent(castsShadow: true)) // Add a shadow. //content.add(box) //commented out to add to hand anchor // Create a left hand anchor and add the box as a child. let handAnchor = AnchorEntity(.hand(.left, location: .palm), trackingMode: .continuous) handAnchor.addChild(box) content.add(handAnchor) // Add the hand anchor to the scene. // Create a sphere. let ball = ModelEntity(mesh: .generateSphere(radius: 0.15)) ball.position = [0.0, 1.5, -1.0] // Initial position of the ball. ball.generateCollisionShapes(recursive: false) // Add collision. ball.name = "Sphere" content.add(ball) ballEntity = ball // Subscribe to collision events between the box and other entities. let event = content.subscribe(to: CollisionEvents.Began.self, on: box) { ce in print("Collision between \(ce.entityA.name) and \(ce.entityB.name) occurred") //ce.entityA.removeFromParent() // removes the colliding object //ce.entityB.removeFromParent() } Task { subs.append(event) } } // Add a drag gesture to the box, allowing the user to move it. .gesture( DragGesture() .targetedToEntity(box) // Target the drag gesture to the box. .onChanged({ value in // Update the position of the box based on the drag gesture. box.position = value.convert(value.location3D, from: .local, to: box.parent!) }) ) } } #Preview(immersionStyle: .full) { ImmersiveView() .environment(AppModel()) }
1
0
104
May ’25
Safari-like toolbar in visionOS
I like the toolbar visionOS's Safari uses for back & forward page, share, etc. It floats above the window. My attempt to do this with ornaments isn't as satisfying as they partially cover the window. My attempts with toolbar haven't produced visible results. Is this Safari-style toolbar for visionOS exposed by Apple in the API's? If so, could someone point me to documentation or sample code? Thanks!
1
0
266
Oct ’25
I need to troubleshoot Transform Drift in ARKit
Hi all, I'm currently developing a real-time object reconstruction app using ARKit. The goal is to scan large objects using ARKit’s depth and transform data, and generate a point cloud. However, I’m facing a major challenge - Transform Drift / World Alignment Issues The localToWorld transform provided by ARKit frequently seems to drift or become unstable across frames. This results in misaligned point clouds even when the device is moved slowly or kept relatively still. In some cases, a static surface scanned over a few seconds results in clearly misaligned fragments. This makes it difficult to accurately stitch a multi-frame point cloud. I have experimented with various lighting conditions and object textures, but the issue persists in all cases. At times, the relative error between frames reaches up to 20 cm, while in other instances the error is minimal; however, the drift gradually accumulates over time, leading to an overall enlargement of the reconstructed object. I have attached images of both cases here. Questions: Are there specific conditions under which ARKit’s world transform is expected to drift? Is there a way to detect or recover from this drift during runtime? Any best practices for maintaining consistent tracking during scanning or measurement sessions?
1
0
146
Jun ’25
Share Extensions embedded in visionOS apps
I'm trying to add a feature to my app to allow a user to import items from other apps, like Safari, via the share sheet. I've done this many times on iOS/iPadOS easily with a Share Extension. From what I can tell, Xcode tells me share extensions are not available on visionOS - though my experience on device tells me differently (It seems Reminders, Notes & more implement them somehow.) I was finally able to get it working on device only...but I can now no longer test in the simulator, and have not found a way to distribute this app. When attempting to run on the simulator, I get this issue: Please try again later. Appex bundle at /Users/jason/Library/Developer/CoreSimulator/Devices/09A70160-4F4F-4F5E-B679-F6F7D876D7EF/data/Library/Caches/com.apple.mobile.installd.staging/temp.6OAEZp/extracted/LaunchBar.app/PlugIns/LaunchBarShareExtension.appex with id co.swiftfox.LaunchBar.ShareExtension specifies a value (com.apple.share-services) for the NSExtensionPointIdentifier key in the NSExtension dictionary in its Info.plist that does not correspond to a known extension point. When trying to archive an upload to test flight, I get this similar error: Invalid Info.plist value. The value for the key 'DTPlatformName' in bundle LaunchBar.app/PlugIns/LaunchBarShareExtension.appex is invalid. (ID: 207610c7-b7e1-48be-959b-22a43cd32d16) The app is for visionOS only - which I'm thinking might be the problem? The share extension is "Designed For iPhone" and requires me to include iPhone as a run destination. In the worst case I can build an iPhone UI for the app but I'd rather not, as it is very specific to visionOS. Has anyone successfully launched a share extension on a visionOS only app? I have an iPad app with a share extension that shows up fine on visionOS, but the issue seems to be specifically with visionOS only apps.
1
0
122
Oct ’25
Sample code not working as expected: Implementing SharePlay for immersive spaces in visionOS
The following sample code project does not seem to work as expected: https://developer.apple.com/documentation/visionos/implementing-shareplay-for-immersive-spaces-in-visionos Have tried to get this project working with a client, but while we were able to see nearby users and make facetime calls, the color changing cube experience always remained a single color. Are there step-by-step instructions that Apple has used to verify this sample code so I can try to recreate this sample code's expected behavior for both nearby participants and those in a Facetime call?
1
0
265
Feb ’26
With manipulation component, once you let go, how to prevent the entity from disappearing while animating it back into the volume
So with the new ManipulationComponent, we can choose "stay" and then if you drag it out of your volume, once you let go it will instantly disappear. We can "animate" it back to inside the volume, eg.: content.subscribe(to: ManipulationEvents.WillRelease.self) { event in Entity.animate( .easeInOut(duration: 1), body: { event.entity.position = [0, 0.2, 0] }, completion: {} ) }, Howeve,r for the duration that it travels outside of the volume it's invisible the whole time. In this apple video, it seems to be visible when dragging and when letting go, but perhaps that's not a volume they're dragging it out of? https://youtu.be/VtenPKrvPOU?si=y1zoZOs2IMyDzOm6&t=1748 Does anyone know how to keep the entity visible even when after letting the entity go while you animate it back towards inside of your volume?
Replies
1
Boosts
1
Views
986
Activity
Jan ’26
ManipulationComponent in both parent and child entities
Hello, In my project, I have attached a ManipulationComponent to Entity A and as expected, I'm able interact with it using the built-in gestures. I have another Entity B which is a child of A that I would like to interact with as well, so I attempted to add a ManipulationComponent to B. However, no gestures seem to be registered on B; I can still interact with A but B cannot be interacted with despite having ManipulationComponents on both entities. So I'm wondering if I'm just doing something wrong, if this is an issue with the ManipulationComponent, or if this is a limitation of the API. Attached is the code used to add the ManipulationComponent to an Entity and it was done on both A and B: let mc = ManipulationComponent() model.components.set(mc) var boxShape = ShapeResource.generateBox(width: 0.25, height: 0.05, depth: 0.25) boxShape = boxShape.offsetBy(translation: simd_float3(0, -0.05, -0.25)) ManipulationComponent.configureEntity(model, collisionShapes: [boxShape]) if var mc = model.components[ManipulationComponent.self] { mc.releaseBehavior = .stay mc.dynamics.inertia = .low model.components.set(mc) } I am using visionOS 26.0; let me know if there's any additional information needed.
Replies
1
Boosts
0
Views
376
Activity
Oct ’25
Current Apple Forum about ARKit and visionOS
Recently, questions about ARKit/visionOS seem to be being asked in the Apple forum by internal Apple engineers. Inexperienced and untested makeshift features are being offered, putting average but experienced developers in a difficult position. They are unable to react and get something useful from the posts. Apple needs to review the situation.
Replies
1
Boosts
0
Views
372
Activity
Sep ’25
The folding and unfolding effect of the NBA sand table
Seeing this magical sand table, the unfolding and folding effects are similar to spreading out cards, which is very interesting. But I don't know how to achieve it. I want to see if there are any ways to achieve this effect and give some ideas. May I ask if this effect can be achieved under the existing API
Replies
1
Boosts
0
Views
99
Activity
May ’25
Displaying multiple immersive movies in spheres in an immersive environment
In visionOS, I'm trying to create an immersive environment which would feature several spheres in which immersive movies are visible. I'm starting from a sample code which creates a sphere, sets an immersive movie as its material, and opens it as an immersive environment. This works fine. But if I create a sphere in an open immersive environment using Reality Composer Pro and sets its material to an immersive movie, I can see the movie on the sphere while I move outside of it but if I try to get inside the sphere, it disappears. What would be the right way of doing this ?
Replies
1
Boosts
1
Views
722
Activity
Oct ’25
ARKit camera transform orientation vector doesn't match physical device heading (despite `.gravityAndHeading`)
Hi all, I'm working on an ARKit-based iOS app where I need to accurately determine the direction the device is facing to localize objects in the real world. I'm using: let config = ARWorldTrackingConfiguration() config.worldAlignment = .gravityAndHeading Thus, I would expect the world alignment to behave as given in the gravityAndHeading page. The AR session is started after verifying that CLLocationManager.headingAccuracy <= 20, and the compass appears to be calibrated. However, I'm seeing a major inconsistency: When the rear camera is physically pointed toward true North, I would expect: cameraTransform.columns.2.z ≈ -1 // (i.e. ARKit's -Z pointing North) But instead, I'm consistently seeing: cameraTransform.columns.2.z ≈ +0.97 // Implies camera is facing South Meanwhile, the translation vector behaves as expected: As I physically move North, cameraTransform.columns.3.z becomes more negative, matching the world’s +Z = South assumption. For example, let's say I have the device in landscapeRight (or landscapeLeft for UIDeviceOrientation). Let's say the device rear camera is pointing towards True North, and I start moving towards True North. I get something like this: Camera Transform = simd_float4x4( [ [0.98446155, -0.030119859, 0.172998, 0.0], [0.023979114, 0.9990097, 0.037477385, 0.0], [-0.17395553, -0.032746706, 0.98420894, 0.0], [0.024039675, -0.037087332, -0.22780673, 0.99999994] ]) As you can see, the cameraTransform.columns.2.z is positive despite the rear camera pointing towards True North, while cameraTransform.columns.3.z is correctly positive as the device is moving towards True North. So here is my question: Why is cameraTransform.columns.2.z positive when the rear camera is physically facing North? Any clarity would be deeply appreciated. I've read the documentation and tested with different heading accuracies and AR session resets, but I keep running into this orientation mismatch. Thanks in advance!
Replies
1
Boosts
0
Views
144
Activity
Jun ’25
Sticky Horizontal AnchorEntity
Hi, I'm trying to use AnchorEntity for horizontal surfaces. It works when the entity is being created, but I'm looking for a way to snap this entity to the nearest surface, after translating it, for example with a DragGesture. What would be the best way to achieve this? Using raycast, creating a new anchor, trackingMode to continuous etc. Do I need to use ARKitSession as I want continuous tracking?
Replies
1
Boosts
0
Views
278
Activity
Feb ’26
Setting immerstionStyle while in immersive space breaks all entities.
I have my immersive space set up like: ImmersiveSpace(id: "Theater") { ImmersiveTeleopView() .environment(appModel) .onAppear() { appModel.immersiveSpaceState = .open } .onDisappear { appModel.immersiveSpaceState = .closed } } .immersionStyle(selection: .constant(appModel.immersionStyle.style), in: .mixed, .full) Which allows me to set the immersive style while in the space (from a Picker on a SwiftUI window). The scene responds correctly but a lot of the functionality of my immersive space is gone after the change in style; in that I am no longer able to enable/disable entities (which I also have a toggles for in the SwiftUI window). I have to exit and reenter the immersive space to regain the ability to change the enabled state of my entities. My appModel.immersionStyle is inspired by the Compositor-Services demo (although I am using a RealityView) listed in https://developer.apple.com/documentation/CompositorServices/interacting-with-virtual-content-blended-with-passthrough and looks like this: public enum IStyle: String, CaseIterable, Identifiable { case mixedStyle, fullStyle public var id: Self { self } var style: ImmersionStyle { switch self { case .mixedStyle: return .mixed case .fullStyle: return .full } } } /// Maintains app-wide state @MainActor @Observable class AppModel { // Immersion Style public var immersionStyle: IStyle = .mixedStyle
Replies
1
Boosts
0
Views
242
Activity
Oct ’25
Possible contradiction between ARKit's definition of UIDeviceOrientation.landscapeRight and the actual definition of UIDeviceOrientation.landscapeRight
So it seems to be that there is a contradiction between how ARKit defines UIDeviceOrientation.landscapeRight, and the actual definition of UIDeviceOrientation.landscapeRight in the UIKit documentation. In the ARKit documentation for ARCamera.transform, it says the following: This transform creates a local coordinate space for the camera that is constant with respect to device orientation. In camera space, the x-axis points to the right when the device is in UIDeviceOrientation.landscapeRight orientation—that is, the x-axis always points along the long axis of the device, from the front-facing camera toward the Home button. The y-axis points upward (with respect to UIDeviceOrientation.landscapeRight orientation), and the z-axis points away from the device on the screen side. Going through the same link, we see the definition of UIDeviceOrientation.landscapeRight given as: The device is in landscape mode, with the device held upright and the front-facing camera on the right side. There seems to be a conflict in the two definitions, that has already been asked and visualized in this StackOverflow thread The resolution of that answer says that ARKit landscapeRight, unlike what is given in UIDeviceOrientation.landscapeRight, has home button on the right, as stated in the ARCamera.transform documentation. It says that more details are given in this StackOverflow thread, but this thread talks about the discrepancy between the definitions of landscapeRight in UIDeviceOrientation and UIInterfaceOrientation, and not anything related to ARKit. So I am wondering, why does ARKit definition of landscapeRight contradict with that of UIDeviceOrientation despite explicitly mentioning it? Is it just a mistake by Apple developers that hasn't been resolved even after so long?
Replies
1
Boosts
0
Views
110
Activity
Jun ’25
VisionOs Development: Seeking Advice on Key Strategic Crossroads
I am a developer working on developing a space journal application. During the development process, I encountered several crucial strategic and technical decisions, and I would like to hear the experiences of those who have gone through similar situations. Here are the simplified versions of several questions I have. Resource allocation: Which problem should I address first? Design direction: In terms of interaction and UI design, how should I balance "immersion" and "usability"? Market selection: Was it easier for a business to survive in the early stages as a B2B or B2C entity? Cost estimation: How can I reasonably present to my investors the development costs of this project? In order to avoid relying solely on intuition in my decisions, I created a short questionnaire, hoping to gather more structured opinions from my colleagues. If you are also exploring VisionOS, I sincerely hope you can take a few minutes to fill it out. The results are extremely important to me, and I would be more than happy to share the final summary findings with you.
Replies
1
Boosts
0
Views
158
Activity
Feb ’26
AVPlayer stutters when using AVPlayerItemVideoOutput
We’re trying to build a custom player for Unity. For this, we’re using AVPlayer with AVPlayerItemVideoOutput to get textures. However, we noticed that playback is not smooth and the stream often freezes. For testing, we used this 8K video: https://deovr.com/nwfnq1 The video was played using the following code: @objc public func playVideo(urlString: String) { guard let url = URL(string: urlString) else { return } let pItem = AVPlayerItem(url: url) playerItem = pItem pItem.preferredForwardBufferDuration = 10.0 let pixelBufferAttributes: [String: Any] = [ kCVPixelBufferPixelFormatTypeKey as String: kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange, kCVPixelBufferMetalCompatibilityKey as String: true, ] let output = AVPlayerItemVideoOutput( pixelBufferAttributes: pixelBufferAttributes ) pItem.add(output) playerItemObserver = pItem.observe(\.status) { [weak self] pItem, _ in guard pItem.status == .readyToPlay else { return } self?.playerItemObserver = nil self?.player.play() } player = AVPlayer(playerItem: pItem) player.currentItem?.preferredPeakBitRate = 35_000_000 } When AVPlayerItemVideoOutput is attached, the video stutters and the log looks like this: 🟢 Playback likely to keep up 🟡 Buffer ahead: 4.08s | buffer: 4.08s 🟡 Buffer ahead: 4.08s | buffer: 4.08s 🟡 Buffer ahead: -0.07s | buffer: 0.00s 🟡 Buffer ahead: 2.94s | buffer: 3.49s 🟡 Buffer ahead: 2.50s | buffer: 4.06s 🟡 Buffer ahead: 1.74s | buffer: 4.30s 🟡 Buffer ahead: 0.74s | buffer: 4.30s 🟠 Playback may stall 🛑 Buffer empty 🟡 Buffer ahead: 0.09s | buffer: 4.30s 🟠 Playback may stall 🟠 Playback may stall 🛑 Buffer empty 🟠 Playback may stall 🟣 Buffer full 🟡 Buffer ahead: 1.41s | buffer: 1.43s 🟡 Buffer ahead: 1.41s | buffer: 1.43s 🟡 Buffer ahead: 1.07s | buffer: 1.43s 🟣 Buffer full 🟡 Buffer ahead: 0.47s | buffer: 1.65s 🟠 Playback may stall 🛑 Buffer empty 🟡 Buffer ahead: 0.10s | buffer: 1.65s 🟠 Playback may stall 🟡 Buffer ahead: 1.99s | buffer: 2.03s 🟡 Buffer ahead: 1.99s | buffer: 2.03s 🟣 Buffer full 🟣 Buffer full 🟡 Buffer ahead: 1.41s | buffer: 2.00s 🟡 Buffer ahead: 0.68s | buffer: 2.27s 🟡 Buffer ahead: 0.09s | buffer: 2.27s 🟠 Playback may stall 🛑 Buffer empty 🟠 Playback may stall When we remove AVPlayerItemVideoOutput from the player, the video plays smoothly, and the output looks like this: 🟢 Playback likely to keep up 🟡 Buffer ahead: 1.94s | buffer: 1.94s 🟡 Buffer ahead: 1.94s | buffer: 1.94s 🟡 Buffer ahead: 1.22s | buffer: 2.22s 🟡 Buffer ahead: 1.05s | buffer: 3.05s 🟡 Buffer ahead: 1.12s | buffer: 4.12s 🟡 Buffer ahead: 1.18s | buffer: 5.18s 🟡 Buffer ahead: 0.72s | buffer: 5.72s 🟡 Buffer ahead: 1.27s | buffer: 7.28s 🟡 Buffer ahead: 2.09s | buffer: 3.03s 🟡 Buffer ahead: 4.16s | buffer: 6.10s 🟡 Buffer ahead: 6.66s | buffer: 7.09s 🟡 Buffer ahead: 5.66s | buffer: 7.09s 🟡 Buffer ahead: 4.66s | buffer: 7.09s 🟡 Buffer ahead: 4.02s | buffer: 7.45s 🟡 Buffer ahead: 3.62s | buffer: 8.05s 🟡 Buffer ahead: 2.62s | buffer: 8.05s 🟡 Buffer ahead: 2.49s | buffer: 3.53s 🟡 Buffer ahead: 2.43s | buffer: 3.38s 🟡 Buffer ahead: 1.90s | buffer: 3.85s We’ve tried different attribute settings for AVPlayerItemVideoOutput. We also removed all logic related to reading frame data, but the choppy playback still remained. Can you advise whether this is a player issue or if we’re doing something wrong?
Replies
1
Boosts
0
Views
411
Activity
Oct ’25
RealityKit_DirectionalLight_Question
My application calculates three distinct Meesus Double [x, y, z] Radian values to light a sphere in RealityKit with DirectionalLight. It is my understanding that I must use (simd_quatf) for each radian value to properly light the sphere in the view. The code correctly [orientates] the sphere with the combined (simd_quatf) DirectionalLight in the view, but the illumination (Z-axis) fails to properly illuminate the sphere with the expected result, compared to associated Meesus web page images. For the moment, I do not know how to correct the (Z-axis). Curious for a suggestion ... :] // Location values. let theLatitude: Double = 51.13107260 let theLongitude: Double = -114.01127910 let currentDate: Date = Date() struct TheCalculatedMoonPhaseTest_ContentView: View { var body: some View { VStack { if #available(macOS 15.0, *) { RealityView { content in let moonSphere_Entity = Entity.createSphere(radius: 0.90, color: .black) moonSphere.Entity.name = "MoonSphere" moonSphere.Entity.position = SIMD3<Float>(x: 0, y: 0, z: 0) content.add(moonSphere.Entity) let sunLight_Entity = createDirectionalLight(latitude: theLatitude, longitude: theLongitude, date: currentDate) content.add(sunLight_Entity) } // End of [RealityView] } else { // Earlier version required. } // End of [if #available(macOS 15.0, *)] } // End of [VStack] .background(Color.black) } // End of [var body: some View] // MARK: - 🟠🟠🟠🟠 [SET THE BACKGROUND COLOUR] 🟠🟠🟠🟠 var backgroundColor: Color = Color.init(.black) // MARK: - 🟠🟠🟠🟠 [CREATE THE DIRECTIONAL LIGHT FOR THE SPHERE] 🟠🟠🟠🟠 func createDirectionalLight(latitude: Double, longitude: Double, date: Date) -> Entity { let directionalLight = DirectionalLight() directionalLight.light.color = .white directionalLight.light.intensity = 1000000 directionalLight.shadow = DirectionalLightComponent.Shadow() directionalLight.shadow?.maximumDistance = 5 directionalLight.shadow?.depthBias = 1 // MARK: 🟠🟠🟠🟠 Retrieve the [MEESUS MOON AGE VALUES] from the [CONSTANT FOLDER] 🟠🟠🟠🟠 let theMeesusMoonAge_LunarAgeDaysValue = 25.90567592898601 if theMeesusMoonAge_LunarAgeDaysValue >= 23.10 && theMeesusMoonAge_LunarAgeDaysValue < (29.530588853 - 1.00) { let someCalculatedX_WestEastRadian: Float = Float(1.00) // Identify the sphere’s DirectionalLight Tilt Angle (Y) radian value :: // Note :: The following Tilt Angle is corrected to [Zenith] with the [MeesusCalculatedTilt_Angle] minus the [MeesusCalculatedPar_Angle]. let someCalculatedY_TiltAngleRadian: Float = Float(1.3396086) // Identify the sphere’s DirectionalLight Illumination (Z) radian Value :: // Note :: The Meesus calculated illumination fraction is converted to degrees, then converted to a radian value. let someCalculatedZ_IlluminationAngleRadian: Float = Float(0.45176168630244457) // <=== 14.3800% Illumination. // Define rotation angles in radians for X, Y, and Z axes. let x_Radians = someCalculatedX_WestEastRadian let y_Radians = someCalculatedY_TiltAngleRadian let z_Radians = someCalculatedZ_IlluminationAngleRadian // Identify and separate the quaternion [simd_quatf] for each Radian. let q_X = simd_quatf(angle: x_Radians, axis: SIMD3<Float>(1, 0, 0)) let q_Y = simd_quatf(angle: y_Radians, axis: SIMD3<Float>(0, 1, 0)) let q_Z = simd_quatf(angle: z_Radians, axis: SIMD3<Float>(0, 0, 1)) // Apply and combine the rotations, where order matters. let combinedRotation = q_Z * q_Y * q_X // Identify the [Combined Rotation]. // The [MyMoonMeesus] :: [WANING CRESCENT] calculated [combinedRotation] :: simd_quatf(real: 0.73715997, imag: SIMD3<Float>(0.24427173, 0.61516714, -0.13599981)) ° Radians // Normalize the [combinedRotation]. let theNormalizesRotation = simd_normalize(combinedRotation) // Identify the [Normalized Combined Rotation]. // The [MyMoonMeesus] :: [WANING CRESCENT] calculated [normalizedRotation] :: simd_quatf(real: 0.73715997, imag: SIMD3<Float>(0.24427173, 0.61516714, -0.13599981)) ° Radians // Assume the [theNormalizesRotation] appears reversed. let theCorrectedRotation = theNormalizesRotation.inverse // Identify the [Reversed Combined Rotation]. // The [MyMoonMeesus] :: [WANING CRESCENT] calculated [correctedRotation] :: simd_quatf(real: 0.73715997, imag: SIMD3<Float>(-0.24427173, -0.61516714, 0.13599981)) ° Radians // Apply the [Corrected Rotation] to the entity. directionalLight.transform.rotation *= theCorrectedRotation // Add the [directionalLight] to the scene :: let anchor = AnchorEntity() anchor.addChild(directionalLight) } // End of [if theMeesusMoonAge_LunarAgeDaysValue >= 23.10 && theMeesusMoonAge_LunarAgeDaysValue < (29.530588853 - 1.00)] return directionalLight } // End of [func createDirectionalLight(latitude: Double, longitude: Double, date: Date) -> Entity] } // End of [struct TheCalculatedMoonPhaseTest_ContentView: View] // MARK: 🟠🟠🟠🟠 [ENTITY HELPER EXTENSION] 🟠🟠🟠🟠 extension Entity { static func createSphere(radius: Float, color: NSColor) -> Entity { let mesh = MeshResource.generateSphere(radius: radius) var material = PhysicallyBasedMaterial() material.baseColor = .init(tint: color) let modelComponent = ModelComponent(mesh: mesh, materials: [material]) let entity = Entity() entity.components.set(modelComponent) entity.components.set(Transform()) return entity } // End of [static func createSphere(radius: Float, color: NSColor) -> Entity] } // End of [extension Entity] // Application Image :: Calgary // Website Image :: timeanddate // mooncalc.org
Replies
1
Boosts
0
Views
176
Activity
Feb ’26
Please provide access to face tracking blendshapes on vision os
Apple, please provide access to face tracking blend shapes on vision os, just like you do on iOS. You have the best eye and face tracking implementation on the market, please let us use it. There is a sizable audience who will buy the headset just for it. I personally know multiple people who are not buying the headset simply because you locked those features out. No raw camera access is needed, just abstracted blendshapes values. You will make the headset so much more useful if you do this simple thing.
Replies
1
Boosts
1
Views
147
Activity
Jun ’25
visualBounds ignores TextComponents set for Entity. Workarounds?
After adding TextComponents to my Entities on visionOS, I have observed that visualBounds will ignore the TextComponents. Documentation states that it should render a rounded rectangle mesh. These mashes are visible on the device, but not visible in the debugger ("Capture Entity Hierarchy") and ignored by visualBounds. Am I missing something? static func makeDirection(_ direction: Direction) -> Entity { let text = Entity() text.name = direction.rawValue text.setScale(SIMD3(repeating: 5), relativeTo: nil) text.transform.rotation = direction.rotation text.components.set(direction.textComponent) return text } My workaround is to add a disabled ModelEntity and take its bounds 😬
Replies
1
Boosts
0
Views
294
Activity
Dec ’25
Rating modal in Apple Vision pro
If I trigger the apple rating modal in an Immersive space it appears on the ground in (0,0,0) I need it to be in front of the user like push notification perimssion does or other permissions requests.
Replies
1
Boosts
0
Views
374
Activity
Oct ’25
Collision Detection Fails After Anchoring ModelEntity to Hand in VisionOS
I'm starting my journey in developing an immersive app for VisionOS. I've been making steady progress, but I've encountered a specific challenge that I haven't been able to resolve. I created two ModelEntity objects — a sphere and a cube — and added a DragGesture to the cube. When I drag the cube over the sphere, the two collide correctly, and the collision is logged in the console. So far, everything works as expected. However, when I try to anchor the cube to my hand, the collision stops working. It's as if the cube loses its ability to detect collisions once it's anchored. Any guidance or clarification on this behavior would be greatly appreciated. //  ImmersiveView.swift //  estudos_vision // //  Created by Lailan Rogerio Rodrigues Matos on 15/05/25. // import SwiftUI import RealityKit import RealityKitContent struct ImmersiveView: View { @Environment(AppModel.self) var appModel @State private var session: SpatialTrackingSession? @State private var box = ModelEntity() @State private var subs: [EventSubscription] = [] @State private var ballEntity: Entity? var body: some View { RealityView { content in // Load initial content from the RealityKit scene. if let immersiveContentEntity = try? await Entity(named: "Immersive", in: realityKitContentBundle) { content.add(immersiveContentEntity) } // Create and run a spatial tracking session. let session = SpatialTrackingSession() let configuration = SpatialTrackingSession.Configuration(tracking: [.hand]) _ = await session.run(configuration) self.session = session // Create a red box. let boxMesh = MeshResource.generateBox(size: 0.2) let material = SimpleMaterial(color: .red, isMetallic: false) box = ModelEntity(mesh: boxMesh, materials: [material]) box.position.y += 0.15 // Position the box slightly above the origin. // Configure the box for user interaction and physics. box.components.set(InputTargetComponent(allowedInputTypes: .indirect)) // Make it interactive. box.generateCollisionShapes(recursive: false) // Generate collision shapes for physics. box.components.set(PhysicsBodyComponent( // Add physics behavior. massProperties: .default, material: .default, mode: .kinematic // Use kinematic mode so it can be moved by user interaction. )) box.components.set(GroundingShadowComponent(castsShadow: true)) // Add a shadow. //content.add(box) //commented out to add to hand anchor // Create a left hand anchor and add the box as a child. let handAnchor = AnchorEntity(.hand(.left, location: .palm), trackingMode: .continuous) handAnchor.addChild(box) content.add(handAnchor) // Add the hand anchor to the scene. // Create a sphere. let ball = ModelEntity(mesh: .generateSphere(radius: 0.15)) ball.position = [0.0, 1.5, -1.0] // Initial position of the ball. ball.generateCollisionShapes(recursive: false) // Add collision. ball.name = "Sphere" content.add(ball) ballEntity = ball // Subscribe to collision events between the box and other entities. let event = content.subscribe(to: CollisionEvents.Began.self, on: box) { ce in print("Collision between \(ce.entityA.name) and \(ce.entityB.name) occurred") //ce.entityA.removeFromParent() // removes the colliding object //ce.entityB.removeFromParent() } Task { subs.append(event) } } // Add a drag gesture to the box, allowing the user to move it. .gesture( DragGesture() .targetedToEntity(box) // Target the drag gesture to the box. .onChanged({ value in // Update the position of the box based on the drag gesture. box.position = value.convert(value.location3D, from: .local, to: box.parent!) }) ) } } #Preview(immersionStyle: .full) { ImmersiveView() .environment(AppModel()) }
Replies
1
Boosts
0
Views
104
Activity
May ’25
Safari-like toolbar in visionOS
I like the toolbar visionOS's Safari uses for back & forward page, share, etc. It floats above the window. My attempt to do this with ornaments isn't as satisfying as they partially cover the window. My attempts with toolbar haven't produced visible results. Is this Safari-style toolbar for visionOS exposed by Apple in the API's? If so, could someone point me to documentation or sample code? Thanks!
Replies
1
Boosts
0
Views
266
Activity
Oct ’25
I need to troubleshoot Transform Drift in ARKit
Hi all, I'm currently developing a real-time object reconstruction app using ARKit. The goal is to scan large objects using ARKit’s depth and transform data, and generate a point cloud. However, I’m facing a major challenge - Transform Drift / World Alignment Issues The localToWorld transform provided by ARKit frequently seems to drift or become unstable across frames. This results in misaligned point clouds even when the device is moved slowly or kept relatively still. In some cases, a static surface scanned over a few seconds results in clearly misaligned fragments. This makes it difficult to accurately stitch a multi-frame point cloud. I have experimented with various lighting conditions and object textures, but the issue persists in all cases. At times, the relative error between frames reaches up to 20 cm, while in other instances the error is minimal; however, the drift gradually accumulates over time, leading to an overall enlargement of the reconstructed object. I have attached images of both cases here. Questions: Are there specific conditions under which ARKit’s world transform is expected to drift? Is there a way to detect or recover from this drift during runtime? Any best practices for maintaining consistent tracking during scanning or measurement sessions?
Replies
1
Boosts
0
Views
146
Activity
Jun ’25
Share Extensions embedded in visionOS apps
I'm trying to add a feature to my app to allow a user to import items from other apps, like Safari, via the share sheet. I've done this many times on iOS/iPadOS easily with a Share Extension. From what I can tell, Xcode tells me share extensions are not available on visionOS - though my experience on device tells me differently (It seems Reminders, Notes & more implement them somehow.) I was finally able to get it working on device only...but I can now no longer test in the simulator, and have not found a way to distribute this app. When attempting to run on the simulator, I get this issue: Please try again later. Appex bundle at /Users/jason/Library/Developer/CoreSimulator/Devices/09A70160-4F4F-4F5E-B679-F6F7D876D7EF/data/Library/Caches/com.apple.mobile.installd.staging/temp.6OAEZp/extracted/LaunchBar.app/PlugIns/LaunchBarShareExtension.appex with id co.swiftfox.LaunchBar.ShareExtension specifies a value (com.apple.share-services) for the NSExtensionPointIdentifier key in the NSExtension dictionary in its Info.plist that does not correspond to a known extension point. When trying to archive an upload to test flight, I get this similar error: Invalid Info.plist value. The value for the key 'DTPlatformName' in bundle LaunchBar.app/PlugIns/LaunchBarShareExtension.appex is invalid. (ID: 207610c7-b7e1-48be-959b-22a43cd32d16) The app is for visionOS only - which I'm thinking might be the problem? The share extension is "Designed For iPhone" and requires me to include iPhone as a run destination. In the worst case I can build an iPhone UI for the app but I'd rather not, as it is very specific to visionOS. Has anyone successfully launched a share extension on a visionOS only app? I have an iPad app with a share extension that shows up fine on visionOS, but the issue seems to be specifically with visionOS only apps.
Replies
1
Boosts
0
Views
122
Activity
Oct ’25
Sample code not working as expected: Implementing SharePlay for immersive spaces in visionOS
The following sample code project does not seem to work as expected: https://developer.apple.com/documentation/visionos/implementing-shareplay-for-immersive-spaces-in-visionos Have tried to get this project working with a client, but while we were able to see nearby users and make facetime calls, the color changing cube experience always remained a single color. Are there step-by-step instructions that Apple has used to verify this sample code so I can try to recreate this sample code's expected behavior for both nearby participants and those in a Facetime call?
Replies
1
Boosts
0
Views
265
Activity
Feb ’26