I found some snapshot API in developer documents, like blows:
RealityKit / Views and attachments / ARView / /snapshot(saveToHDR:completion:)
SceneKit / SCNView / snapshot()
Is there a similar API in visionOS?and if not, how can I implement snapshot for realityview and usdz?
Discuss spatial computing on Apple platforms and how to design and build an entirely new universe of apps and games for Apple Vision Pro.
Selecting any option will automatically load the page
Post
Replies
Boosts
Views
Activity
I’ve been having some issues removing anchors. I can add anchors with no issue. They will be there the next time I run the scene. I can also get updates when ARKit sends them. I can remove anchors, but not all the time. The method I’m using is to call removeAnchor() on the data provider.
worldTracking.removeAnchor(forID: uuid)
// Yes, I have also tried `removeAnchor(_ worldAnchor: WorldAnchor)`
This works if there are more than one anchor in a scene. When I’m down to one remaining anchor, I can remove it. It seems to succeed (does not raise an error) but the next time I run the scene the removed anchor is back. This only happens when there is only one remaining anchor.
do {
// This always run, but it doesn't seem to "save" the removal when there is only one anchor left.
try await worldTracking.removeAnchor(forID: uuid)
} catch {
// I have never seen this block fire!
print("Failed to remove world anchor \(uuid) with error: \(error).")
}
I posted a video on my website if you want to see it happening.
https://stepinto.vision/labs/lab-051-issues-with-world-tracking/
Here is the full code. Can you see if I’m doing something wrong? Is this a bug?
struct Lab051: View {
@State var session = ARKitSession()
@State var worldTracking = WorldTrackingProvider()
@State var worldAnchorEntities: [UUID: Entity] = [:]
@State var placement = Entity()
@State var subject : ModelEntity = {
let subject = ModelEntity(
mesh: .generateSphere(radius: 0.06),
materials: [SimpleMaterial(color: .stepRed, isMetallic: false)])
subject.setPosition([0, 0, 0], relativeTo: nil)
let collision = CollisionComponent(shapes: [.generateSphere(radius: 0.06)])
let input = InputTargetComponent()
subject.components.set([collision, input])
return subject
}()
var body: some View {
RealityView { content in
guard let scene = try? await Entity(named: "WorldTracking", in: realityKitContentBundle) else { return }
content.add(scene)
if let placementEntity = scene.findEntity(named: "PlacementPreview") {
placement = placementEntity
}
} update: { content in
for (_, entity) in worldAnchorEntities {
if !content.entities.contains(entity) {
content.add(entity)
}
}
}
.modifier(DragGestureImproved())
.gesture(tapGesture)
.task {
try! await setupAndRunWorldTracking()
}
}
var tapGesture: some Gesture {
TapGesture()
.targetedToAnyEntity()
.onEnded { value in
if value.entity.name == "PlacementPreview" {
// If we tapped the placement preview cube, create an anchor
Task {
let anchor = WorldAnchor(originFromAnchorTransform: value.entity.transformMatrix(relativeTo: nil))
try await worldTracking.addAnchor(anchor)
}
} else {
Task {
// Get the UUID we stored on the entity
let uuid = UUID(uuidString: value.entity.name) ?? UUID()
do {
try await worldTracking.removeAnchor(forID: uuid)
} catch {
print("Failed to remove world anchor \(uuid) with error: \(error).")
}
}
}
}
}
func setupAndRunWorldTracking() async throws {
if WorldTrackingProvider.isSupported {
do {
try await session.run([worldTracking])
for await update in worldTracking.anchorUpdates {
switch update.event {
case .added:
let subjectClone = subject.clone(recursive: true)
subjectClone.isEnabled = true
subjectClone.name = update.anchor.id.uuidString
subjectClone.transform = Transform(matrix: update.anchor.originFromAnchorTransform)
worldAnchorEntities[update.anchor.id] = subjectClone
print("🟢 Anchor added \(update.anchor.id)")
case .updated:
guard let entity = worldAnchorEntities[update.anchor.id] else {
print("No entity found to update for anchor \(update.anchor.id)")
return
}
entity.transform = Transform(matrix: update.anchor.originFromAnchorTransform)
print("🔵 Anchor updated \(update.anchor.id)")
case .removed:
worldAnchorEntities[update.anchor.id]?.removeFromParent()
worldAnchorEntities.removeValue(forKey: update.anchor.id)
print("🔴 Anchor removed \(update.anchor.id)")
if let remainingAnchors = await worldTracking.allAnchors {
print("Remaining Anchors: \(remainingAnchors.count)")
}
}
}
} catch {
print("ARKit session error \(error)")
}
}
}
}
For visionOS 2.0+, it has been announced the object tracking feature. Is there any support for PolySpatial in Unity or is it only available in Swift and Xcode?
When entering the mixed space, there is a window behind the robot, and I want this window to be moved to another position as code
Topic:
Spatial Computing
SubTopic:
ARKit
In Vision OS app, We have two types of windows:
Main App Window – This is the default window that launches when the app starts. It displays the video listings and other primary content.
Immersive Space Window – This opens only when a user starts streaming or playing a video.
Issue:
When entering the immersive space, the main app window remains visible in front of it unless manually closed. To avoid this, I currently close the main window when transitioning to immersive space and reopen it when exiting. However, this causes the app to restart instead of resuming from its previous state.
Desired Behavior:
I want the main app window to retain its state and seamlessly resume from where it was before entering immersive mode, rather than restarting.
Attempts & Challenges:
Tried managing opacity, visibility, and state preservation, but none worked as expected.
Couldn’t find a way to push the main window to the background while bringing the immersive space to the foreground.
Looking for a solution to keep the main window’s state intact while transitioning between immersive and normal modes.
I want to record animation with entity, then export it to .usd without using Reality Composer Pro, how to achieve that?
I want to display a model image in the windowGroup window. This image is not unique. To display the model image, how should I convert it into an image
As in the title: openImmersiveSpace works as expected. The ImmersiveSpace in the ID opens normally but the function never resolves a Result. Here is a workaround I used to make user it was on the UI thread and scenePhase was active:
`
@MainActor func openSpaceWithStateCheck() async {
if scenePhase == .active {
Task {
switch await openImmersiveSpace(id: "RoomCaptureInteraction") {
case .opened:
isCapturingImagery = true
break
case .error:
print("!! An error occurred when trying to open the immersive space captureRoomImagery")
case .userCancelled:
print("!! The user declined opening immersive space captureRoomImagery")
@unknown default:
print("!! unknown default result of opening space")
break
}
}
} else {
print("Scene not active, deferring immersive space opening")
}
}
I'm on visionOS 2.4 and SDK 2.2.
I have tried uninstalling the app and rebuilding. Tried simply opening an empty ImmersiveSpace.
The consistency of the ImmersiveSpace opening at least means I can work around it. Even dismissImmersiveSpace works normally and closes the immersive space. But a workaround seems hamfisted.
Hi, I am a new developer. I want to add articulated objects and deformable objects into my AR game. I haven't found any tutorial on this, I hope to interact with these objects. Please let me know if this is available in visionOS.
Hi everyone, I'm developing a MR Vision Pro app where I’d like to anchor virtual objects (such as UI elements) around the user's arm. However, I’ve noticed that Vision Pro seems to mask out the area where the user’s real arm is, hiding virtual content in that region so that you see your real arm.
Is there a way to render virtual elements on the user's arm—so that it looks like the object is placed directly on the arm despite the real-world passthrough? I was hoping there might be a way to adjust the depth or behavior of this masked-out region. Any insights or workarounds would be greatly appreciated! Thanks :)
Hi there,
I was looking to add a particle emitter to my augmented reality app I'm developing using RealityKit. I'm targeting iOS. I noticed in the documentation for the ParticleEmitterComponent that it looks like iOS 18.0+ is supported, but when I try to use the ParticleEmitterComponent in my code in XCode, I get an error that it isn't found. Furthermore, this StackOverflow post seems to indicate that particle systems are not available for iOS. Would it be possible to get clarification on this?
Hi team I don't know from this error has popped up any suggestions please
Thanks
Zippy Games
Hi, I am trying to create a simple effect to create feather edges on the image using Reality Composer Pro. Something like this:
As you can see it has softer edges on all sides that dissolves into transparency with the background.
this is what I have been able to achieve on my own.
I want to use the "feather" input node value (float) from 0.0 to 1.0 to increase or decrease the strength of the feather edges.
Topic:
Spatial Computing
SubTopic:
Reality Composer Pro
Tags:
Reality Composer
Shader Graph Editor
visionOS
This modifier in visionOS 2.5 works perfectly with LazyVgrid inside a Stack in ScrollView:
.hoverEffect { effect, isActive, _ in
effect.scaleEffect(isActive ? 1.1 : 1.0)
But the grid does not scroll in visionOS 26 beta 1 unless the scaleEffect is commented out.
FB17941468
I am developing an app in VisionPro using RealityKit and ARKit. I want my RealityKit entity looks more realistic. So it is important to render its shadow based on light in real world.
e.g. When I turn on the light in real world, the shadow of the entity will change. Can this effect be implemented in VisionPro?
It's much easier to use custom material to bridge metal shader onto Reality model than using LowLevelTexture does the same thing.
Why VisionOS doesn't support this material:
While using Screen Mirroring in developer mode within my immersive space, I noticed an alignment issue with the computer cursor (transparent circle). When I move it toward an attachment view, the cursor remains horizontal instead of aligning with the surface of the attachment view. It shows correctly on a 2D window only wrong on attachment view.
Is this behavior a bug, or could it be caused by a missing or incorrect configuration on the attachment view?
Want help, thanks.
Hello,
I checked following documentations.
Vision | Apple Developer Documentation
Discover Swift enhancements in the Vision framework - WWDC24 - Videos - Apple Developer
I saw Vision Framework is available on visionOS.
So I want to know that if it's possible using Vision Framework on visionOS for tracking human and animal body poses. Or are there some limits to use this on visionOS?
I'm playing about with the hand tracking systems in reality kit / Vision Pro
I thought it would be interesting if I could attach a virtual object to a hand when the hand is gripping (thought it would be fun to attach a basic cylinder to mimic a wand from Harry Potter)
I'm able to detect when the user is gripping but having trouble placing an object as though it's within the hand.
The simplest version of this is using an AnchorEntity pointing to the user's palm which kind of works, but quickly breaks the illusion when you rotate the wrist or hand.
It seems as though I will have to roll my own anchor entity using the various points of the user's hand and I thought calculating some median point between the thumb and little finger tips would be a good start but it's proven a little difficult as we need both rotation and position.
I'm already out of my depth with reality kit and matrices (and thanks to ChatGPT) I have some code, but as soon as I apply the position manually (as opposed to a hand anchor entity) it fails to render on the user's hand.
It feels like this should already have been something someone has looked in to, any ideas on what might be the issue here?
Note: HandTrackingSystem.handTracking is a HandTrackingProvider()
guard let anchors = HandTrackingSystem.handTracking.latestAnchors.leftHand else {
return
}
if
let thumb = anchors.handSkeleton?.joint(.thumbTip),
let little = anchors.handSkeleton?.joint(.littleFingerTip)
{
let thumbPos = simd_make_float3(thumb.anchorFromJointTransform.columns.3)
let littlePos = simd_make_float3(little.anchorFromJointTransform.columns.3)
let midPos = (thumbPos + littlePos) / 2
let direction = normalize(littlePos - thumbPos)
let rotation = simd_quatf(from: [0, 1, 0], to: direction)
wandEntity.transform.translation = midPos
wandEntity.transform.rotation = rotation
content.add(wandEntity)
}
Hello,
I'm working on a visionOS project that uses Reality Composer Pro, and we are managing our project files with Git.
We've noticed that simply opening and closing the Reality Composer Pro application consistently generates changes in the following files, even when no explicit modifications have been made by the developer:
{ProjectName}/Packages/RealityKitContent/Package.realitycomposerpro/PluginData/*******/ShaderGraphEditorPluginID/ShaderGraphEditorPluginID
{ProjectName}/Packages/RealityKitContent/Package.realitycomposerpro/WorkspaceData/SceneMetadataList.json
Could you please clarify the purpose of these files? Why do they appear as modified when no direct changes are made from our end?
More importantly, is it safe to add these files to our .gitignore to prevent them from being tracked by Git? We are concerned that ignoring these files might lead to unexpected issues or inconsistencies when other team members pull the latest changes, especially if these files contain critical project metadata or state that needs to be synchronized.
Any insights or recommended best practices for managing Reality Composer Pro projects with Git would be greatly appreciated.
Thank you for your time and assistance.
Topic:
Spatial Computing
SubTopic:
Reality Composer Pro
Tags:
Reality Composer
RealityKit
Reality Composer Pro