Hello! I'm currently porting a videogame console emulator to iOS and I'm trying to make the renderer (tested on MacOS) work on iOS as well.
The emulator core is written in C++ and uses metal-cpp for rendering, whereas the iOS frontend is written in Swift with SwiftUI. I have an Objective-C++ bridging header for bridging the Swift and C++ sides.
On the Swift side, I create an MTKView. Inside the MTKView delegate, I run the emulator for 1 video frame and pass it the view's backing layer for it to render the final output image with. The emulator runs and returns, but when it returns I get a crash in Swift land (callstack attached below), inside objc_release, which indicates I'm doing something wrong with memory management.
My bridging interface (ios_driver.h):
#pragma once
#include <Foundation/Foundation.h>
#include <QuartzCore/QuartzCore.h>
void iosCreateEmulator();
void iosRunFrame(CAMetalLayer* layer);
Bridge implementation (ios_driver.mm):
#import <Foundation/Foundation.h>
extern "C" {
#include "ios_driver.h"
}
<...>
#define IOS_EXPORT extern "C" __attribute__((visibility("default")))
std::unique_ptr<Emulator> emulator = nullptr;
IOS_EXPORT void iosCreateEmulator() { ... }
// Runs 1 video frame of the emulator and
IOS_EXPORT void iosRunFrame(CAMetalLayer* layer) {
void* layerBridged = (__bridge void*)layer;
// Pass the CAMetalLayer to the emulator
emulator->getRenderer()->setMTKLayer(layerBridged);
// Runs the emulator for 1 frame and renders the output image using our layer
emulator->runFrame();
}
My MTKView delegate:
class Renderer: NSObject, MTKViewDelegate {
var parent: ContentView
var device: MTLDevice!
init(_ parent: ContentView) {
self.parent = parent
if let device = MTLCreateSystemDefaultDevice() {
self.device = device
}
super.init()
}
func mtkView(_ view: MTKView, drawableSizeWillChange size: CGSize) {}
func draw(in view: MTKView) {
var metalLayer = view.layer as! CAMetalLayer
// Run the emulator for 1 frame & display the output image
iosRunFrame(metalLayer)
}
}
Finally, the emulator's render function that interacts with the layer:
void RendererMTL::setMTKLayer(void* layer) {
metalLayer = (CA::MetalLayer*)layer;
}
void RendererMTL::display() {
CA::MetalDrawable* drawable = metalLayer->nextDrawable();
if (!drawable) {
return;
}
MTL::Texture* texture = drawable->texture();
<rest of rendering follows here using the drawable & its texture>
}
This is the Swift callstack at the time of the crash:
To my understanding, I shouldn't be violating ARC rules as my bridging header uses CAMetalLayer* instead of void* and Swift will automatically account for ARC when passing CoreFoundation objects to Objective-C. However I don't have any other idea as to what might be causing this. I've been trying to debug this code for a couple of days without much success.
If you need more info, the emulator code is also on Github
Metal renderer: https://github.com/wheremyfoodat/Panda3DS/blob/ios/src/core/renderer_mtl/renderer_mtl.cpp#L58-L68
Bridge implementation: https://github.com/wheremyfoodat/Panda3DS/blob/ios/src/ios_driver.mm
Bridging header: https://github.com/wheremyfoodat/Panda3DS/blob/ios/include/ios_driver.h
Any help is more than appreciated. Thank you for your time in advance.
Delve into the world of graphics and game development. Discuss creating stunning visuals, optimizing game mechanics, and share resources for game developers.
Selecting any option will automatically load the page
Post
Replies
Boosts
Views
Activity
Since macOS 15.3.2, we have observed that when another window is moved near the App Store's install button, the button disappears.
We have attached a related video in the Feedback submission here https://feedbackassistant.apple.com/feedback/20444423
Our application overlays a transparent, watermark-window on top of the system window, which causes the install button in the App Store to be hidden when a user attempts to install an application.Could you advise on how to avoid this issue?
I have been tasked with creating content for the Apple Vision Pro. Just the 3D content and animation, not the programming end of things.
I can't seem to get any kind of mesh deformation animation to import into Reality Composer Pro. By that I mean bones/skin, or even point cache.
I work on PC, and my main software is 3DS Max, but I'm borrowing an iMac for this job, and was instructed to use RCP on it for testing before handing things off to the programmer. My files open and play fine in other USD programs, like Omniverse, or USD View, just not Reality Composer Pro.
I've seen the dinosaur demo in AVP, so I know mesh deformation is possible. If there are other essential tools that might make this possible, I have not been made aware of them.
I am experimenting with bouncing things off of Blender, in case that exports better, but not really having luck there either -though my results are different.
Thanks.
Topic:
Graphics & Games
SubTopic:
RealityKit
Hi everyone,
I’m currently learning about ParticleEmitterComponentParticleEmitterComponent and exploring the sample app provided in the Simulating particles in your visionOS app documentation.
In the sample app, when I set the EmitterPreset to fireworks from the settings panel on the left side of the window and choose SystemImage, I noticed two issues:
The image applied to mainEmitter appears clipped or cropped.
The image on spawnedEmitter does not update to the selected SystemImage.
What I want to achieve:
Apply the same SystemImage to both mainEmittermainEmitter and spawnedEmitterspawnedEmitter so that it displays correctly without clipping.
Remove the animation that changes the size of spawnedEmitterspawnedEmitter over time and keep it at a constant size.
Could someone explain which properties should be adjusted to achieve this behavior? Any guidance or examples would be greatly appreciated.
Thanks in advance!
Post can be removed.
RealityKit spatial audio crackles and pops on iOS 26.0 beta 5.
It works correctly on iOS 18.6 and visionOS 26.0 beta 5.
The APIs used are AudioPlaybackController, Entity.prepareAudio, Entity.play
Videos of the expected and observed behavior are attached to the feedback FB19423059.
The audio should be a consistent, repeating sound, but it seems oddly abbreviated and the volume varies unexpectedly.
Thank you for investigating this issue.
Hello all... is there a way to close a contour if you have found say two points on each side top "extension"? see image attached. So in end desire a trapezoid type shape. Code example would be very appreciated. thank you :) Think I have it as a CGPath. So a way to edit a CGPath, or close the top from a top left to a top right point?
In the SceneKit framework, I want to render a point cloud and need to set the minimumPointScreenSpaceRadius property of the SCNGeometryElement class, but it doesn't work. I searched for related issues on the Internet, but didn't get a final solution.
Here is my code:
- (SCNGeometry *)createGeometryWithVector3Data:(NSData *)vData colorData: (NSData * _Nullable)cData{
int stride = sizeof(SCNVector3);
long count = vData.length / stride;
SCNGeometrySource *dataSource = [SCNGeometrySource geometrySourceWithData:vData
semantic:SCNGeometrySourceSemanticVertex
vectorCount:count
floatComponents:YES
componentsPerVector:3
bytesPerComponent:sizeof(float)
dataOffset:0
dataStride:stride];
SCNGeometryElement *element = [SCNGeometryElement geometryElementWithData:nil
primitiveType:SCNGeometryPrimitiveTypePoint
primitiveCount:count
bytesPerIndex:sizeof(int)];
element.minimumPointScreenSpaceRadius = 1.0; // not work
element.maximumPointScreenSpaceRadius = 20.0;
SCNGeometry *pointCloudGeometry;
SCNGeometrySource *colorSource;
if (cData && cData.length != 0) {
colorSource = [SCNGeometrySource geometrySourceWithData:cData
semantic:SCNGeometrySourceSemanticColor
vectorCount:count
floatComponents:YES
componentsPerVector:3
bytesPerComponent:sizeof(float)
dataOffset:0
dataStride:stride];
pointCloudGeometry = [SCNGeometry geometryWithSources:@[dataSource, colorSource] elements:@[element]];
} else {
pointCloudGeometry = [SCNGeometry geometryWithSources:@[dataSource] elements:@[element]];
}
return pointCloudGeometry;
}
I have a scene built up in RealityComposerPro, in which I've added a ParticleEmitter with isEmitting set to False and 'Loop' set to True.
In my app, when I toggle isEmitting to True there can be a delay of a few seconds before the ParticleEmitter starts.
However, if I programatically add the emitter in code at that point, it starts immediately.
To be clear, I'm seeing this on the VisionOS simulator - I don't have access to a device at this time.
Am I misunderstanding how to control the ParticleEmitter when I need precise control on when it starts.
Hello there,
I'm having trouble matching what I see in the scenekit editor and the output of the resulting scene in a scnview.
For a glitter effect I have set a high value on the diffuse intensity which looks fine in the editor but when running the game the colors are much darker. To see if the intensity value is merely capped I have set the same multiplier on the hat below - but it is blown out which looks to me like there is some grading going on
I have tried to switch on hdr rendering but that didn't make a difference.
I tried disabling linear rendering and that simply made everything darker still - which I expect.
Does someone have an idea what else this could be? What rendering is the scenekit editor using and how can I match it?
Interestingly when I take a screenshot of the editor window for this post, the image is also blown out... what is going on? :)
Thanks so much for any pointers,
Seb
I am working on a custom resolve tile shader for a client. I see a big difference in performance depending on where we write to:
1- the resolve texture of the color attachment
2- a rw tile shader texture set via [renderEncoder setTileTexture: myResolvedTexture]
Option 2 is more than twice as slow than option 1.
Our compute shader writes to 4 UAVs so just using the resolve texture entry is not possible.
Why such a difference as there is no more data being written? Can option 2 be as fast as option 1?
I can demonstrate the issue in a modified version of the Multisample code sample.
My app is being rejected and all I'm being told is that it is spam.
I've tried improving various aspects of the game, but I just receive the same copy and paste rejection message each time.
I have no idea if I'm moving in the right direction or what part of my game needs to be changed or improved. Is there a game quality benchmark document or some kind of resource I can use to better understand why my game is being rejected and how to bring it to a level that meets apple's standards.
I'm implementing optimized matmul on metal: https://github.com/crynux-ai/metal-matmul/blob/main/metal/1_shared_mem.metal
I notice that performance is significantly different with different threadgroup memory set in
[computeEncoder setThreadgroupMemoryLength]
All other lines are exactly same, the only difference is this parameter.
Matmul performance is roughly 250 GFLops if I set 32768 (max bytes allowed on this M1 Max),
but 400 GFLops if I set 8192.
Why does this happen? How can I optimize it?
Topic:
Graphics & Games
SubTopic:
Metal
Hello !
We are working on a real-time 2-player online game targeting multiple Apple devices.
The following issue only occurs on tvOS:
When selecting matchmaking to connect with another random player, the native Game Center interface opens and begins the matchmaking process.
Almost immediately after clicking "start", the following log appears in the console, and the matchmaking screen remains indefinitely without completing:
Timeout while starting matching with request: <GKMatchRequestInternal 0x30d62f690> {
defaultNumberOfPlayers : 0
isLateJoin : 0
localPlayerID : U:bea182d69b85f0839e3958742fbc4609
matchType : 0
maxPlayers : 2
minPlayers : 2
playerAttributes : 4294967295
playerGroup : 1
preloadedMatch : 0
recipientPlayerIDs : <__NSArrayM 0x3034ed5c0> {}
recipients : <__NSArrayM 0x3034ee280> {}
restrictToAutomatch : 0
version : 1
archivedSharePlayInviteeTokensFromProgrammaticInvite, inviteMessage, localizableInviteMessage, messagesBasedRecipients, properties, queueName, recipientProperties, rid, sessionToken : (null)
} . Error: (null)
However, as shown in the code snippet below, the task does not complete when the log appears. But when we manually cancel the matchmaking process, the "User cancel" log is correctly triggered.
var gkMatchRequest = GKMatchRequest.Init();
gkMatchRequest.MinPlayers = 2;
gkMatchRequest.MaxPlayers = 2;
var matchRequestTask = GKMatchmakerViewController.Request(gkMatchRequest);
matchRequestTask.ContinueWith(t => { Debug.LogException(t.Exception); }, TaskContinuationOptions.OnlyOnFaulted);
matchRequestTask.ContinueWith(t => { Debug.Log("User cancel"); }, TaskContinuationOptions.OnlyOnCanceled);
matchRequestTask.ContinueWith(t => { Debug.Log("Success"); }, TaskContinuationOptions.OnlyOnRanToCompletion);
We have tested this on multiple Apple TV and network types (Wi-Fi, 5G, Ethernet), but we consistently encounter this bug along with the same log message.
Could you please help us understand or resolve this issue?
Thank you.
Hello,
We are working on a real-time 2-player online game targeting multiple Apple devices. The following issue only occurs on tvOS:
When selecting matchmaking to connect with another player, the native Game Center interface opens and begins the matchmaking process.
Almost immediately, the following log appears in the console, and the matchmaking screen remains indefinitely without completing:
Timeout while starting matching with request: <GKMatchRequestInternal 0x30d62f690> {
defaultNumberOfPlayers : 0
isLateJoin : 0
localPlayerID : U:bea182d69b85f0839e3958742fbc4609
matchType : 0
maxPlayers : 2
minPlayers : 2
playerAttributes : 4294967295
playerGroup : 1
preloadedMatch : 0
recipientPlayerIDs : <__NSArrayM 0x3034ed5c0> {}
recipients : <__NSArrayM 0x3034ee280> {}
restrictToAutomatch : 0
version : 1
archivedSharePlayInviteeTokensFromProgrammaticInvite, inviteMessage, localizableInviteMessage, messagesBasedRecipients, properties, queueName, recipientProperties, rid, sessionToken : (null)
} . Error: (null)
However, the task does not complete when the log appears (our Debug.Log are nerver called).
But if we manually cancel the matchmaking process, the "User cancel" log is correctly triggered.
Here is a code snippet for the request :
var gkMatchRequest = GKMatchRequest.Init();
gkMatchRequest.MinPlayers = 2;
gkMatchRequest.MaxPlayers = 2;
var matchRequestTask = GKMatchmakerViewController.Request(gkMatchRequest);
matchRequestTask.ContinueWith(t => { Debug.LogException(t.Exception); }, TaskContinuationOptions.OnlyOnFaulted);
matchRequestTask.ContinueWith(t => { Debug.LogInfo("User cancel"); }, TaskContinuationOptions.OnlyOnCanceled);
matchRequestTask.ContinueWith(t => { Debug.LogInfo("Success"); }, TaskContinuationOptions.OnlyOnRanToCompletion);
We have tested this on multiple devices and network types (Wi-Fi, 5G, Ethernet), but we consistently encounter this bug along with the same log message.
Could you please help us understand or resolve this issue?
Thank you.
I am currently developing a mobile and server-side application using the new ObjectCaptureSession on iOS and PhotogrammetrySession on MacOS.
I have two questions regarding the newly updated APIs.
From WWDC23 session: "Meet Object Capture for iOS", I know that the Object Capture API uses Point Cloud data captured from iPhone LiDAR sensor. I want to know how to use the Point Cloud data captured on iPhone ObjectCaptureSession and use it to create 3D models on PhotogrammetrySession on MacOS.
From the example code from WWDC21, I know that the PhotogrammetrySession utilizes depth map from captured photo images by embedding it into the HEIC image and use those data to create a 3D asset on PhotogrammetrySession on MacOS. I would like to know if Point Cloud data is also embedded into the image to be used during 3D reconstruction and if not, how else the Point Cloud data is inserted to be used during reconstruction.
Another question is, I know that Point Cloud data is returned as a result from request to the PhtogrammetrySession.Request. I would like to know if this PointCloud data is the same set of data captured during ObjectCaptureSession from WWDC23 that is used to create ObjectCapturePointCloudView.
Thank you to everyone for the help in advance. It's a real pleasure to be developing with all the updates to RealityKit and the Object Capture API.
Hi, I’m learning MAUI and was trying to use VNDocumentCameraViewController provided by Vision Kit to scan documents and it is working fine but I realized that I was not able to customize some of the options provided by default like, disabling the auto scan option. Is there any way to disable the auto scan option or are there any other alternatives with the same founctionalities as VNDocumentCameraViewController that are more customizable?
Hi, is there any way to use front camera to do the motion capture?
I want recognize if the user raised there hands up with the front camera on iPhone.
I was able to do it with the back camera, not the front.
Also, if there is any sample code, or document, I would be super happy.
Waiting for your reply!!
Topic:
Graphics & Games
SubTopic:
RealityKit
Tags:
Swift Student Challenge
Swift
ARKit
Swift Playground
Hello everyone,
I must have missed something but why isn't there a depthAttachmentPixelFormat to the new Metal 4 MTL4RenderPipelineDescriptor, unlike the old MTLRenderPipelineDescriptor?
So how do you set the depth pixel format?
Thanks in advance!
I would love to use Background GPU Access to do some video processing in the background.
However the documentation of BGContinuedProcessingTaskRequest.Resources.gpu clearly states:
Not all devices support background GPU use. For more information, see Performing long-running tasks on iOS and iPadOS.
Is there a list available of currently released devices that do (or don't) support GPU background usage? That would help to understand what part of our user base can use this feature. (And what hardware we need to test this on as developers.)
For example it seems that it isn't supported on an iPad Pro M1 with the current iOS 26 beta. The simulators also seem to not support the background GPU resource. So would be great to understand what hardware is capable of using this feature!