Script Graph runtime JS API

Question

Created 4d

Replies 2

Boosts 0

Participants 2

This post is from the WWDC26 RealityKit & Reality Composer Pro Q&A.

RCP 3 Script Graphs in 27.0 appear to be JavaScript-backed and hot-reloadable. SceneKit historically exposed its scene graph to JavaScript via JavaScriptCore, so Apple has bridged a 3D framework to JS before. Questions about the runtime surface:

Is there a public runtime API to evaluate JavaScript against a RealityKit scene, or to drive Script Graph behavior programmatically at runtime — or is the JS layer private SPI used only by RCP-authored, baked graphs?
If there's a runtime JS layer, what object model is exposed — which entity/component types are bridged, and is that surface documented or intended to become public?
What's the hot-reload mechanism, and is it reachable outside RCP's Preview on Device?
Can a script graph be constructed or parameterized at runtime from a string, or is authoring strictly RCP-side with baked output loaded as an asset?

We ship JavaScript scripting via JavaScriptCore ourselves, so a public RealityKit-to-JS binding would be a natural way to extend it — trying to understand what's public vs. private and the intended direction.

Answered by Vision Pro Engineer in 891515022

Hi @arthurfromberlin! You can check the new RealityKitScripting Swift Package we just released.

It will allow you to evaluate your JS against RealityKit. This is a public Swift Package and you can load your script at runtime from a string.

Something similar to:

import RealityKit

struct ContentView: View {
    let script = """
        const RealityKit = require("RealityKit");
        const Math3D = require("Math3D");

        this.update = function(deltaTime) {
            const rotation = new Math3D.Quaternion(deltaTime * Math.PI, new Math3D.Vector3(0, 1, 0));
            this.entity.orientation.multiply(rotation);
        };

        this.didAdd = function() {
            console.log(`Script attached to: "${this.entity.name}"`);
        };
    """

    var body: some View {
        RealityView { content in
            let entity = ModelEntity(mesh: .generateBox(size: 0.2), materials: [SimpleMaterial(color: .blue, isMetallic: false)])
            entity.name = "Box"
            entity.components.set(ScriptingComponent(source: script))
            content.add(entity)
        }
        .scriptingSystem()
    }
}

Answer 1

Vision Pro Engineer OP

Apple

4d

Recommended

Hi @arthurfromberlin! You can check the new RealityKitScripting Swift Package we just released.

It will allow you to evaluate your JS against RealityKit. This is a public Swift Package and you can load your script at runtime from a string.

Something similar to:

import RealityKit

struct ContentView: View {
    let script = """
        const RealityKit = require("RealityKit");
        const Math3D = require("Math3D");

        this.update = function(deltaTime) {
            const rotation = new Math3D.Quaternion(deltaTime * Math.PI, new Math3D.Vector3(0, 1, 0));
            this.entity.orientation.multiply(rotation);
        };

        this.didAdd = function() {
            console.log(`Script attached to: "${this.entity.name}"`);
        };
    """

    var body: some View {
        RealityView { content in
            let entity = ModelEntity(mesh: .generateBox(size: 0.2), materials: [SimpleMaterial(color: .blue, isMetallic: false)])
            entity.name = "Box"
            entity.components.set(ScriptingComponent(source: script))
            content.add(entity)
        }
        .scriptingSystem()
    }
}

Answer 2

arthurfromberlin OP

4d

@Vision Pro Engineer This is great, thank you — exactly what we were hoping existed. We already ship our own JavaScriptCore-based scripting layer, and I've been digging through the package to understand how it'd fit alongside ours.

Two questions the repo doesn't quite answer, both about deployment rather than API:

Runtime location / .reality files. When a .reality file authored with scripts is opened in QuickLook or a system RealityView, what provides the runtime? Is RealityKitScripting OS-resident on 27+ (so system viewers execute embedded ScriptingComponents without each app or file bundling it), or must the consuming app link the package? I assume the .reality carries only the ScriptingComponent source string and the runtime must be present in the opening process — just want to confirm.
Bundle size / App Clips. The GitHub release ships as an embedded binary xcframework. At GA, will it stay an app-embedded dependency, or become an OS framework apps link against? This matters for us because we target App Clips (size-constrained) and earlier OS versions, so we need to know whether adopting it means shipping the runtime or linking a system one.

And one architecture question: is this the same runtime that RCP 3 Script Graphs execute on — i.e. does a type exposed via @Scriptable / TypeSchema become available to both text scripts and the visual graph?

Thanks again — genuinely useful!