Spatial Computing on visionOS: Windows, Volumes, Immersive Spaces, and World Anchors

Choose presentation by task

Windows are appropriate for text, lists, controls, media, and workflows that benefit from conventional layout. Volumes provide a bounded region for 3D content that users can place alongside other apps. Immersive spaces can surround the user or blend digital content with the environment. The most immersive option is not automatically the most effective.

Design a progression. A user might browse projects in a window, inspect a model in a volume, and enter an immersive space only for detailed review. Preserve selection, camera intent, and document state across transitions. The spatial API should expose the same project and entity IDs regardless of presentation.

Keep domain entities separate from render entities

A domain entity represents the product’s durable object: a machine, annotation, lesson step, room asset, or shared marker. A render entity is the RealityKit or UI object used to display it in the current scene. Do not use runtime object identity as the database key. Map stable domain IDs to render entities through a scene adapter.

This pattern allows an object to appear as a row in a window, a miniature in a volume, and a full-scale model in an immersive space. It also supports asset replacement, level-of-detail changes, and multi-user state without rewriting business logic. The domain layer owns state; the presentation layer owns transforms, materials, animation, and interaction affordances.

Model anchors as resolvable references

Anchors connect digital content to a coordinate space. A product-level anchor record should include a stable ID, type, owner, content association, creation context, and any data needed to resolve it in a session. The platform anchor or transform used at runtime is an implementation detail that may not be available until permissions and tracking are ready.

Handle unresolved and relocalizing states explicitly. Content can appear in a neutral browser or staging area while the app attempts to place it. If an anchor cannot be resolved, show that state rather than silently placing the object at the origin. When an anchor is updated, define how existing sessions reconcile the change.

Input is spatial intent, not a pointer clone

Eyes, hands, indirect gestures, keyboards, and accessibility technologies can all contribute to interaction. Product code should describe intent—select, activate, move, resize, inspect—rather than bind core behavior to one gesture. The presentation layer translates available input into these actions and supplies appropriate feedback.

Targets need comfortable size, spacing, and depth. Avoid requiring prolonged precision at a distance. Moving 3D objects should include constraints, snapping, and an undo path. For critical actions, combine clear visual state with confirmation rather than relying on a subtle gesture event.

Scene state and lifecycle

Spatial scenes may open, close, move, or coexist. Treat each scene as a view over shared application state rather than a separate copy of the data. A central store can own projects, sessions, synchronization, and permissions, while scene-specific controllers manage visible entities and local selection. Persist user-authored changes at meaningful checkpoints instead of every render update.

Plan for interruption. Tracking quality can change, the immersive space can close, and assets can fail to load. The app should preserve work and provide a clear path back to a window. Lifecycle events belong in architecture diagrams and tests, not only in UI code.

Assets, scale, and performance

Every 3D asset should have known units, orientation, pivot, bounding volume, and material assumptions. Validate these during ingestion so runtime code does not contain per-model correction tables. Generate levels of detail and lightweight previews where scenes can contain many objects. Stream or lazily load assets based on visibility and task.

Performance includes more than frame rate. Asset decode time, memory pressure, network delay, shader complexity, animation, physics, and scene reconstruction all compete for resources. Instrument transitions into volumes and immersive spaces so teams can see where users encounter blank or unstable moments.

Shared and multi-device state

A shared spatial experience requires agreement on entity IDs, authority, time, and coordinate alignment. Synchronize semantic actions such as “move entity 42 to approved pose” rather than continuous raw transforms when possible. Define who can edit, how conflicts are resolved, and which actions are auditable. Local interpolation can make motion smooth without making every client authoritative.

Cross-device support is easier when the backend stores neutral scene data and asset references. A phone may show a map or camera AR view, while visionOS presents the same entities in a volume. Each client resolves product coordinates into its runtime space and reports capability limits.

Comfort, accessibility, and restraint

Comfort is an architectural requirement. Avoid unnecessary motion, sudden scale changes, content that invades personal space, and interactions that require awkward posture. Provide seated and standing use where the task allows. Respect reduced-motion preferences and ensure important information is not communicated only through depth, color, or audio.

Restraint improves spatial design. Keep controls close to the content they affect, but do not cover the object with panels. Use familiar window patterns for dense workflows. Immersion should help users understand or accomplish something that is genuinely difficult on a flat display.

Conclusion

A durable visionOS architecture separates domain state from presentation, treats anchors as resolvable references, and models interaction as intent. Windows, volumes, and immersive spaces can then serve different phases of the same task without fragmenting the data model. The best spatial API gives the platform enough information to render naturally while keeping the product portable beyond one device.

Define a scene synchronization API

A scene synchronization service can expose projects, entities, asset versions, anchor references, and semantic commands. A client requests a project snapshot, subscribes to changes after a revision, and resolves only the entities relevant to its current window, volume, or immersive space. The service does not need to know the exact RealityKit hierarchy; it needs to know durable identity and product intent.

Commands might include create annotation, update approved pose, attach asset, change workflow status, or remove shared content. Include the base revision so conflicts are detected. Continuous local motion can remain local until the user commits an action. For shared experiences, clients can broadcast ephemeral presence separately from durable project edits.

Responses should provide capability hints without dictating presentation. An entity can declare that it has a 3D asset, a text summary, a preferred scale range, and an anchor requirement. A visionOS client chooses a volume or immersive representation, while a phone or web client uses the same record in another form.

Delivery checklist for spatial apps

The architecture should make a graceful downgrade normal. A project remains understandable in a window even when an immersive space is unavailable, and a 3D entity remains identifiable while its model is still loading.

• Give every domain entity a stable ID independent of RealityKit objects.
• Validate asset units, pivots, bounds, materials, and level of detail before release.
• Preserve task state when moving among windows, volumes, and immersive spaces.
• Handle unresolved anchors and tracking changes as visible states.
• Use intent-based actions so gaze, hands, keyboards, and accessibility input can coexist.
• Instrument asset loading, memory, scene transitions, and frame timing on real hardware.
• Offer a non-immersive route to essential information and settings.
• Test comfort, reach, readability, reduced motion, and seated use.