Spatial Systems Journal • Game Engines

Unreal Engine + OpenXR: A Portable Spatial API Architecture

A portable architecture for Unreal XR projects that separates OpenXR runtime concerns, Unreal actors, spatial backend state, coordinate conversion, and extensions.

Unreal + Openxr — neon spatial systems typography card

OpenXR gives Unreal projects a common path to multiple XR runtimes, while Unreal supplies rendering, world management, input, networking, and content tools. Portability still requires architecture. A project can adopt the OpenXR plugin and remain tightly coupled to one device through assumptions about extensions, input, tracking origin, performance, or store services. A spatial backend can create another coupling point if backend records are mapped directly onto transient actors.

A portable design separates runtime capabilities, engine presentation, and domain state. OpenXR-facing code reports what the active runtime can do. Unreal-facing code renders and interacts with actors and components. The spatial service owns durable entities, anchors, permissions, and shared world data. Adapters connect these layers with explicit transforms and capability checks.

Define the runtime boundary

Treat OpenXR as the device and runtime boundary, not the product data model. It supplies sessions, spaces, views, input paths, composition, and extensions. Wrap the project’s required capabilities behind interfaces such as pose provider, hand provider, anchor provider, passthrough provider, and capability registry. The rest of the game should depend on those interfaces rather than extension-specific calls.

This approach makes missing features visible. A runtime can report that hand tracking or a specific anchor mode is unavailable, allowing the experience to select a controller path or local-only placement. It also keeps extension code in focused modules that can be tested and replaced.

Keep durable entities outside Actor lifetime

An Unreal Actor may be streamed out, destroyed, replicated, or reconstructed. A durable spatial entity may remain valid for years. Store entity identity, revision, ownership, semantic type, and canonical pose in a domain subsystem or data layer. Spawn Actors as presentations when the relevant world partition, level, or user scope becomes active.

Attach a lightweight component with the durable entity ID and presenter state. Do not use Actor names or memory addresses as backend identifiers. When an Actor is destroyed, unregister the presentation without deleting the domain record unless an explicit product action requests deletion.

Coordinate conversion and georeferencing

Unreal worlds, tracking spaces, and geographic coordinates serve different purposes. Use a dedicated transform chain from backend frame to application world to tracking space. Document units, axis orientation, origin, and precision. Large geographic worlds may require georeferencing or origin rebasing, while local XR interaction needs stable, high-precision transforms near the user.

Keep canonical geographic or facility coordinates independent from world-origin shifts. When the engine rebases or the XR origin recenters, update the adapter transform. Validate with known landmarks and round-trip tests. Small orientation errors can become obvious at scale, so include quaternions and axis conversions in automated fixtures.

Use semantic commands for interaction

Grab, place, annotate, approve, and teleport are product actions, not just input events. Unreal input and interaction code should emit semantic commands with entity IDs, proposed state, frame, and base revision. The domain service validates constraints and authority, then returns accepted state. Local prediction can keep interaction responsive while awaiting confirmation.

This is especially important in multi-user or regulated workflows. A continuous stream of Actor transforms does not explain why an object moved or whether the move was permitted. A command can be audited, undone, rejected with a reason, and replayed across clients.

Capabilities and extensions

OpenXR extensions allow runtimes to expose additional features, but extension presence should not be scattered through gameplay Blueprints. Build a capability registry at startup and publish high-level flags and service implementations. Product features can require, prefer, or optionally use a capability. The UI can explain when a feature is unavailable on the current system.

Avoid making a vendor extension the only storage format for durable data. If an extension returns a runtime handle, map it to a product-level anchor or entity record. Keep enough neutral metadata to migrate, recreate, or represent the content on clients that do not support that extension.

Networking and shared spatial state

Unreal replication is effective for session state, but it may not be the sole durable backend. Decide which authority owns each value. The server may own gameplay state, a spatial service may own persistent world entities, and the local runtime owns current tracking. Define synchronization points instead of allowing circular updates among all three.

For shared XR, synchronize domain poses in a shared application frame and let each client transform them into local tracking space. Include timestamps and revisions. Interpolate presentation movement, but do not overwrite durable state with every rendered frame. Conflict policy should be explicit for simultaneous edits.

World partition, streaming, and level of detail

Large spatial experiences need scope. Query entities by cell, region, route, or task, then spawn only the representations needed for the active world. Align backend partitioning with engine streaming where practical, but keep the contract independent enough to change one side. A region response can include lightweight bounds and asset references before full models load.

Use hierarchical level of detail, instancing, and asset budgets appropriate to the target headset. Runtime portability includes performance portability: a scene that runs on a high-end PC runtime may overwhelm a standalone device. Capability profiles can choose assets, effects, and density without changing domain identity.

Testing the abstraction

Automated tests should instantiate the domain and coordinate layers without an XR runtime. Verify actor registration, command generation, revision conflicts, frame conversion, streaming scope, and fallback capability selection. Mock runtime services to simulate missing extensions, tracking loss, recentering, and anchor failure.

Device testing should then focus on runtime behavior, input, comfort, and frame timing. Maintain a matrix of runtimes and hardware, and log capability results at startup. A portable architecture does not guarantee identical behavior; it gives the product a controlled way to adapt when behavior differs.

Conclusion

OpenXR reduces runtime fragmentation, but project portability comes from boundaries. Keep OpenXR capabilities behind services, Unreal Actors as presentations, and durable spatial entities in a domain layer. Explicit coordinate transforms, semantic commands, neutral anchor records, and capability-driven fallbacks let one product support multiple runtimes without hiding real differences between them.

Choose a clean Blueprint and C++ boundary

Blueprints are effective for experience logic, visual feedback, and rapid iteration. C++ modules are well suited to transport, coordinate math, caching, runtime services, and performance-sensitive integration. Expose stable, high-level functions and events to Blueprints rather than raw HTTP responses or extension handles. This keeps visual scripts focused on product behavior.

Use data assets or typed structures for entity presentation rules, capability profiles, and asset mappings. Avoid hard-coded runtime checks in many Blueprints. A central subsystem can publish events such as entity added, entity changed, anchor resolved, capability changed, or command rejected. Blueprints subscribe and update their presenters.

Error handling should cross the boundary in a structured form. A Blueprint needs a user-facing category and recovery action, while logs retain detailed diagnostics. Do not convert every failure to a boolean; distinguish offline, unauthorized, stale revision, unsupported capability, invalid pose, and asset unavailable.

Portable release checklist

Portability is achieved when a missing capability produces a defined product state rather than a crash or a hidden code branch. The architecture should make differences explicit while preserving durable projects and entity identity across runtimes.

  • Wrap runtime and extension capabilities behind Unreal interfaces or subsystems.
  • Keep persistent entity records independent from Actor and level lifetime.
  • Use one transform chain for backend, georeferenced world, and tracking spaces.
  • Emit semantic commands with revisions instead of treating replicated transforms as history.
  • Align backend region queries with world streaming without coupling the schemas.
  • Profile asset density and effects for each device capability tier.
  • Mock missing extensions, tracking loss, anchor failure, and recentering in tests.
  • Log the observed runtime capability matrix for support and diagnostics.