AI + LLM
Ground AI in Measurable Space
Use models to interpret scenes and user intent while deterministic spatial services measure, transform, validate, and execute.
From image understanding to spatial action
Vision models can identify likely objects and describe a scene. Language models can translate a user request into a plan. Neither should silently invent metric geometry. An AI spatial API connects those capabilities to calibrated data, explicit frames, deterministic queries, and constrained tools.
A useful workflow begins with evidence: image, depth, pose, map, or engine state. Perception services produce candidate entities and confidence. A scene service normalizes identity and relationships. Geometry services answer distance, containment, visibility, route, and placement questions. The language layer orchestrates these tools and explains the result with provenance.
Scene entities and relations
Give the model a compact scene representation instead of an unbounded dump of pixels or vertices. Each entity can include a stable ID, type, attributes, pose or image region, bounding volume, confidence, and references to heavier assets. Relations such as inside, on, behind, connected, visible, and reachable should be produced by named services or rules.
Keep observation and interpretation separate. “Pixels in frame 81” is evidence. “Candidate valve” is a model output. “Asset V-204” is an identity resolution. “Accessible from route R-9” is a domain result. This lineage lets the agent express uncertainty and lets developers diagnose which layer failed.
Small tools beat one magical endpoint
Agent tools should perform bounded operations: resolve entity, describe view, transform pose, entities within radius, raycast scene, calculate route, create anchor, or retrieve asset. Inputs should use typed IDs, units, frames, and constrained options. Outputs should be structured and include confidence or failure reasons.
A single “understand this space” endpoint is difficult to secure and evaluate. Small tools can be authorized independently, tested with fixtures, and composed into longer workflows. Model output remains untrusted input; the execution service validates every call.
Ground nouns, then relations
Resolve a phrase such as “the blue crate by the door” to one or more candidate entity IDs before performing an action. If multiple candidates fit, return them with distinguishing properties and ask for clarification. Once the noun is grounded, deterministic services evaluate nearest, behind, reachable, fits, or safe-to-place.
After each tool call, return concrete state to the language layer. The final response can explain that a distance is approximate, a route is blocked, or an anchor could not be resolved. This loop is more reliable than asking the model to infer everything in one pass.
Context, latency, and privacy
Retrieve the smallest spatial context that supports the task. Filter by current view, region, role, and intent. Summarize distant entities and provide handles for detail. Keep high-frequency perception and pose local when possible, and send compact detections or crops rather than continuous raw video.
Camera and environment data can expose documents, screens, bystanders, and private spaces. Separate transient inference buffers from durable records, redact before transfer, and avoid logging raw payloads. Tool authorization should constrain which entities, regions, and actions the agent may access.
Evaluation for spatial agents
Evaluate geometry and behavior, not only fluent language. Use scenes with known coordinates to test entity resolution, transforms, relation classification, route validity, tool selection, and refusal when evidence is insufficient. Include repeated objects, occlusion, mirrored images, stale anchors, conflicting sensors, and ambiguous requests.
Record the request, observations, tool calls, outputs, and final answer. A polished explanation should fail if it names the wrong object or frame. A concise answer with correct provenance is more valuable than a confident story.
Continue the field guide
Turn Spatial Data Into a Reliable Contract
Use the journal for implementation patterns, standards context, engine integration, and platform-specific design decisions.
Read the Spatial Systems Journal