Lab

Where we test technology before it reaches a client project. Each entry is a question, the approach we are taking and what we are watching for, not a result.

Prototype: something is built and running. Exploring: reading, measuring, small tests.

A breadboard with electronic components and coloured wires

Geospatial Experiments

01Geospatial ExperimentsPrototype, 2026

Geospatial anchor stability across times of day

How much does the position and heading accuracy of ARCore geospatial anchors vary at the same stop between early morning, midday and dusk?

Approach: We log the accuracy values reported by the ARCore Geospatial API and the apparent drift of a fixed reference anchor at a handful of outdoor stops, visiting each at different hours over several weeks. A small Unity test app records the data, and we compare it against each stop's sky visibility and the position of the sun.
What we are watching for: We are watching for a consistent relationship between time of day and accuracy at a given stop. If one exists, it would tell us when to schedule content and when to fall back to simpler modes.

Unity
AR Foundation
ARCore Geospatial API
GIS data
C#

02Geospatial ExperimentsExploring, 2026

Cesium 3D tiles streamed into Unity for city-scale AR

Can streamed 3D tiles give geospatial AR a usable occlusion and alignment reference across a whole city centre without modelling each site?

Approach: We stream Cesium 3D tiles into a Unity AR Foundation scene and use the tile geometry as an invisible occluder and as an alignment check against geospatial anchors. We are testing tile level of detail, memory use on mid-range phones and how closely tile edges match real building edges seen from the street.
What we are watching for: We are looking for whether tile geometry matches reality closely enough to occlude AR content behind buildings. A positive result would remove most per-site modelling from city-scale projects.

Unity
Cesium
AR Foundation
ARCore Geospatial API
C#

Spatial Interfaces

03Spatial InterfacesExploring, 2026

Hand-tracked text input in passthrough

Which hand-only text entry method is fastest and least tiring for short inputs such as a name or a search term in mixed reality?

Approach: We implement three input methods on Quest 3 in passthrough: a floating keyboard with pinch selection, a keyboard anchored to the palm and a swipe-style keyboard. Studio testers enter short phrases while we record time, corrections and self-reported fatigue.
What we are watching for: We are watching for a method that holds up for repeated short entries without controllers. A result would shape how we design forms and search in museum and enterprise mixed reality.

Unity
Meta Quest SDK
OpenXR
Hand & body tracking
C#

04Spatial InterfacesPrototype, 2025

Projection-mapped tangible table

Can physical blocks on a projected table be tracked reliably enough with a depth camera alone to drive a shared planning interface for several people?

Approach: We project a map onto a table and track wooden blocks with an overhead depth camera, with small ESP32 tags in the blocks as a check on the vision tracking. Unity renders the projection and reads block positions over OSC. We test how many blocks and hands the camera can tell apart before tracking breaks down.
What we are watching for: We are watching for the point at which depth tracking alone stops being reliable and tags become necessary. That threshold sets the budget for tangible interfaces in exhibitions and workshops.

Unity
Depth cameras
Projection mapping
Microcontrollers & sensors
OSC / MQTT
C#

XR Experiments

05XR ExperimentsExploring, 2026

Vision Pro vs Quest 3 input for museum visitors

Does gaze-and-pinch on Apple Vision Pro or hand tracking on Quest 3 lead to fewer errors for first-time visitors interacting with anchored content?

Approach: We port the same object-anchored story interface to visionOS and to Quest 3 and run short sessions with first-time users in the studio. We log selection errors, time to the first successful interaction and how often people need a hint.
What we are watching for: We are looking at whether one input model is clearly easier for people who have never worn a headset. The answer would inform device recommendations for loan programmes.

Unity
visionOS
Meta Quest SDK
OpenXR
Hand & body tracking
Eye tracking

06XR ExperimentsPrototype, 2025

WebXR versus native AR fidelity on the same scene

How far behind native AR Foundation is WebXR today in tracking stability, rendering quality and start-up time for a marker-free outdoor scene?

Approach: We build one small AR scene twice, in Unity with AR Foundation and in the browser with WebXR and Three.js, and run them side by side on the same phones. We compare tracking drift, frame rate, visual quality and the time from tapping a link or an icon to seeing content.
What we are watching for: We are watching for the gap to narrow to the point where a web build is acceptable for short campaign experiences. That would change our default recommendation for projects where installing an app is a barrier.

WebXR
Three.js
TypeScript
Unity
AR Foundation

Related: AR Development, XR Development

AI Experiments

07AI ExperimentsPrototype, 2026

Latency masking for a conversational LLM guide

What should a user see and hear during the several seconds between asking a spoken question and receiving a grounded answer in a headset?

Approach: We build variations of the waiting state for our Spatial Guide prototype: an immediate transcript, a staged progress indicator, a spoken acknowledgement and a partial streamed answer. Studio testers ask the same set of questions under each variation and tell us when the wait felt too long.
What we are watching for: We are looking for the variation that makes a fixed wait feel acceptable. A result would become our default pattern for AI agents in XR.

Unity
Large language models
Speech recognition & synthesis
Node.js
Meta Quest SDK

Related: AI + XR, Mixed Reality

Computer Vision

08Computer VisionPrototype, 2026

On-device recognition of small artefacts

Can a model running entirely on a headset recognise small, similar museum artefacts reliably enough to anchor content without a server round trip?

Approach: We train a compact classifier on photogrammetry renders and photographs of a set of small replica objects and run it on Quest 3 through Unity Sentis. We vary lighting, distance and viewing angle and record recognition rate, confusion between similar objects and inference time.
What we are watching for: We are looking at whether on-device recognition is good enough to replace or bootstrap a cloud vision call. A positive result would lower latency and keep camera frames off the network.

On-device ML
Computer vision
Unity
Photogrammetry
Python

Related: AI + XR, Cultural Heritage XR

Unity Prototypes

09Unity PrototypesPrototype, 2026

Passthrough text legibility test rig

What combination of type size, panel opacity and distance keeps text readable in Quest 3 passthrough under typical gallery lighting?

Approach: A Unity scene cycles through type sizes, weights, panel backings and distances in passthrough while testers read short passages aloud under spot and diffuse lighting in the studio. We record reading errors and the settings people choose when allowed to adjust them.
What we are watching for: We are looking for thresholds we can write into a style guide for mixed reality panels. A result would replace rules borrowed from screens with ones measured in passthrough.

Unity
Meta Quest SDK
OpenXR
C#

Generative Systems

10Generative SystemsExploring, 2026

Generative soundscapes from room geometry

Can a soundscape generated from a scanned room's dimensions, materials and openings feel like it belongs to that room rather than to a preset?

Approach: We scan rooms with a headset or LiDAR, extract volume, surface materials and openings, and feed them as parameters to a generative audio system that produces an evolving ambient layer. The output is played spatially in the same room and compared with a preset soundscape.
What we are watching for: We are listening for whether people can tell which soundscape was generated from the room they are standing in. A reliable difference would open up site-specific audio for installations without composing per site.

Generative AI
3D scanning
Unity
Meta Quest SDK
TouchDesigner

11Generative SystemsPaused, 2025

Procedural reconstruction of missing architecture from photographs

How much of a demolished facade can be reconstructed as a plausible 3D model from a few historical photographs and its surviving neighbours, and how should uncertainty be shown?

Approach: We combine photogrammetry of surviving buildings with a procedural facade grammar fitted to historical photographs, generating candidate reconstructions in Blender. Elements that are inferred rather than documented are tagged so they can be rendered differently in AR.
What we are watching for: We are watching for whether a grammar-based approach produces results a heritage specialist would accept as plausible, with the uncertainty legible to visitors. The work is paused while the anchoring experiments take priority.

Blender
Photogrammetry
Generative AI
Python
Unity

Experiments that mature become case studies in Work or explanations in Insights. If one of these questions is also yours, we are happy to compare notes.

Next step

Working on a similar question?

Research groups, universities and product teams sometimes bring us in to run an experiment before a bigger decision. Tell us what you want to find out.

Prefer email? mrerfansol@gmail.com

Lab