codegaze.jpg

CodeGaze

Open-source IntelliJ plugin + OpenXR viewer that streams your editor into VR and records which code token your eye or head ray hits.

Context

CodeGaze is the VR research instrument behind GAZEAIDE, released as an open-source research preview (MIT licence) in September 2026. Where GAZEAIDE is about acting on gaze, CodeGaze is about measuring it: which exact code token a developer's eye (or head) ray lands on while they read code in VR.

Tagline: See your live IntelliJ editor in VR. Record where a measured eye or head ray meets the code. Export the session.

What it does

Displays the active IntelliJ editor on a floating virtual monitor in the headset, with syntax highlighting and scrolling (editor image ~5 Hz; head movement rendered at the headset's native refresh rate).
Prefers real eye gaze through OpenXR's standard eye-gaze extension when valid; otherwise falls back to the centre of the headset view. Every sample is labelled eye, head or simulated, with fallback reason and validity.
Maps each estimated point to a code token, its source occurrence and — via IntelliJ PSI — its resolved symbol (e.g. distinguishing variable vs. method references), against the exact frame the viewer received, preserving unsaved document revisions and offsets.
Saves continuously to local disk and exports a ZIP with session.json, samples.csv, samples.jsonl and frames.jsonl (visible token rectangles; no screenshots).
Includes analyze.py to derive token dwell intervals, a desktop mouse simulator and a synthetic demo server so supervisors can try it without a headset.

Three viewers, one plugin

Native OpenXR viewer (built with Godot) for Windows + Bigscreen Beyond 2e / SteamVR, Quest via Link, or any OpenXR headset — capability-detected eye or head tracking.
Browser WebXR viewer for Quest Browser over USB (adb reverse), using head direction — works from a Mac without a gaming GPU.
Browser simulator on any OS with explicitly labelled simulated mouse points.

Research rigour

The README is explicit about what is and isn't validated: geometry, tracking policy, mapping, export and IntelliJ PSI are covered by automated tests and CI; headset interoperability and measured gaze accuracy still require a documented hardware acceptance procedure. It warns that a token under an estimated coordinate isn't proof of attention, that head direction isn't eye tracking, and that gaze doesn't equal comprehension. Recordings stay local — no upload or analytics.

Tech

Java · IntelliJ Platform SDK / PSI (IntelliJ 2024.3+) · Godot (OpenGL compatibility renderer) · OpenXR (XR_EXT_eye_gaze_interaction) · WebXR · Python analysis tools · GitHub Actions CI · SteamVR · Bigscreen Beyond 2e eye tracking

Why this shows I can…

Build research infrastructure other people can actually use: installable releases, docs, data-format spec, contributing guide.
Handle hard cross-device engineering (OpenXR, WebXR, desktop) with honest capability detection instead of overclaiming.
Think like a researcher about validity, timestamps, data loss and privacy.

Results

v0.1.0 released with plugin ZIP and Windows viewer build.
CI build and test pipeline on GitHub.