skip to content
← (hackathons)

nexstep.png

Nexstep

iPhone-on-a-cane app that learns which cue (AR lines, vibration, sound) breaks a Parkinson's patient's freezing of gait — and fires it pre-emptively.

nexstep.png[01]

Nexstep: nexstep-dashboard.png

nexstep-2.png[02]

Nexstep: nexstep-active-cue.png

nexstep-3.png[03]

Nexstep: nexstep-cue-library.png

Context

Built at Hong Kong Talent Engage (3–5 July 2026). Nexstep tackles Freezing of Gait (FoG) in Parkinson's disease — the sudden feeling of feet "glued to the floor", one of the most disabling symptoms and a leading cause of falls. It connects to Heidi's ongoing research interest in freeze of gait and augmentation approaches.

Tagline: The cane that learns which cue breaks your freeze — and fires it before you ask. Built from one iPhone and a 3D-printed clip.

The problem

Sensory cues (a beat, a vibration, a line on the floor) can break a freeze, but no single cue works for everyone: a 2023 RCT (Klaver et al.) found auditory and tactile cues each helped many people — just not the same people. Every cueing aid on the market picks one cue for everyone and waits to be switched on. Personalisation is the evidence-based answer.

What we built

A fully functional iOS app that turns an iPhone clipped to a cane (via a 3D-printed mount) into an adaptive cueing system — fully offline and on-device. Everything was written during the hackathon, using only Apple system frameworks.

  • Sense: 100 Hz device motion; a Freeze Index computed with FFT band-power (3–8 Hz vs. 0.5–3 Hz), fused with akinetic (sudden-stop) detection, camera stillness and double-tap detection.
  • Decide: a transparent contextual bandit (ε-greedy, prior-smoothed, per-hotspot) picks single cues or cross-channel combinations from the user's enabled set.
  • Cue: Core Haptics vibration through the cane, audio (metronome, music, anti-habituation randomised beats), speech in English and Cantonese, and LiDAR-anchored ARKit floor stepping lines — a 16-cue catalog, each tagged with its level of clinical evidence.
  • Learn: every freeze and its outcome is logged on-device and folded back into the policy, so cue selection personalises over time.
  • Freeze hotspots: recurring freeze locations are clustered and the app pre-arms via GPS ~10 m before you arrive.
  • Manual fallback: double-tap the cane or say a word to fire your preferred cue, which also teaches the model. "Report a fall" feeds the learning loop.
  • Designed as a sound walking cane first — if the phone is removed or dead, it's still a cane.
  • Target users: Parkinson's patients at Hoehn & Yahr stage 2–3.

Tech

Swift · SwiftUI · Core Motion · Core Haptics · AVFoundation · ARKit (LiDAR) · Core Location · Accelerate/vDSP (FFT) · XcodeGen · 3D-printed mount

Links