Human interaction · Browser ML
Gesture recognition
A browser-based hand gesture controller for touchless reading.
The problem
Reading manga or webtoons while eating makes a touchscreen an awkward interface. A touchless controller needs deliberate gestures without scrolling again on the return stroke.
What I built
IrfanLLM uses MediaPipe hand landmarks and 71-dimensional geometric features. A 100-tree Random Forest trained from video demonstrations is compiled to JavaScript for browser inference. Geometric checks, a 0.30-second scroll cooldown, and virtual air buttons translate gestures into navigation. The existing demo, bookmarklet, and extension remain available.
Constraints & lessons
Camera permission, hand visibility, lighting, and browser support affect the experience. A classifier score does not include camera capture or landmark extraction latency. The demo requires explicit camera access and must be tested in the visitor’s environment.
Result & evidence
The source portfolio reports 98.6% cross-validation accuracy. This is an evaluation on the training workflow’s folds, not a measured accuracy across all users or real-world reading sessions. The working browser controller offers a concrete way to explore that gap.