Blinky
Autonomous AI desktop tutor & computer-use agent. Dual-engine architecture (System 1 reflex fast-path + System 2 agentic loop), full Windows-to-Linux Wayland/Hyprland port, spatial OCR/@ref grounding, and an authenticated Expo mobile companion.
Desktop agents that automate real software suffer from two fatal bottlenecks: multi-second conversational latency when executing simple OS tasks through LLMs, and brittle coordinate targeting across divergent display compositors (Windows UIA vs Linux Wayland/Hyprland).
Desktop agents that automate real software suffer from two fatal bottlenecks: multi-second conversational latency when executing simple OS tasks through LLMs, and brittle coordinate targeting across divergent display compositors (Windows UIA vs Linux Wayland/Hyprland). The platform needed sub-5ms reflex execution, robust spatial screen grounding, and zero-compromise Linux parity.
Architected a dual-engine agent loop (System 1 & System 2): built a fast-path reflex classifier that intercepts atomic system commands (volume, media, window management, lock) and executes them directly via blinky-daemon in <5ms without LLM overhead.
Engineered the full Windows-to-Linux port: built the pure-Rust blinky-daemon with native Hyprland/Wayland compositor bindings, PipeWire screen capture, kernel uinput virtual keyboard/mouse actuation, and ATSPI accessibility tree parsing.
Built the spatial computer-use perception pipeline: fused high-speed Tesseract OCR and OmniParser YOLO detection to index on-screen targets with stable @ref tags (@eN) and normalized bounding boxes, cached and attached directly to daemon frame buffers.
Integrated cross-runtime MCP schema normalization with time/step circuit breakers and a mobile companion (Expo/React Native) featuring OkHttp TLS cert pinning and an Antigravity human-in-the-loop approval gate.