Control your Mac using hand gestures — no mouse, no keyboard needed.
Handsi is a local, privacy-first gesture control system that tracks your hands via webcam and maps gestures to OS actions like switching desktops, scrolling, clicking, and window management.
Installation Guide - Download the DMG and start using Handsi in 2 minutes
Connecting might take a minute - initializing backend takes a long time.
Developer Installation - Build from source or contribute
Usage Guide — guide to using Handsi
Stand or walk around while reading on your screen. Control scrolling, zooming, changing tabs, and page navigation without being tethered to your desk.
Keep both hands free to eat while controlling playback, volume, and navigation with simple gestures.
Use your trackpad for precise work with one hand, while your other hand handles clicking, scrolling, or switching desktops via gestures.
Perfect for when you want to get handsi!
- Real-time hand tracking (single webcam)
- Gesture toggle: enable/disable control mode
- Configurable gesture → action mapping
- Actions:
- Mouse movement
- Click
- Scroll (vertical/horizontal)
- Volume control
- Zoom in/out
- Tab through windows
- Desktop switching (next/previous workspace)
- Window management (view all windows)
- Copy, paste, undo
- etc...
- Habit detection
- Prevents abscentmindedly touching your face
- Must be enabled in settings and gestures
Planned:
- Adjustable signals learned over time
- Voice + motion labeling
- Onboarding and calibration
- Gesture combos (temporal and physical grammar)
| Platform | Status | Notes |
|---|---|---|
| macOS | ✅ Supported | Apple Silicon and Intel |
| Linux | ⏳ Planned | Future release |
| Windows | ⏳ Planned | Future release |
- Installation Guide — Download DMG and install
- Usage Guide — How to use gestures and configure settings
- Developer Installation — Set up dev environment
- Build Guide — Build from source
- Release Guide — Create GitHub releases
- Tauri Guide — Tauri architecture and migration
- Implementation Details — Technical specifications
- Architecture — System design and data flow
- Frontend: HTML/CSS/JavaScript (Tauri WebView)
- GUI Framework: Tauri (Rust)
- Backend: Python 3.11
- Computer Vision: MediaPipe, OpenCV
- Platform Integration: PyObjC (macOS)
Contributions are welcome! Please:
- Fork the repository
- Create a feature branch:
git checkout -b feature/my-feature - Make your changes following CLAUDE.md guidelines
- Commit:
git commit -m "Add my feature" - Push:
git push origin feature/my-feature - Open a Pull Request
License information to be added
Built with Tauri + Python + MediaPipe