Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
78200d0
Enhance documentation in CLAUDE.md and add Pyright configuration for …
stedwick Oct 3, 2025
34140d1
Add type stubs for Raspberry Pi specific libraries to enable type che…
stedwick Oct 3, 2025
22ebfac
Update Raspberry Pi camera imports to include type stubs for compatib…
stedwick Oct 3, 2025
3835c08
Add dummy functions for Windows and Linux in main.py to support platf…
stedwick Oct 3, 2025
1853687
Enhance hotkey_nix_uinput.py with type stubs for compatibility with n…
stedwick Oct 3, 2025
6d244e1
Add type stubs for Xlib imports in hotkey_nix_x11.py to enhance compa…
stedwick Oct 3, 2025
a004fc1
Enhance hotkey_win_mac.py with type annotations for improved code cla…
stedwick Oct 3, 2025
5e456da
Add type stubs for Quartz imports in mouse_mac.py to improve type che…
stedwick Oct 3, 2025
733a8f8
Add type stubs for evdev imports in mouse_nix_uinput.py to improve co…
stedwick Oct 3, 2025
e8d7ded
Add type stubs for Xlib imports in mouse_nix_x11.py to improve compat…
stedwick Oct 3, 2025
1a6933d
Add type stubs for windll in mouse_win.py to improve compatibility wi…
stedwick Oct 3, 2025
4041529
Update documentation to emphasize the importance of running `pyright`…
stedwick Oct 3, 2025
ef7a9ad
Add independent X/Y axis speed options to client configuration
stedwick Oct 3, 2025
8cdd7ae
Merge branch 'xkeys' into pyright
stedwick Oct 3, 2025
7c733e0
Enhance documentation to emphasize mandatory virtual environment acti…
stedwick Oct 3, 2025
1c9cf4c
Refactor hotkey function to improve readability and maintainability
stedwick Oct 3, 2025
198812e
Refactor hotkey definitions for improved clarity and consistency
stedwick Oct 3, 2025
f9505da
Update hotkey handling to include type hints for better type checking
stedwick Oct 3, 2025
a773eae
Add type hints to mouse capabilities for improved type checking
stedwick Oct 4, 2025
59a711f
Refactor mouse capabilities type hints for enhanced type safety
stedwick Oct 4, 2025
61ae82c
Refactor mouse capabilities type definition for clarity and type safety
stedwick Oct 4, 2025
4807716
Refactor type hints in mouse and evdev stubs for improved type safety
stedwick Oct 4, 2025
dc3ea92
Update .gitignore to ignore all virtual environment directories
stedwick Oct 4, 2025
c4901aa
Add xkeys argument to enable speed multiplier for macOS clients
stedwick Oct 6, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
83 changes: 83 additions & 0 deletions .cursorrules
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
# Cursor IDE Rules for pi-nav

## Project Overview

PhilNav is a cross-platform head-tracking mouse system with client-server architecture:

- **Server**: Raspberry Pi with camera for IR blob detection
- **Client**: Windows/Mac/Linux desktop for mouse control
- **Communication**: UDP multicast protocol

## Critical Development Rules

### Virtual Environment (MANDATORY)

**ALWAYS activate the virtual environment first before any development work:**

```bash
source .venv/bin/activate
```

This is required for all Python operations, including running code, installing packages, and type checking.

### Type Checking (MANDATORY)

**ALWAYS run `pyright` after any code changes.** This project maintains zero errors and zero warnings across all platforms.

```bash
# Activate venv first, then check types
source .venv/bin/activate
pyright

# Check with statistics
pyright --stats

# Check specific directories
pyright client_win-mac-nix/
pyright server_raspberrypi/
```

### Code Quality Standards

- Follow PEP 8 with 4-space indentation
- Use explicit function names (`send_udp_packet`, not `sendPkt`)
- Platform-specific modules: `mouse_<platform>.py`, `hotkey_<platform>.py`
- Module constants: UPPERCASE
- Runtime flags: lowercase_with_underscores

### Project Structure

- `client_win-mac-nix/`: Cross-platform client code
- `server_raspberrypi/`: Pi-specific server code
- `stubs/`: Type stubs for platform-specific libraries (if present)
- Use `CLAUDE.md` for extended documentation

### Development Commands

```bash
# ALWAYS activate venv first
source .venv/bin/activate

# Server (Pi)
python3 server_raspberrypi/main.py --verbose --preview

# Client (Desktop)
python3 client_win-mac-nix/main.py --verbose --speed 21 --smooth 3

# Install dependencies
pip install -r requirements.txt
```

### Testing & Validation

- Test on multiple platforms when possible
- Use `--verbose` flags for debugging
- Capture logs for manual verification
- Document platform coverage in PRs

## Deployment Notes

- Default UDP ports: 4245/4246
- Document permission changes (uinput, etc.)
- Update `server_raspberrypi/bashrc` for Pi packages
- Coordinate port changes with downstream clients
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -121,7 +121,7 @@ celerybeat.pid

# Environments
.env
.venv
.venv*
env/
venv/
ENV/
Expand Down
4 changes: 3 additions & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,15 +8,17 @@

## Build, Test, and Development Commands

- **CRITICAL**: Always activate the virtual environment first with `source .venv/bin/activate`
- `python3 server_raspberrypi/main.py --verbose --preview` starts the camera server with on-device preview for tuning thresholds and masks.
- `python3 client_win-mac-nix/main.py --verbose --speed 21 --smooth 3` launches the desktop client with the defaults Phil ships; tweak flags to validate new features.
- Use `--x-speed` and `--y-speed` for independent X/Y axis control (default Y is 1.25x faster than X).
- Activate the virtual environment with `source .venv/bin/activate` when present, then log new dependencies in `requirements.txt` before `pip install -r requirements.txt`.
- Log new dependencies in `requirements.txt` before `pip install -r requirements.txt`.

## Coding Style & Naming Conventions

- Follow PEP 8 with 4-space indentation; prefer explicit verbs in function names (`send_udp_packet`, not `sendPkt`).
- Keep module-level constants uppercase (`DEFAULT_SPEED`), runtime flags lowercase with underscores, and platform-specific modules following the `mouse_<platform>.py` / `hotkey_<platform>.py` pattern.
- **CRITICAL**: Always run `pyright` after any code changes to ensure type correctness. The project maintains zero errors and zero warnings across all platforms.
- Run `ruff check .` and `black .` if you introduce them; document new tooling both here and in `CLAUDE.md`.

## Testing Guidelines
Expand Down
27 changes: 27 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,9 @@ Communication happens over UDP multicast (ports 4245/4246).
### Running the Server (Raspberry Pi)

```bash
# IMPORTANT: Activate virtual environment first
source .venv/bin/activate

# Basic run
python3 server_raspberrypi/main.py

Expand All @@ -50,6 +53,9 @@ python3 server_raspberrypi/main.py --fps 75 --width 320 --height 240
### Running the Client (PC)

```bash
# IMPORTANT: Activate virtual environment first
source .venv/bin/activate

# Basic run
python3 client_win-mac-nix/main.py

Expand Down Expand Up @@ -118,6 +124,26 @@ sudo shutdown -r now

The client automatically detects the platform and loads appropriate mouse/hotkey modules using Python's `platform.system()` and match/case statements.

### Type Checking with Pyright

The project includes comprehensive type checking support:

```bash
# IMPORTANT: Activate virtual environment first
source .venv/bin/activate

# Install Pyright (if not already installed)
pip install pyright

# Check entire project
pyright

**Cross-Platform Type Checking**: The server code includes conditional imports and type stubs in the `stubs/` directory that allow full type checking even on non-Pi development machines:

Configuration is in `pyrightconfig.json` with platform-specific file exclusions and stub path configuration.

**IMPORTANT**: Always run `pyright` after making any code changes to ensure type correctness. The project maintains zero errors and zero warnings across all platforms.

## Important Configuration Parameters

### Client
Expand All @@ -135,3 +161,4 @@ The client automatically detects the platform and loads appropriate mouse/hotkey
- `--blob-min-threshold`: Brightness threshold for blob detection (default 200)
- `--contours`: Enable contour tracking for better detection with dull stickers
- Camera controls: `--gain`, `--brightness`, `--contrast`, `--exposure`
```
43 changes: 32 additions & 11 deletions GEMINI.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,21 +4,42 @@ This project, PhilNav, is a high-performance, open-source head-tracking mouse sy

The system is built with a client-server architecture:

* **Server:** Runs on a Raspberry Pi equipped with a PiCamera Module 3 NoIR. It uses Python with `picamera2` for camera interfacing and `opencv-python` for real-time blob detection of the reflective sticker. The server calculates the movement delta and broadcasts it over the local network via UDP.
- **Server:** Runs on a Raspberry Pi equipped with a PiCamera Module 3 NoIR. It uses Python with `picamera2` for camera interfacing and `opencv-python` for real-time blob detection of the reflective sticker. The server calculates the movement delta and broadcasts it over the local network via UDP.

* **Client:** Runs on a Windows, macOS, or Linux machine. It listens for the UDP broadcasts from the server and uses platform-specific libraries to control the mouse cursor.
- **Client:** Runs on a Windows, macOS, or Linux machine. It listens for the UDP broadcasts from the server and uses platform-specific libraries to control the mouse cursor.

The communication between the client and server is done using the OpenTrack protocol, with the movement data packed as `struct` doubles.

## Virtual Environment & Type Checking

**CRITICAL**: Always activate the virtual environment first before any development work:

```bash
source .venv/bin/activate
```

This project uses Pyright for comprehensive static type checking across all platforms (Windows, Mac, Linux, Raspberry Pi). **Always run `pyright` after making any code changes** to ensure type correctness. The project maintains zero errors and zero warnings.

```bash
# Activate venv first, then check types
source .venv/bin/activate

# Check entire project
pyright

# Check with statistics
pyright --stats
```

# Building and Running

## Dependencies

### Server (Raspberry Pi)

* `python3-opencv`
* `picamera2`
* `libcamera`
- `python3-opencv`
- `picamera2`
- `libcamera`

These can be installed via `apt`:

Expand All @@ -28,7 +49,7 @@ sudo apt install python3-opencv

### Client (Desktop)

* `evdev` (for Linux/Wayland)
- `evdev` (for Linux/Wayland)

This can be installed via `pip`:

Expand All @@ -54,8 +75,8 @@ The application can be configured using various command-line arguments on both t

# Development Conventions

* **Code Style:** The code follows standard Python conventions (PEP 8).
* **Modularity:** The project is organized into two main components: `client_win-mac-nix` and `server_raspberrypi`. Within the client, platform-specific logic is further separated into individual modules (e.g., `mouse_mac.py`, `mouse_win.py`, `mouse_nix_uinput.py`).
* **Configuration:** All configuration is handled through command-line arguments via the `argparse` library. There are no configuration files.
* **Networking:** Communication is done via UDP multicast, which simplifies network configuration as the client and server don't need to know each other's IP addresses.
* **Error Handling:** The code includes basic error handling, such as timeouts for network operations.
- **Code Style:** The code follows standard Python conventions (PEP 8).
- **Modularity:** The project is organized into two main components: `client_win-mac-nix` and `server_raspberrypi`. Within the client, platform-specific logic is further separated into individual modules (e.g., `mouse_mac.py`, `mouse_win.py`, `mouse_nix_uinput.py`).
- **Configuration:** All configuration is handled through command-line arguments via the `argparse` library. There are no configuration files.
- **Networking:** Communication is done via UDP multicast, which simplifies network configuration as the client and server don't need to know each other's IP addresses.
- **Error Handling:** The code includes basic error handling, such as timeouts for network operations.
60 changes: 44 additions & 16 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,8 @@
# PhilNav-Python-RaspberryPi


## Intro

PhilNav is a *very good* infrared head mouse, sorta like the discontinued NaturalPoint SmartNav. It runs at 75 FPS for buttery smooth mouse movements and pixel perfect accuracy.
PhilNav is a _very good_ infrared head mouse, sorta like the discontinued NaturalPoint SmartNav. It runs at 75 FPS for buttery smooth mouse movements and pixel perfect accuracy.

[Note: zero hardware or software is from the SmartNav, everything is built and programmed 100% from scratch by me, Phil.]

Expand All @@ -15,9 +14,33 @@ Watch Phil's YouTube video here: https://youtu.be/JTLs7z0PO-k

[![IMAGE ALT TEXT HERE](https://img.youtube.com/vi/JTLs7z0PO-k/0.jpg)](https://www.youtube.com/watch?v=JTLs7z0PO-k)

## Quick Start

**IMPORTANT**: Always activate the virtual environment first:

```bash
source .venv/bin/activate
```

### Running the Client (PC)

```bash
source .venv/bin/activate
python3 client_win-mac-nix/main.py --verbose --speed 21 --smooth 3
```

### Running the Server (Raspberry Pi)

```bash
source .venv/bin/activate
python3 server_raspberrypi/main.py --verbose --preview
```

For detailed setup instructions, see `CLAUDE.md`.

There's \*lots\* of assembly required since the Raspberry Pi has to be built. You can buy a pre-built PhilNav from Phil here: https://philipb.gumroad.com/l/philnav

-----
---

PhilNav has exceeded all my expectations. It's better than other head-mice, and I hit all of my goals:

Expand All @@ -35,21 +58,22 @@ The trade-offs I made include:
1. No clicking. You'll have to use a switch, pedal, or other dwell clicking software.

There are many other head-mice, but mine's the best:
* https://smylemouse.com
* https://abilitare.com
* https://glassouse.com
* https://www.orin.com/access/headmouse/
* https://www.spectronics.com.au/product/trackerpro-2
* https://www.quha.com
* https://eviacam.crea-si.com/

- https://smylemouse.com
- https://abilitare.com
- https://glassouse.com
- https://www.orin.com/access/headmouse/
- https://www.spectronics.com.au/product/trackerpro-2
- https://www.quha.com
- https://eviacam.crea-si.com/

It uses a client/server model; the server runs on a Raspberry Pi with a Picam 3 NoIR camera, and the client runs on your Windows, Apple macOS, or Linux PC. They communicate over Wi-Fi or ethernet via UDP multicast.

It's free and open source on GitHub, written from scratch in Python3 by Philip Brocoum. There are no dependencies on the 2002 SmartNav hardware or software (it's discontinued anyway by NaturalPoint as of 2018).
It's free and open source on GitHub, written from scratch in Python3 by Philip Brocoum. There are no dependencies on the 2002 SmartNav hardware or software (it's discontinued anyway by NaturalPoint as of 2018).

## Running PhilNav

Start by running the following Python scripts on the client and server. You may have to ```pip install``` a bunch of things first. I put my reflective sticker on my headset mic boom.
Start by running the following Python scripts on the client and server. You may have to `pip install` a bunch of things first. I put my reflective sticker on my headset mic boom.

```
# server
Expand All @@ -60,31 +84,34 @@ pip install ...
```

#### Server - Raspberry Pi

```
python3 server_raspberrypi/main.py --verbose --preview
```

#### Client PC - Win/Mac/Nix

```
python3 client_win-mac-nix/main.py --verbose
```

On the Raspberry Pi you should see a preview of the camera, and on the client machine the mouse should start to move. Use ```--help``` to change your settings to your liking.
On the Raspberry Pi you should see a preview of the camera, and on the client machine the mouse should start to move. Use `--help` to change your settings to your liking.

Here are my own settings:

```
python3 server_raspberrypi/main.py
python3 server_raspberrypi/main.py

python3 client_win-mac-nix/main.py --speed 21 --smooth 3 --deadzone 0.04 --keepawake 56 --timeout $((60*60*8))
```

(If you have a firewall, ports 4245 & 4246 must be open to send/recv UDP.)

#### Note: Linux using `/dev/uinput` (with Wayland & X11)

I've modernized it to send mouse movements to /dev/uinput instead of X11 calls, so it works on Wayland (& X11) and should be future-proof. However, this requires permission to read and write /dev/uinput. You can run as root, or give your user permission:

You'll need to install ```pip install evdev``` or ```sudo apt install python3-evdev``` and add your user to the input group.
You'll need to install `pip install evdev` or `sudo apt install python3-evdev` and add your user to the input group.

```
# Add user to input group
Expand All @@ -100,6 +127,7 @@ sudo shutdown -r now
Watch the YouTube video here:

Parts:

- Raspberry Pi 5 - https://www.canakit.com/canakit-raspberry-pi-5-starter-kit-turbine-black.html
- Pi Breadboard kit - https://www.canakit.com/raspberry-pi-gpio-breakout-bundle.html
- Pi Camera Module 3 NoIR - https://www.canakit.com/raspberry-pi-camera-module-3.html
Expand All @@ -109,6 +137,6 @@ Parts:
- Reflective tape - https://www.amazon.com/dp/B06VTTV6PR
- Selfie Stick - https://www.amazon.com/dp/B09W99QJBP

-----
---

With this open source project, disabled folks are not at the mercy of a private company that might discontinue products, and we are not stuck on Windows =)
9 changes: 6 additions & 3 deletions client_win-mac-nix/hotkey_nix_uinput.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,7 @@
from evdev import InputDevice, ecodes
try:
from evdev import InputDevice, ecodes # type: ignore[import-untyped]
except ImportError:
from stubs.evdev_stubs import InputDevice, ecodes # type: ignore[import-not-found]
from time import time
import glob
import select
Expand Down Expand Up @@ -55,14 +58,14 @@ def hotkey_run(callback=None, multiplier_callback=None):
now = time()
if now - hotkey_time_f7 > 0.25:
hotkey_time_f7 = now
callback()
callback() # type: ignore[misc]

# Check for F8 press while shift is held
if event.code == F8_KEYCODE and event.value == 1 and shift_pressed[fd]:
now = time()
if now - hotkey_time_f8 > 0.25:
hotkey_time_f8 = now
multiplier_callback()
multiplier_callback() # type: ignore[misc]
except BlockingIOError:
continue

Expand Down
Loading