Device Monitoring System

A modular application that monitors various device components and publishes statistics to an MQTT broker, with an integrated web interface for visualization. Currently supports:

• GPU Monitoring: GPU statistics (utilization, memory, temperature)
  • NVIDIA GPUs on Linux/Windows
  • Apple Silicon GPUs on macOS
• CPU Monitoring: CPU usage, frequency, and temperature
  • Cross-platform support (Linux, Windows, macOS)
  • Apple Silicon optimized on macOS ARM64
• Web Interface: Real-time visualization of MQTT messages and device statistics

Installation and Setup

Automatic Installation (Recommended)

The easiest way to install the MQTT Device Monitor is to use the provided installation script:

# Clone the repository
git clone https://github.com/drascom/mqtt-device-monitor.git
cd mqtt-device-monitor

# Run the installation script
# For Linux: run with sudo to enable service installation
sudo ./install.sh

# For macOS: no sudo required
./install.sh

The installation script will:

1. Check for and install required dependencies
2. Create a Python virtual environment
3. Install required Python packages
4. Create a .env file if it doesn't exist
5. Set up a system service (systemd on Linux, launchd on macOS)

Uninstallation

To uninstall the MQTT Device Monitor, use the provided uninstall script:

# For Linux: run with sudo to remove system services
sudo ./uninstall.sh

# For macOS: no sudo required
./uninstall.sh

The uninstall script will:

1. Stop and remove the system service
2. Remove desktop entries (on Linux)
3. Optionally delete the entire project folder

Manual Installation

Clone the Repository

git clone https://github.com/drascom/mqtt-device-monitor.git
cd mqtt-device-monitor

Set Up Python Environment with uv

1. Install uv (if not already installed):

# On macOS/Linux
curl -LsSf https://astral.sh/uv/install.sh | sh

# Or with pip
pip install uv

2. Create and activate a virtual environment:

uv venv
source .venv/bin/activate  # On Linux/macOS
# OR
.venv\Scripts\activate     # On Windows

3. Install dependencies:

uv pip install -r requirements.txt

4. For macOS users, for better CPU temperature monitoring, install the osx-cpu-temp utility:

# Using Homebrew
brew install osx-cpu-temp

# Or build from source
git clone https://github.com/lavoiesl/osx-cpu-temp.git
cd osx-cpu-temp
make
sudo make install

Configuration

1. Copy the .env.template file to .env:

   cp .env.template .env

2. Edit the .env file and set a unique DEVICE_ID for each device and configure other settings:

   # Common MQTT settings
   MQTT_BROKER=your.mqtt.server.address
   MQTT_PORT=443
   MQTT_USERNAME=your_username
   MQTT_PASSWORD=your_password
   
   # MQTT client settings
   DEVICE_ID=computer1  # Change this to a unique identifier for each device
   MQTT_INTERVAL=5
   MQTT_VERSION=3.1.1
   MQTT_KEEPALIVE=60
   
   # GUI settings
   HOST=0.0.0.0
   PORT=9876
   DEBUG=False
   SECRET_KEY=your-secret-key
   

3. The application will automatically detect and use all available monitoring modules (currently gpu and cpu).

4. The application publishes statistics in two ways:

   • Combined Data: All module data is combined into a single message and published to devices/{DEVICE_ID} (e.g., devices/computer1)
   • Module-Specific Data: For backward compatibility, each module's data is also published to devices/{MODULE_NAME}/{DEVICE_ID} (e.g., devices/gpu/computer1)

   This allows you to subscribe to whichever topic structure is most convenient for your use case.

Running the Application

The application can run in several modes:

Run both MQTT client and GUI simultaneously (recommended)

python main.py

This is the recommended way to run the application. It will start the MQTT client in a background thread and the GUI in the main thread. The application will automatically ensure the MQTT client is properly connected before the GUI becomes fully operational.

Run MQTT client and GUI separately

If needed, you can also start the MQTT client and GUI in separate terminals:

# Terminal 1: Start the MQTT client
python main.py --mqtt

# Terminal 2: Start the GUI (after MQTT client is running)
python main.py --gui

Run only the MQTT client

python main.py --mqtt

Run only the GUI

python main.py --gui

Specify a custom port

python main.py --port 8080

If the specified port is already in use, the application will automatically find an available port. The default port is 9876 if not specified in the .env file.

Enable debug mode

To enable debug mode, set MQTT_DEBUG=True in your .env file:

# In your .env file
MQTT_DEBUG=True

This enables verbose logging, showing detailed information about module loading, MQTT connections, and data processing. Useful for troubleshooting issues.

Running as a System Service

Option 1: Using the Installation Script (Recommended)

The easiest way to set up the application as a system service is to use the installation script:

# For Linux
sudo ./install.sh

# For macOS
./install.sh

The script will automatically create and enable the appropriate service for your platform (systemd on Linux, launchd on macOS).

Option 2: Manual Setup

To manually set up the application as a background service that starts automatically on boot:

For Linux (systemd):

1. Create a systemd service file:

sudo nano /etc/systemd/system/mqtt-device-monitor.service

2. Paste the following content (adjust paths and username as needed):

[Unit]
Description=MQTT Device Monitor Service
After=network.target

[Service]
User=your_username
WorkingDirectory=/path/to/mqtt-device-monitor
ExecStart=/path/to/mqtt-device-monitor/.venv/bin/python main.py
Restart=on-failure
Environment=PYTHONUNBUFFERED=1

[Install]
WantedBy=multi-user.target

3. Enable and start the service:

sudo systemctl daemon-reload
sudo systemctl enable mqtt-device-monitor.service
sudo systemctl start mqtt-device-monitor.service

4. Check the service status:

sudo systemctl status mqtt-device-monitor.service

5. View logs if needed:

journalctl -u mqtt-device-monitor.service -f

For macOS (launchd):

1. Create a launchd plist file:

nano ~/Library/LaunchAgents/com.mqtt-device-monitor.plist

2. Paste the following content (adjust paths as needed):

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>Label</key>
    <string>com.mqtt-device-monitor</string>
    <key>ProgramArguments</key>
    <array>
        <string>/path/to/mqtt-device-monitor/.venv/bin/python</string>
        <string>/path/to/mqtt-device-monitor/main.py</string>
    </array>
    <key>RunAtLoad</key>
    <true/>
    <key>KeepAlive</key>
    <true/>
    <key>StandardErrorPath</key>
    <string>/path/to/mqtt-device-monitor/error.log</string>
    <key>StandardOutPath</key>
    <string>/path/to/mqtt-device-monitor/output.log</string>
    <key>WorkingDirectory</key>
    <string>/path/to/mqtt-device-monitor</string>
</dict>
</plist>

3. Load the service:

launchctl load ~/Library/LaunchAgents/com.mqtt-device-monitor.plist

4. Check if the service is running:

launchctl list | grep mqtt-device-monitor

Platform Support

The application is designed to work across multiple platforms:

• Linux: Full support for CPU and NVIDIA GPU monitoring
• macOS:
  • Intel: CPU monitoring with limited GPU support
  • Apple Silicon (ARM64): Optimized CPU and GPU monitoring
• Windows: Basic CPU monitoring with NVIDIA GPU support

Architecture

The application uses a modular architecture with the following components:

MQTT Client

The MQTT client is responsible for:

• Collecting system metrics from various modules (CPU, GPU, etc.)
• Publishing metrics to the MQTT broker
• Subscribing to all topics on the MQTT broker
• Forwarding received messages to the GUI via a message broker

GUI

The GUI provides a web interface for:

• Visualizing MQTT messages in real-time
• Displaying device statistics
• Publishing custom MQTT messages
• Monitoring connection status

Message Broker

A central message broker handles communication between the MQTT client and GUI:

• The MQTT client handles all MQTT communication with the external broker
• The GUI receives data from the MQTT client via the message broker
• The GUI sends publish requests to the MQTT client via the message broker

This architecture centralizes all MQTT logic in the MQTT client component, making the system more maintainable and reducing the number of connections to the MQTT broker.

Debug Bar

The application includes a debug bar that provides real-time information about:

• MQTT connection status
• Message statistics
• Performance metrics
• Request details

To toggle the debug bar, click the 🪲 Debug button in the bottom right corner of the web interface.

Logging

The application uses a configurable logging system:

• Set DEBUG=true in the .env file to enable detailed logging
• In non-debug mode, only warnings and errors are logged
• Request logs from the web interface are suppressed by default
• MQTT message details are not logged to keep the console clean

MQTT Message Format

The application publishes JSON messages in two formats:

Combined Data Format (Topic: devices/{DEVICE_ID})

{
  "device_id": "computer1",
  "timestamp": 1623456789,
  "modules": {
    "gpu": {
      "gpu_util": 45,
      "mem_util": 30,
      "mem_total": 8192,
      "mem_used": 2458,
      "temp": 65
    },
    "cpu": {
      "cpu_util": 25,
      "per_cpu_util": [20, 30, 25, 22],
      "cpu_freq": 2500,
      "temp": 45
    }
  }
}

Module-Specific Format (Topic: devices/{MODULE_NAME}/{DEVICE_ID})

GPU Module

{
  "gpu_util": 45,
  "mem_util": 30,
  "mem_total": 8192,
  "mem_used": 2458,
  "temp": 65
}

CPU Module

{
  "cpu_util": 25,
  "per_cpu_util": [20, 30, 25, 22],
  "cpu_freq": 2500,
  "temp": 45
}

Subscribing to Monitors

You can subscribe to different levels using MQTT wildcards:

Combined Data (Device-First Structure):

• All devices: devices/#
• A specific device: devices/computer1

Module-Specific Data (Module-First Structure):

• All modules on all devices: devices/#
• All devices for a specific module: devices/gpu/#
• A specific device across all modules: devices/+/computer1
• A specific module on a specific device: devices/gpu/computer1