Skip to content

Webhook Integration

Jump to bottom
Eric Fitzgerald edited this page Jan 24, 2026 · 4 revisions

Webhook Integration

This page explains how to set up and use TMI's webhook system to receive real-time notifications when threat models, diagrams, and other resources change.

Overview

TMI's webhook system allows you to subscribe to events and receive HTTP POST notifications when those events occur. This enables real-time integrations with external systems.

Common use cases:

  • Notify team channels (Slack, Teams) when threats are identified
  • Create tickets in issue trackers when threats are created
  • Trigger CI/CD pipelines when threat models are updated
  • Send alerts to SIEM systems for critical threats
  • Archive threat models to external storage
  • Synchronize threat data with other security tools

Architecture

The webhook system uses a worker-based architecture for reliable delivery:

Event Occurs → Redis Stream → Worker Processes → HTTP POST → Your Endpoint
                                                             ↓
                                                   Status Callback (optional)

Components:

  1. Event Emitter: Publishes events to Redis Streams when resources change
  2. Event Consumer: Reads events from stream and creates delivery records
  3. Challenge Worker: Verifies new subscriptions using challenge-response
  4. Delivery Worker: Delivers webhooks with retry and exponential backoff
  5. Cleanup Worker: Removes old delivery records and inactive subscriptions

See the operator webhook configuration guide for deployment details.

Quick Start

There is an example webhook application, written in Python and designed to be hosted in AWS Lambda, available here that you can clone/fork and use as a basis to implement your own webhook applications.

1. Create a Webhook Endpoint

Your endpoint must accept POST requests and respond with 200 OK:

Minimal example (Express.js):

const express = require('express');
const app = express();

app.use(express.json());

app.post('/webhooks/tmi', (req, res) => {
    const event = req.body;
    console.log('Received event:', event.event_type, event);

    // Process event asynchronously
    processEvent(event).catch(console.error);

    // Respond immediately
    res.status(200).send('OK');
});

app.listen(3000);

Important: Always respond quickly (within 30 seconds) to avoid timeouts.

2. Register Webhook Subscription

Create a webhook subscription via the API:

curl -X POST https://api.tmi.dev/webhook/subscriptions \
  -H "Authorization: Bearer $YOUR_JWT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "My Integration",
    "url": "https://your-domain.com/webhooks/tmi",
    "events": ["threat_model.created", "threat_model.updated"],
    "secret": "your-shared-secret-for-hmac"
  }'

Response:

{
  "id": "550e8400-e29b-41d4-a716-446655440000",
  "status": "pending_verification",
  "name": "My Integration",
  "url": "https://your-domain.com/webhooks/tmi",
  "events": ["threat_model.created", "threat_model.updated"]
}

3. Respond to Challenge

TMI will send a challenge request to verify your endpoint:

Challenge request:

{
  "type": "webhook.challenge",
  "challenge": "abc123def456"
}

Required response:

{
  "challenge": "abc123def456"
}

Example handler:

app.post('/webhooks/tmi', (req, res) => {
    const event = req.body;

    // Handle challenge
    if (event.type === 'webhook.challenge') {
        return res.json({ challenge: event.challenge });
    }

    // Handle other events
    processEvent(event);
    res.status(200).send('OK');
});

After successful verification, your subscription becomes active and starts receiving events.

Event Types

Threat Model Events

threat_model.created:

{
  "event_type": "threat_model.created",
  "threat_model_id": "550e8400-e29b-41d4-a716-446655440000",
  "resource_id": "550e8400-e29b-41d4-a716-446655440000",
  "resource_type": "threat_model",
  "owner_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
  "timestamp": "2025-01-15T14:30:00Z",
  "data": {
    "name": "Production API Threat Model",
    "description": "Threat model for production API services",
    "owner": "security-team@example.com"
  }
}

threat_model.updated:

{
  "event_type": "threat_model.updated",
  "threat_model_id": "550e8400-e29b-41d4-a716-446655440000",
  "resource_id": "550e8400-e29b-41d4-a716-446655440000",
  "resource_type": "threat_model",
  "owner_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
  "timestamp": "2025-01-15T15:45:00Z",
  "data": {
    "name": "Production API Threat Model - Updated",
    "description": "Updated threat model",
    "modified_at": "2025-01-15T15:45:00Z"
  }
}

threat_model.deleted:

{
  "event_type": "threat_model.deleted",
  "threat_model_id": "550e8400-e29b-41d4-a716-446655440000",
  "resource_id": "550e8400-e29b-41d4-a716-446655440000",
  "resource_type": "threat_model",
  "owner_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
  "timestamp": "2025-01-15T16:00:00Z",
  "data": {
    "name": "Production API Threat Model"
  }
}

Diagram Events

diagram.created, diagram.updated, diagram.deleted:

{
  "event_type": "diagram.created",
  "threat_model_id": "550e8400-e29b-41d4-a716-446655440000",
  "resource_id": "7d8f6e5c-4b3a-2190-8765-fedcba987654",
  "resource_type": "diagram",
  "owner_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
  "timestamp": "2025-01-15T14:35:00Z",
  "data": {
    "title": "System Architecture",
    "diagram_type": "data_flow"
  }
}

Document Events

document.created, document.updated, document.deleted:

{
  "event_type": "document.created",
  "threat_model_id": "550e8400-e29b-41d4-a716-446655440000",
  "resource_id": "abc-123-def-456",
  "resource_type": "document",
  "owner_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
  "timestamp": "2025-01-15T14:40:00Z",
  "data": {
    "name": "Security Requirements.pdf",
    "content_type": "application/pdf",
    "size": 524288
  }
}

Security

URL Validation

All webhook URLs must:

  • Use HTTPS (required in production)
  • Have valid DNS hostname
  • Pass deny list checks

Blocked patterns:

  • Localhost: 127.0.0.1, ::1, localhost
  • Private IPs: 10.*, 192.168.*, 172.16-31.*
  • Link-local: 169.254.*, fe80::
  • Cloud metadata endpoints (AWS, GCP, Azure, etc.)

Administrators can add custom deny list patterns via the deny list API.

HMAC Signature Verification

Always verify signatures to ensure webhooks are from TMI:

Signature header:

X-Webhook-Signature: sha256=5d41402abc4b2a76b9719d911017c592

Verification example (Node.js):

const crypto = require('crypto');

function verifyWebhook(payloadBody, signature, secret) {
    const hmac = crypto.createHmac('sha256', secret);
    hmac.update(payloadBody);
    const expectedSig = `sha256=${hmac.digest('hex')}`;

    // Constant-time comparison
    return crypto.timingSafeEqual(
        Buffer.from(signature),
        Buffer.from(expectedSig)
    );
}

app.post('/webhooks/tmi', (req, res) => {
    const signature = req.headers['x-webhook-signature'];
    const payloadBody = JSON.stringify(req.body);

    if (!verifyWebhook(payloadBody, signature, WEBHOOK_SECRET)) {
        return res.status(401).send('Invalid signature');
    }

    // Process webhook...
    res.status(200).send('OK');
});

Verification example (Python):

import hmac
import hashlib

def verify_webhook(payload_body, signature, secret):
    """Verify webhook HMAC signature"""
    expected = hmac.new(
        secret.encode('utf-8'),
        payload_body.encode('utf-8'),
        hashlib.sha256
    ).hexdigest()

    expected_sig = f"sha256={expected}"

    # Constant-time comparison
    return hmac.compare_digest(signature, expected_sig)

@app.route('/webhooks/tmi', methods=['POST'])
def handle_webhook():
    signature = request.headers.get('X-Webhook-Signature')
    payload_body = request.get_data(as_text=True)

    if not verify_webhook(payload_body, signature, WEBHOOK_SECRET):
        return 'Unauthorized', 401

    # Process webhook...
    return 'OK', 200

Webhook Headers

Every webhook request includes:

POST /webhooks/tmi HTTP/1.1
Host: your-domain.com
Content-Type: application/json
X-Webhook-Event: threat_model.created
X-Webhook-Delivery-Id: 7fa85f64-5717-4562-b3fc-2c963f66afa6
X-Webhook-Subscription-Id: 7d8f6e5c-4b3a-2190-8765-fedcba987654
X-Webhook-Signature: sha256=5d41402abc4b2a76b9719d911017c592
User-Agent: TMI-Webhook/1.0

Response Headers for Addon Invocations

When your webhook handles addon invocations (X-Webhook-Event: addon.invoked), you can control how TMI processes the completion status using the X-TMI-Callback response header:

  • No header or any value except async: TMI auto-completes the invocation as completed
  • X-TMI-Callback: async: TMI marks invocation as in_progress, expecting your service to call back with status updates

See Addon-System#callback-modes for detailed documentation on callback modes.

Delivery Guarantees

Retry Logic

Failed deliveries are retried with exponential backoff:

  1. Attempt 1: Immediate
  2. Attempt 2: After 1 minute
  3. Attempt 3: After 5 minutes
  4. Attempt 4: After 15 minutes
  5. Attempt 5: After 30 minutes

After 5 attempts, delivery is marked failed.

Success Criteria

A delivery succeeds when:

  • HTTP status code is 2xx (200-299)
  • Response received within 30 seconds

Idempotency

Use X-Webhook-Delivery-Id header to detect duplicate deliveries:

const processedDeliveries = new Set();

app.post('/webhooks/tmi', (req, res) => {
    const deliveryId = req.headers['x-webhook-delivery-id'];

    // Check if already processed
    if (processedDeliveries.has(deliveryId)) {
        console.log('Duplicate delivery, skipping');
        return res.status(200).send('OK');
    }

    // Process webhook
    processEvent(req.body);

    // Mark as processed
    processedDeliveries.add(deliveryId);

    res.status(200).send('OK');
});

Rate Limits

Default Quotas (Per Owner)

  • Max Subscriptions: 10 active subscriptions
  • Subscription Creation: 10 requests/minute, 20 requests/day
  • Event Publication: 12 events/minute

Custom Quotas

Administrators can configure custom quotas:

curl -X POST https://api.tmi.dev/webhook/quotas \
  -H "Authorization: Bearer $ADMIN_JWT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "owner_id": "user-uuid",
    "max_subscriptions": 50,
    "max_subscription_requests_per_minute": 10,
    "max_subscription_requests_per_day": 500,
    "max_events_per_minute": 1000
  }'

API Endpoints

Subscription Management

Create subscription:

POST /webhook/subscriptions

List subscriptions:

GET /webhook/subscriptions

Get subscription details:

GET /webhook/subscriptions/{id}

Delete subscription:

DELETE /webhook/subscriptions/{id}

Delivery History

List deliveries:

GET /webhook/deliveries

Get delivery details:

GET /webhook/deliveries/{id}

Admin Endpoints

Manage deny list:

GET /webhook/deny-list
POST /webhook/deny-list
DELETE /webhook/deny-list/{id}

Manage quotas:

GET /webhook/quotas/{owner_id}
POST /webhook/quotas
DELETE /webhook/quotas/{owner_id}

See REST-API-Reference for complete API documentation.

Best Practices

Endpoint Implementation

  1. Respond Quickly: Return 200 OK immediately, process asynchronously

    app.post('/webhooks/tmi', (req, res) => {
    queue.add('process-webhook', req.body);
    res.status(200).send('OK');
});

  2. Implement Idempotency: Use delivery ID to prevent duplicate processing

  3. Verify Signatures: Always verify HMAC signatures

  4. Handle Errors Gracefully: Return 2xx for success, 4xx/5xx for errors

  5. Log Everything: Log all webhook receipts for debugging

Security

  1. HTTPS Only: Never expose HTTP endpoints (TMI will only accept https endpoints when not in development mode)

  2. Strong Secrets: Use strong, random secrets (min 32 characters)

    # Generate strong secret
openssl rand -base64 32

  3. IP Allowlisting: Consider restricting to TMI server IPs

  4. Input Validation: Validate all incoming data

  5. Rate Limiting: Implement rate limiting on your endpoint

Monitoring

  1. Track Failures: Monitor subscription publication_failures

  2. Check Last Success: Monitor last_successful_use timestamp

  3. Query Deliveries: Review delivery status for debugging

    curl https://api.tmi.dev/webhook/deliveries \
  -H "Authorization: Bearer $TOKEN"

  4. Set Up Alerts: Alert on high failure rates

Performance

  1. Async Processing: Process webhooks asynchronously

  2. Queue System: Use message queue (RabbitMQ, Redis, SQS)

  3. Batch Operations: Batch database operations when possible

  4. Caching: Cache frequently accessed data

  5. Horizontal Scaling: Deploy multiple webhook handler instances

Integration Examples

Slack Notifications

Send threat notifications to Slack:

const { WebClient } = require('@slack/web-api');
const slack = new WebClient(process.env.SLACK_TOKEN);

async function notifySlack(event) {
    if (event.event_type === 'threat_model.created') {
        await slack.chat.postMessage({
            channel: '#security',
            text: `New threat model created: ${event.data.name}`,
            blocks: [
                {
                    type: 'section',
                    text: {
                        type: 'mrkdwn',
                        text: `*New Threat Model*\n${event.data.name}\n${event.data.description}`
                    }
                },
                {
                    type: 'actions',
                    elements: [
                        {
                            type: 'button',
                            text: { type: 'plain_text', text: 'View in TMI' },
                            url: `https://tmi.dev/threat-models/${event.threat_model_id}`
                        }
                    ]
                }
            ]
        });
    }
}

Email Notifications

Send email when critical threats are identified:

import smtplib
from email.mime.text import MIMEText

def send_email_notification(event):
    if event['event_type'] == 'threat.created':
        threat = event['data']

        if threat.get('severity') == 'critical':
            msg = MIMEText(f"""
            Critical threat identified:

            Title: {threat['title']}
            Description: {threat['description']}
            Threat Model: {threat['threat_model_name']}

            View in TMI: https://tmi.dev/threats/{event['resource_id']}
            """)

            msg['Subject'] = f"[CRITICAL] {threat['title']}"
            msg['From'] = 'security@example.com'
            msg['To'] = 'security-team@example.com'

            with smtplib.SMTP('smtp.example.com', 587) as smtp:
                smtp.starttls()
                smtp.login('username', 'password')
                smtp.send_message(msg)

Logging to SIEM

Forward events to SIEM system:

import requests

def forward_to_siem(event):
    """Forward webhook event to SIEM"""
    siem_endpoint = 'https://siem.example.com/api/events'

    siem_event = {
        'source': 'TMI',
        'event_type': event['event_type'],
        'timestamp': event['timestamp'],
        'severity': map_severity(event),
        'resource_id': event['resource_id'],
        'resource_type': event['resource_type'],
        'owner_id': event['owner_id'],
        'data': event['data']
    }

    response = requests.post(
        siem_endpoint,
        json=siem_event,
        headers={'Authorization': f'Bearer {SIEM_TOKEN}'}
    )
    response.raise_for_status()

Troubleshooting

Subscription Not Receiving Events

Checks:

  1. Subscription status is active (not pending_verification)
  2. Event types in subscription match events being triggered
  3. Threat model filter (threat_model_id) is correct
  4. Rate limit not exceeded
  5. Endpoint is responding with 2xx status

Debug:

# Check subscription status
curl https://api.tmi.dev/webhook/subscriptions/{id} \
  -H "Authorization: Bearer $TOKEN"

# Check recent deliveries
curl https://api.tmi.dev/webhook/deliveries \
  -H "Authorization: Bearer $TOKEN"

Verification Failed

Common issues:

  1. Endpoint not returning challenge in response body
  2. Response takes >30 seconds
  3. Non-2xx HTTP status code
  4. Network connectivity issues

Fix:

// Correct challenge response
app.post('/webhooks/tmi', (req, res) => {
    if (req.body.type === 'webhook.challenge') {
        return res.json({ challenge: req.body.challenge });
    }
    // ...
});

High Failure Rate

Diagnose:

# Check failure reasons
curl https://api.tmi.dev/webhook/deliveries \
  -H "Authorization: Bearer $TOKEN" | jq '.[] | select(.status=="failed")'

Common causes:

  1. Endpoint unavailable or slow
  2. Invalid HMAC signature verification
  3. Endpoint returning 4xx/5xx errors
  4. Network issues (firewall, DNS)

Solutions:

  1. Improve endpoint reliability
  2. Fix signature verification logic
  3. Return 2xx for successful processing
  4. Check network connectivity

Webhook Operations (for Operators)

This section covers operational management of the webhook system.

Worker Configuration

TMI runs four background workers for webhook processing:

Worker Interval Source File
Event Consumer 5-second block api/webhook_event_consumer.go
Challenge Worker 30 seconds api/webhook_challenge_worker.go
Delivery Worker 2 seconds api/webhook_delivery_worker.go
Cleanup Worker 1 hour api/webhook_cleanup_worker.go

Workers start automatically when the server starts with Redis and PostgreSQL available.

Default Quotas

Per-owner webhook quotas (from api/webhook_store.go):

Limit Default Value
Max Subscriptions 10
Max Events Per Minute 12
Max Subscription Requests Per Minute 10
Max Subscription Requests Per Day 20

Redis Configuration

The webhook system uses Redis Streams:

# Check stream exists
XINFO STREAM tmi:events

# Check consumer groups
XINFO GROUPS tmi:events
# Expected group: webhook-consumers

# View rate limit keys
KEYS webhook:ratelimit:sub:*
KEYS webhook:ratelimit:events:*

Database Tables

Webhook tables (created by migration 002_business_domain.up.sql):

  • webhook_subscriptions - Subscription definitions
  • webhook_deliveries - Delivery records and status
  • webhook_quotas - Per-owner rate limits
  • webhook_url_deny_list - SSRF prevention patterns

Monitoring Queries

-- Active subscriptions
SELECT COUNT(*) FROM webhook_subscriptions WHERE status = 'active';

-- Delivery status breakdown
SELECT status, COUNT(*) as count
FROM webhook_deliveries
GROUP BY status;

-- High failure rate subscriptions
SELECT id, name, url, publication_failures, last_successful_use
FROM webhook_subscriptions
WHERE status = 'active' AND publication_failures > 5
ORDER BY publication_failures DESC;

-- Pending deliveries (backlog)
SELECT COUNT(*) as pending_count
FROM webhook_deliveries
WHERE status = 'pending';

Log Patterns

Monitor webhook activity:

# Event consumer activity
grep "webhook-consumer" /var/log/tmi/tmi.log

# Challenge verification
grep "challenge worker" /var/log/tmi/tmi.log

# Delivery attempts
grep "delivering webhook" /var/log/tmi/tmi.log

# Worker errors
grep "ERROR.*webhook" /var/log/tmi/tmi.log

For complete operational documentation, see the Webhook Operations Guide.

Related Documentation

Home

Getting Started

Deployment

Operation

Troubleshooting

Development

Integrations

Tools

API Reference

Reference

Clone this wiki locally