Skip to content
Open
4 changes: 2 additions & 2 deletions op-monitorism/cmd/monitorism/cli.go
Original file line number Diff line number Diff line change
Expand Up @@ -38,8 +38,8 @@ func newCli(GitCommit string, GitDate string) *cli.App {
Commands: []*cli.Command{
{
Name: "multisig",
Usage: "Monitors OptimismPortal pause status, Safe nonce, and Pre-Signed nonce stored in 1Password",
Description: "Monitors OptimismPortal pause status, Safe nonce, and Pre-Signed nonce stored in 1Password",
Usage: "Monitors multisig essential values (Threshold, Signer counts number, Signer lists, Balance) from a notion database.",
Description: "Monitors multisig essential values (Threshold, Signer counts number, Signer lists, Balance) from a notion database.",
Flags: append(multisig.CLIFlags("MULTISIG_MON"), defaultFlags...),
Action: cliapp.LifecycleCmd(MultisigMain),
},
Expand Down
2 changes: 1 addition & 1 deletion op-monitorism/go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ require (
github.com/prometheus/client_model v0.6.1
github.com/stretchr/testify v1.10.0
github.com/urfave/cli/v2 v2.27.5
golang.org/x/exp v0.0.0-20241009180824-f66d83c29e7c
gopkg.in/yaml.v3 v3.0.1
)

Expand Down Expand Up @@ -74,7 +75,6 @@ require (
github.com/xrash/smetrics v0.0.0-20240521201337-686a1a2994c1 // indirect
github.com/yusufpapurcu/wmi v1.2.3 // indirect
golang.org/x/crypto v0.32.0 // indirect
golang.org/x/exp v0.0.0-20241009180824-f66d83c29e7c // indirect
golang.org/x/net v0.34.0 // indirect
golang.org/x/sync v0.10.0 // indirect
golang.org/x/sys v0.29.0 // indirect
Expand Down
183 changes: 174 additions & 9 deletions op-monitorism/multisig/README.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,179 @@
### Multisig Monitor
### Multisig Registry Monitor

The multisig monitor reports the paused status of the `OptimismPortal` contract. If set, the latest nonce of the configured `Safe` address. And also if set, the latest presigned nonce stored in One Password. The latest presigned nonce is identified by looking for items in the configured vault that follow a `ready-<nonce>.json` name. The highest nonce of this item name format is reported.
The Multisig Registry Monitor is a service designed to continuously monitor Gnosis Safe multisig wallets by comparing their onchain state with records stored in a Notion database. The service validates threshold settings, signer counts, and balance levels to ensure multisig configurations remain accurate and secure.

- **NOTE**: In order to read from one password, the `OP_SERVICE_ACCOUNT_TOKEN` environment variable must be set granting the process permission to access the specified vault.
The service provides real-time alerting through webhooks and comprehensive metrics for monitoring multisig wallet health across networks.

⚠️ The service requires valid Notion API credentials and database access ⚠️

## 1. Usage

### 1. Run Monitoring Service

To start the multisig monitoring service, use the following command:

```shell
go run ../cmd/monitorism multisig --notion.database.id 24ffb7d8d2cc80e8885ee1bb3bc1f53b --notion.token secret_abc123 --l1.node.url https://mainnet.infura.io/v3/your-key --nickname multisig_registry
```
OPTIONS:
--l1.node.url value [$MULTISIG_MON_L1_NODE_URL] Node URL of L1 peer (default: "127.0.0.1:8545")
--optimismportal.address value [$MULTISIG_MON_OPTIMISM_PORTAL] Address of the OptimismPortal contract
--nickname value [$MULTISIG_MON_NICKNAME] Nickname of chain being monitored
--safe.address value [$MULTISIG_MON_SAFE] Address of the Safe contract
--op.vault value [$MULTISIG_MON_1PASS_VAULT_NAME] 1Pass Vault name storing presigned safe txs following a 'ready-<nonce>.json' item name format

**Core Monitoring Features:**

| Feature | Description |
| ------- | ----------- |
| Threshold Validation | Compares onchain Safe threshold with Notion records |
| Signer Count Validation | Verifies number of Safe owners matches Notion data |
| Balance Monitoring | Tracks native token balances in USD with risk assessment |
| Risk Level Assessment | Automatically categorizes Safes based on balance thresholds |
| Webhook Alerts | Sends formatted alerts to Discord/Slack for anomalies |

### 2. Configuration Options

**Required Arguments:**
| Argument | Example Value | Explanation |
| -------- | ------------- | ----------- |
| `--notion.database.id` | 24ffb7d8d2cc80e8885ee1bb3bc1f53b | Notion database ID containing Safe records |
| `--notion.token` | secret_abc123 | Notion integration token (API key) |
| `--l1.node.url` | https://mainnet.infura.io/v3/your-key | Ethereum RPC node URL |
| `--nickname` | mainnet | Network identifier for metrics labeling |

**Optional Arguments:**
| Argument | Default Value | Explanation |
| -------- | ------------- | ----------- |
| `--webhook.url` | "" | Webhook URL for sending alerts (Discord/Slack) |
| `--high.value.threshold.usd` | 1000000 | USD threshold for high-value Safe validation ($1M) |

### 3. Notion Database Schema

The service expects a Notion database with the following properties:
**Required Columns:**
| Column Name | Type | Description |
| ----------- | ---- | ----------- |
| Name | title | Safe wallet name |
| Address | text | Ethereum address of the Safe |
| Threshold | number | Required number of signatures |
| Signer count | number | Total number of Safe owners |
| Risk Band | select | Risk level (Critical, High, Medium, Low) |

**Optional Columns:**
| Column Name | Type | Description |
| ----------- | ---- | ----------- |
| Networks | multi-select | Supported networks |
| Multisig Lead | people | Responsible team members |
| Has Monitoring | checkbox | Monitoring status |
| Has Backup Chat | checkbox | Backup chat status |
| Last Review By | people | Last Reviewer of the Template |
| Last Review Date | date | Last Review date by a Reviewer |


### 4. Alert Examples

The service automatically sends alerts for various conditions:

**Threshold Mismatch:**
<img width="1081" height="330" alt="9e1522fd9f2b24d8e9be96f43531b4e00ca0cb17b00a1647834695205988b29d" src="https://github.com/user-attachments/assets/c7759302-10a2-4f20-8ffb-4297db4e1fea" />


**Signer Mismatch:**
<img width="924" height="281" alt="61600282cec0e8bc627e4919b8af31fe185b1fa4b8413d3605d09731fa54839d" src="https://github.com/user-attachments/assets/8dd5f9a1-5c84-4505-99e2-6701c07c9ba1" />



**Safe Balance Criticity Alerts:**
<img width="1086" height="334" alt="e2e8a6d8640404b48baa6a268dc1c52291eba1867d1c42487f510618e78e7de5" src="https://github.com/user-attachments/assets/4c813df2-23b9-45bf-9230-67345ff97b20" />



### 5. Metrics Server

The service exposes Prometheus metrics for monitoring:

**Key Metrics:**
```golang
multisig_registry_threshold_mismatch // 1 if mismatch detected, 0 if matches
multisig_registry_signer_count_mismatch // 1 if signer count differs, 0 if matches
multisig_registry_safe_native_balance_eth // ETH balance of each Safe
multisig_registry_safe_native_balance_usd // USD balance of each Safe
multisig_registry_safe_risk_level // Risk level: 1=low, 2=medium, 3=high, 4=critical
multisig_registry_safe_accessible // 1 if Safe accessible, 0 if not
multisig_registry_unexpected_errors // Counter for various error types
multisig_registry_total_safes_monitored // Total number of Safes being monitored
```

### 7. Price Data Sources

ETH price fetching includes automatic failover:

1. **Primary:** CoinGecko API
2. **Fallback:** Binance API

If the primary source fails, the service automatically switches to the backup source to ensure continuous monitoring.
You can also add more custom feed if necessary for more robusteness.

### 8. Command Examples

**Basic Monitoring (Mainnet):**
```shell
./monitorism multisig \
--notion.database.id=your-database-id \
--notion.token=secret_your-token \
--l1.node.url=https://mainnet.infura.io/v3/your-key \
--nickname=mainnet
```

**With Webhook Alerts:**
```shell
./monitorism multisig \
--notion.database.id=your-database-id \
--notion.token=secret_your-token \
--l1.node.url=https://mainnet.infura.io/v3/your-key \
--nickname=mainnet \
--webhook.url=https://hooks.slack.com/services/your/webhook/url
```

**Custom High-Value Threshold ($5M):**
```shell
./monitorism multisig \
--notion.database.id=your-database-id \
--notion.token=secret_your-token \
--l1.node.url=https://mainnet.infura.io/v3/your-key \
--nickname=mainnet \
--high.value.threshold.usd=5000000
```

**Using Environment Variables:**
```shell
export OP_MONITORISM_NOTION_DATABASE_ID=your-database-id
export OP_MONITORISM_NOTION_TOKEN=secret_your-token
export OP_MONITORISM_WEBHOOK_URL=https://hooks.slack.com/services/your/webhook/url
export OP_MONITORISM_HIGH_VALUE_THRESHOLD_USD=2000000

./monitorism multisig --l1.node.url=https://mainnet.infura.io/v3/your-key --nickname=mainnet
```

### 9. Options and Configuration

Using the `--help` flag will show all available options:

**OPTIONS:**

```shell
--l1.node.url value Node URL of L1 peer (default: "127.0.0.1:8545") [$OP_MONITORISM_L1_NODE_URL]
--nickname value Nickname of chain being monitored [$OP_MONITORISM_NICKNAME]
--notion.database.id value Notion database ID containing Safe records [$OP_MONITORISM_NOTION_DATABASE_ID]
--notion.token value Notion integration token (API key) [$OP_MONITORISM_NOTION_TOKEN]
--webhook.url value Webhook URL for sending alerts (optional) [$OP_MONITORISM_WEBHOOK_URL]
--high.value.threshold.usd value USD threshold for high-value Safe validation (default: 1000000) [$OP_MONITORISM_HIGH_VALUE_THRESHOLD_USD]
--log.level value The lowest log level that will be output (default: INFO) [$OP_MONITORISM_LOG_LEVEL]
--log.format value Format the log output. Supported formats: 'text', 'terminal', 'logfmt', 'json', 'json-pretty' (default: text) [$OP_MONITORISM_LOG_FORMAT]
--log.color Color the log output if in terminal mode (default: false) [$OP_MONITORISM_LOG_COLOR]
--help, -h show help
```


For additional support, check the logs with `--log.level=DEBUG` for detailed troubleshooting information.

## TODO
- [ ] Add an argument for the pricefeed with an array and passing more API easily.
- [ ] Add more network that (Ethereum L1).
- [ ] Compute the balance also on the token and not only the native token.

93 changes: 50 additions & 43 deletions op-monitorism/multisig/cli.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,54 +5,55 @@ import (

opservice "github.com/ethereum-optimism/optimism/op-service"

"github.com/ethereum/go-ethereum/common"

"github.com/urfave/cli/v2"
)

const (
L1NodeURLFlagName = "l1.node.url"
NicknameFlagName = "nickname"

// Notion flags
NotionDatabaseIDFlagName = "notion.database.id"
NotionTokenFlagName = "notion.token"

NicknameFlagName = "nickname"
OptimismPortalAddressFlagName = "optimismportal.address"
SafeAddressFlagName = "safe.address"
OnePassVaultFlagName = "op.vault"
// Webhook flags
WebhookURLFlagName = "webhook.url"

// Risk threshold flags
HighValueThresholdFlagName = "high.value.threshold.usd"
)

type CLIConfig struct {
L1NodeURL string
Nickname string
OptimismPortalAddress common.Address
L1NodeURL string
Nickname string

// Notion configuration (required)
NotionDatabaseID string
NotionToken string

// Optional
SafeAddress *common.Address
OnePassVault *string
// Webhook configuration (optional)
WebhookURL string

// Risk threshold configuration
HighValueThresholdUSD int
}

func ReadCLIFlags(ctx *cli.Context) (CLIConfig, error) {
cfg := CLIConfig{
L1NodeURL: ctx.String(L1NodeURLFlagName),
Nickname: ctx.String(NicknameFlagName),
}

portalAddress := ctx.String(OptimismPortalAddressFlagName)
if !common.IsHexAddress(portalAddress) {
return cfg, fmt.Errorf("--%s is not a hex-encoded address", OptimismPortalAddressFlagName)
L1NodeURL: ctx.String(L1NodeURLFlagName),
Nickname: ctx.String(NicknameFlagName),
NotionDatabaseID: ctx.String(NotionDatabaseIDFlagName),
NotionToken: ctx.String(NotionTokenFlagName),
WebhookURL: ctx.String(WebhookURLFlagName),
HighValueThresholdUSD: ctx.Int(HighValueThresholdFlagName),
}
cfg.OptimismPortalAddress = common.HexToAddress(portalAddress)

safeAddress := ctx.String(SafeAddressFlagName)
if len(safeAddress) > 0 {
if !common.IsHexAddress(safeAddress) {
return cfg, fmt.Errorf("--%s is not a hex-encoded address", SafeAddressFlagName)
}
addr := common.HexToAddress(safeAddress)
cfg.SafeAddress = &addr
// Notion validation
if cfg.NotionDatabaseID == "" {
return cfg, fmt.Errorf("--%s is required", NotionDatabaseIDFlagName)
}

onePassVault := ctx.String(OnePassVaultFlagName)
if len(onePassVault) > 0 {
cfg.OnePassVault = &onePassVault
if cfg.NotionToken == "" {
return cfg, fmt.Errorf("--%s is required", NotionTokenFlagName)
}

return cfg, nil
Expand All @@ -66,27 +67,33 @@ func CLIFlags(envVar string) []cli.Flag {
Value: "127.0.0.1:8545",
EnvVars: opservice.PrefixEnvVar(envVar, "L1_NODE_URL"),
},
&cli.StringFlag{
Name: OptimismPortalAddressFlagName,
Usage: "Address of the OptimismPortal contract",
EnvVars: opservice.PrefixEnvVar(envVar, "OPTIMISM_PORTAL"),
Required: true,
},
&cli.StringFlag{
Name: NicknameFlagName,
Usage: "Nickname of chain being monitored",
EnvVars: opservice.PrefixEnvVar(envVar, "NICKNAME"),
Required: true,
},
&cli.StringFlag{
Name: SafeAddressFlagName,
Usage: "Address of the Safe contract",
EnvVars: opservice.PrefixEnvVar(envVar, "SAFE"),
Name: NotionDatabaseIDFlagName,
Usage: "Notion database ID containing Safe records",
EnvVars: opservice.PrefixEnvVar(envVar, "NOTION_DATABASE_ID"),
Required: true,
},
&cli.StringFlag{
Name: OnePassVaultFlagName,
Usage: "1Pass vault name storing presigned safe txs following a 'ready-<nonce>.json' item name format",
EnvVars: opservice.PrefixEnvVar(envVar, "1PASS_VAULT_NAME"),
Name: NotionTokenFlagName,
Usage: "Notion integration token (API key)",
EnvVars: opservice.PrefixEnvVar(envVar, "NOTION_TOKEN"),
Required: true,
}, &cli.StringFlag{
Name: WebhookURLFlagName,
Usage: "Webhook URL for sending alerts (optional)",
EnvVars: opservice.PrefixEnvVar(envVar, "WEBHOOK_URL"),
},
&cli.IntFlag{
Name: HighValueThresholdFlagName,
Usage: "USD threshold for high-value Safe validation (e.g., 1000000 for $1M)",
Value: 1000000, // Default to $1M
EnvVars: opservice.PrefixEnvVar(envVar, "HIGH_VALUE_THRESHOLD_USD"),
},
}
}
Loading