feat(devops): add out-of-the-box prometheus metrics and grafana observability stack#9344
feat(devops): add out-of-the-box prometheus metrics and grafana observability stack#9344vishva-ux wants to merge 1 commit into
Conversation
vishva-ux
requested review from
dheeru0198,
mguptahub and
pablohashescobar
as code owners
📝 WalkthroughWalkthroughThis PR adds Prometheus-based monitoring for the API service. It integrates django-prometheus middleware and metrics endpoint into the Django app, and adds Prometheus and Grafana services with provisioning configuration and a dashboard to the local Docker Compose stack. ChangesPrometheus and Grafana monitoring integration
Estimated code review effort: 2 (Simple) | ~12 minutes Sequence Diagram(s)sequenceDiagram
participant PlaneAPI
participant Prometheus
participant Grafana
Prometheus->>PlaneAPI: Scrape /metrics on api:8000
PlaneAPI-->>Prometheus: Return request rate/latency metrics
Grafana->>Prometheus: Query metrics via datasource
Prometheus-->>Grafana: Return time-series data
Grafana-->>Grafana: Render plane-dashboard.json panels
Estimated code review effortEstimated code review effort: 2 (Simple) | ~12 minutes Suggested labelsSuggested labels: devops, monitoring, infrastructure Suggested reviewersSuggested reviewers: DevOps/infrastructure maintainers familiar with the Docker Compose and API settings Poem: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (2)
apps/api/requirements/base.txt (1)
78-80: 🔒 Security & Privacy | 🔵 Trivial | ⚡ Quick winBump django-prometheus to the current stable release
django-prometheus==2.3.1is several releases behind; the current stable release is 2.5.0. If there isn’t a compatibility reason to stay pinned here, update the version to keep the dependency current.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/api/requirements/base.txt` around lines 78 - 80, The dependency pin for django-prometheus in the requirements file is outdated. Update the version in the existing django-prometheus entry to the current stable release, keeping the package name and pin format intact, and make sure any related dependency constraints in apps/api/requirements/base.txt remain consistent if there are compatibility concerns.docker-compose-local.yml (1)
166-168: 🔒 Security & Privacy | 🔵 Trivial | ⚡ Quick winHardcoded Grafana admin credential inconsistent with rest of file.
Every other service here (plane-mq, plane-minio, plane-db) sources credentials via
env_file/${VAR}interpolation, but Grafana's admin password is hardcoded asadmindirectly in the compose file.🔧 Proposed fix
environment: - - GF_SECURITY_ADMIN_PASSWORD=admin + - GF_SECURITY_ADMIN_PASSWORD=${GRAFANA_ADMIN_PASSWORD:-admin} - GF_USERS_ALLOW_SIGN_UP=false🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docker-compose-local.yml` around lines 166 - 168, The Grafana service is using a hardcoded admin password in its environment block, unlike the other services that source secrets from env files or variable interpolation. Update the Grafana configuration to use an interpolated variable or env_file-backed value for GF_SECURITY_ADMIN_PASSWORD, and keep the existing GF_USERS_ALLOW_SIGN_UP setting unchanged so the credential handling matches the rest of the compose file.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@apps/api/plane/urls.py`:
- Line 23: The Prometheus metrics endpoint is currently mounted publicly through
the root include in the URL configuration, so update the URL wiring in the
`urlpatterns` setup to stop exposing `django_prometheus.urls` to unauthenticated
external callers. Move the metrics route behind an internal-only path or protect
it with access control at the proxy/app layer, and make sure the change is
applied where the `include("django_prometheus.urls")` entry is registered.
In `@deployments/grafana/provisioning/datasources/datasource.yml`:
- Around line 3-8: The Prometheus datasource provisioning in datasource.yml is
missing a fixed uid, which can break dashboard bindings that reference the
datasource by uid. Update the datasource entry to set the uid explicitly to
Prometheus so the provisioned datasource matches the references used by
plane-dashboard.json and remains stable across Grafana restarts.
---
Nitpick comments:
In `@apps/api/requirements/base.txt`:
- Around line 78-80: The dependency pin for django-prometheus in the
requirements file is outdated. Update the version in the existing
django-prometheus entry to the current stable release, keeping the package name
and pin format intact, and make sure any related dependency constraints in
apps/api/requirements/base.txt remain consistent if there are compatibility
concerns.
In `@docker-compose-local.yml`:
- Around line 166-168: The Grafana service is using a hardcoded admin password
in its environment block, unlike the other services that source secrets from env
files or variable interpolation. Update the Grafana configuration to use an
interpolated variable or env_file-backed value for GF_SECURITY_ADMIN_PASSWORD,
and keep the existing GF_USERS_ALLOW_SIGN_UP setting unchanged so the credential
handling matches the rest of the compose file.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: a005e0dc-d074-4819-a97e-799f5a97141a
📒 Files selected for processing (8)
apps/api/plane/settings/local.pyapps/api/plane/urls.pyapps/api/requirements/base.txtdeployments/grafana/dashboards/plane-dashboard.jsondeployments/grafana/provisioning/dashboards/dashboard.ymldeployments/grafana/provisioning/datasources/datasource.ymldeployments/prometheus/prometheus.ymldocker-compose-local.yml
| path("api/instances/", include("plane.license.urls")), | ||
| path("api/v1/", include("plane.api.urls")), | ||
| path("auth/", include("plane.authentication.urls")), | ||
| path("", include("django_prometheus.urls")), |
There was a problem hiding this comment.
🔒 Security & Privacy | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Check whether nginx or ingress configs expose the api's / or /metrics path externally
rg -n "metrics" -g '*.conf' -g 'nginx*' -g 'docker-compose*.yml'Repository: makeplane/plane
Length of output: 153
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Map likely deployment and routing files, then inspect the relevant ones.
git ls-files | rg '(^|/)(urls\.py|docker-compose.*\.yml|docker-compose.*\.yaml|nginx.*|.*\.conf|.*ingress.*|.*helm.*|values.*\.ya?ml)$'
echo
echo '--- urls.py references to django_prometheus ---'
rg -n "django_prometheus|metrics" apps api . --glob '!*node_modules/*' --glob '!*dist/*' --glob '!*build/*'
echo
echo '--- compose / ingress / nginx references to api exposure ---'
rg -n "api:|ports:|expose:|django_prometheus|metrics|ingress|nginx|proxy_pass|server_name|host:" \
docker-compose* . \
--glob '!**/node_modules/**' --glob '!**/dist/**' --glob '!**/build/**' --glob '!**/.git/**'Repository: makeplane/plane
Length of output: 21403
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo '--- apps/api/plane/urls.py ---'
sed -n '1,120p' apps/api/plane/urls.py
echo
echo '--- docker-compose.yml ---'
sed -n '1,240p' docker-compose.yml
echo
echo '--- docker-compose-local.yml ---'
sed -n '1,240p' docker-compose-local.yml
echo
echo '--- apps/admin/nginx/nginx.conf ---'
sed -n '1,220p' apps/admin/nginx/nginx.conf
echo
echo '--- apps/space/nginx/nginx.conf ---'
sed -n '1,220p' apps/space/nginx/nginx.conf
echo
echo '--- apps/web/nginx/nginx.conf ---'
sed -n '1,220p' apps/web/nginx/nginx.conf
echo
echo '--- deployments/prometheus/prometheus.yml ---'
sed -n '1,220p' deployments/prometheus/prometheus.ymlRepository: makeplane/plane
Length of output: 11941
/metrics is exposed without auth in apps/api/plane/urls.py:23.
django_prometheus.urls is mounted at the root, and docker-compose-local.yml publishes api:8000 to the host, so these internal request/DB metrics are reachable by anyone who can hit that port. Lock /metrics to the internal network or add proxy auth if the path is forwarded in production.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@apps/api/plane/urls.py` at line 23, The Prometheus metrics endpoint is
currently mounted publicly through the root include in the URL configuration, so
update the URL wiring in the `urlpatterns` setup to stop exposing
`django_prometheus.urls` to unauthenticated external callers. Move the metrics
route behind an internal-only path or protect it with access control at the
proxy/app layer, and make sure the change is applied where the
`include("django_prometheus.urls")` entry is registered.
| datasources: | ||
| - name: Prometheus | ||
| type: prometheus | ||
| access: proxy | ||
| url: http://prometheus:9090 | ||
| isDefault: true |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '\n== datasource.yml ==\n'
cat -n deployments/grafana/provisioning/datasources/datasource.yml
printf '\n== dashboard refs ==\n'
python3 - <<'PY'
import json, re, pathlib
p = pathlib.Path('deployments/grafana/dashboards/plane-dashboard.json')
data = json.loads(p.read_text())
text = p.read_text()
# Find occurrences of datasource references around uid/name keys
for key in ['uid', 'name']:
print(f"\nOccurrences of '{key}':")
for m in re.finditer(rf'"{key}"\s*:\s*"[^"]+"', text):
start = max(0, m.start()-120)
end = min(len(text), m.end()+120)
snippet = text[start:end]
print(snippet.replace('\n', ' '))
PYRepository: makeplane/plane
Length of output: 2195
🌐 Web query:
Grafana provisioning datasource uid auto-generated if omitted dashboard datasource uid references Grafana 8.3
💡 Result:
When you omit the uid field in a Grafana datasource provisioning file, Grafana automatically generates a random UID for that datasource [1][2][3]. Starting in Grafana 8.3, Grafana shifted from referencing datasources by name to referencing them by UID within dashboard JSON definitions [4][5][6]. Because of this change, failing to explicitly specify a uid in your provisioning file can lead to significant issues [4][5]. If the UID is omitted, Grafana generates a new, random UID each time the system is redeployed or restarted [4][5]. Consequently, existing dashboards—which store a reference to a specific UID—will fail to find the datasource because the UID has changed, effectively "bricking" the dashboards [5][7][8]. To ensure stability and portability of dashboards across different Grafana environments (such as development, testing, and production), it is highly recommended to explicitly define a static, consistent uid for every provisioned datasource [4][7][9]. Example of explicit UID provisioning: datasources: - name: MyDataSource type: postgres uid: my_consistent_uid_001 #... other configuration options By manually setting the uid, you ensure that dashboard references remain valid across deployments, even if you rename the datasource later [4][6].
Citations:
- 1: https://grafana.com/docs/grafana/v11.5/administration/provisioning/
- 2: https://github.com/grafana/grafana/blob/main/conf/provisioning/datasources/sample.yaml
- 3: https://grafana.com/docs/administration/provisioning/
- 4: Provisioning: set stable datasource UID with provisioning grafana/grafana#41769
- 5: Provisioned datasources need a stable UID grafana/grafana#41763
- 6: https://github.com/grafana/grafana/blob/7bf0e764/apps/dashboard/pkg/migration/schemaversion/v36.go
- 7: https://community.grafana.com/t/provisioning-of-postgres-datasources-not-working/126709/3
- 8: Datasources: Update of a Datasource UID does not Update Dashboards that reference the Datasource grafana/grafana#101212
- 9: Make datasource UID editable in GUI grafana/grafana#82962
🌐 Web query:
site:grafana.com datasource provisioning uid Grafana dashboard uid reference Prometheus
💡 Result:
When provisioning data sources in Grafana, you can explicitly define a unique identifier (UID) in your configuration file to ensure it remains consistent across different environments [1][2][3]. This UID is critical for dashboards, as they must reference this identifier to correctly link to the intended data source [4][5]. To provision a data source with a specific UID, include the uid field in your YAML provisioning file: apiVersion: 1 datasources: - name: My Prometheus type: prometheus uid: my-prometheus-uid # Explicitly defined UID access: proxy url: http://prometheus:9090 In your dashboard JSON files, the datasource field can reference this data source in two primary ways: 1. By Name: You can use the name of the data source as a string (e.g., "datasource": "My Prometheus") [5]. 2. By UID: You can use an object that specifies the type and the UID, or simply the UID string depending on your Grafana version and specific configuration, to ensure the dashboard points to the exact data source regardless of its display name [4][5]. To avoid issues where dashboards fail to find the data source (especially during migrations or when moving between systems), it is best practice to always provide a static, explicit UID in your data source provisioning YAML rather than allowing Grafana to generate a random one [1][5]. If you are exporting dashboards for use across different systems, ensuring the data source UID is identical in both environments prevents broken references [6][2]. Note that in recent versions of Grafana, there has been a move toward enforcing stricter UID formats to improve consistency [7]. Always verify that your defined UID is unique within your Grafana organization [2].
Citations:
- 1: https://community.grafana.com/t/provisioning-of-postgres-datasources-not-working/126709/3
- 2: https://grafana.com/docs/administration/provisioning/
- 3: https://grafana.com/docs/grafana/latest/installation/provisioning/
- 4: https://community.grafana.com/t/should-provisioned-dashboards-have-datasource-uids/65463
- 5: https://community.grafana.com/t/should-provisioned-dashboards-have-datasource-uids/65463/8
- 6: https://community.grafana.com/t/how-to-get-correct-data-source-uid-in-my-dashboards-json-file-while-exporting-it-on-a-different-system/90282
- 7: https://grafana.com/whats-new/2025-05-05-enforcing-stricter-data-source-uid-format/
Add a stable uid to the Prometheus datasource
deployments/grafana/dashboards/plane-dashboard.json binds panels to uid: "Prometheus", but this provisioning file lets Grafana generate a different UID on startup. Add uid: Prometheus here so the dashboard keeps resolving the datasource.
🔧 Proposed fix
datasources:
- name: Prometheus
type: prometheus
access: proxy
url: http://prometheus:9090
+ uid: Prometheus
isDefault: true📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| datasources: | |
| - name: Prometheus | |
| type: prometheus | |
| access: proxy | |
| url: http://prometheus:9090 | |
| isDefault: true | |
| datasources: | |
| - name: Prometheus | |
| type: prometheus | |
| access: proxy | |
| url: http://prometheus:9090 | |
| uid: Prometheus | |
| isDefault: true |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@deployments/grafana/provisioning/datasources/datasource.yml` around lines 3 -
8, The Prometheus datasource provisioning in datasource.yml is missing a fixed
uid, which can break dashboard bindings that reference the datasource by uid.
Update the datasource entry to set the uid explicitly to Prometheus so the
provisioned datasource matches the references used by plane-dashboard.json and
remains stable across Grafana restarts.
1 participant
Description
This PR integrates a self-contained monitoring and observability pipeline for local developer and self-hosted deployments. It adds a Prometheus metrics exporter directly to the Django API and mounts a local Prometheus + Grafana stack in Docker Compose to display live performance analytics.
Key Changes:
django-prometheusdependency tobase.txtand integrateddjango_prometheusmiddleware to track API request speeds and database query times.prometheus(port9091) andgrafana(port3002) services todocker-compose-local.ymlwith host-isolated volume persistence.Type of Change
Test Scenarios
docker compose -f docker-compose-local.yml up -d --buildto compile dependencies.http://localhost:8000/metricsendpoint is live and serving standard Prometheus metrics.up) athttp://localhost:9091/targets.http://localhost:3002(login:admin/admin) and verified the custom Plane API Analytics dashboard is receiving and rendering real-time performance graphs.References
Summary by CodeRabbit
New Features
Bug Fixes
Chores