Skip to content

feat: enrich OTLP telemetry identity - #240

Merged
jingxiang-z merged 5 commits into
mainfrom
agent/otlp-identity-inventory-metrics
Aug 7, 2026
Merged

feat: enrich OTLP telemetry identity#240
jingxiang-z merged 5 commits into
mainfrom
agent/otlp-identity-inventory-metrics

Conversation

@jingxiang-z

@jingxiang-z jingxiang-z commented Aug 4, 2026

Copy link
Copy Markdown
Collaborator

Description

Checklist

  • I am familiar with the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

Summary by CodeRabbit

  • New Features

    • Added richer hardware identity information for GPUs, CPUs, NVLink, and NVSwitch resources.
    • Added hardware inventory and agent uptime metrics.
    • Enhanced metrics, events, and incidents with consistent resource identity details.
    • Improved hostname handling and deterministic metric labels.
  • Bug Fixes

    • Preserved original event metadata when adding identity information.
    • Omitted unavailable inventory values and invalid future start times.
    • Improved GPU model and fabric information collection when supported.
  • Tests

    • Added coverage for hardware identity mapping, inventory metrics, hostname normalization, and event enrichment.

@coderabbitai

coderabbitai Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • ✅ Review completed - (🔄 Check again to review again)

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 7b754050-28a0-4dbf-8445-a2fa540d02cc

📥 Commits

Reviewing files that changed from the base of the PR and between 039ac47 and 7250bfe.

📒 Files selected for processing (4)
  • internal/exporter/collector/identity.go
  • internal/exporter/collector/identity_test.go
  • internal/exporter/converter/otlp.go
  • internal/exporter/converter/otlp_test.go
🚧 Files skipped from review as they are similar to previous changes (4)
  • internal/exporter/collector/identity_test.go
  • internal/exporter/converter/otlp.go
  • internal/exporter/collector/identity.go
  • internal/exporter/converter/otlp_test.go

📝 Walkthrough

Walkthrough

The change adds GPU and host identity catalogs, collects GPU model and fabric metadata, propagates fabric identity through inventory APIs, and enriches OTLP resources, metrics, events, and incidents.

Changes

Entity identity enrichment

Layer / File(s) Summary
GPU inventory identity data
third_party/fleet-intelligence-sdk/api/v1/types.go, third_party/fleet-intelligence-sdk/pkg/machine-info/*
GPU inventory now includes model, cluster, and nullable clique identity data. Collection handles unsupported or failed NVML calls.
GPU fabric identity propagation
internal/inventory/*, internal/backendclient/types.go, internal/validation/outbound/validator.go
Inventory types and mappings preserve cluster and clique identifiers. Inventory hashes include these fields, and outbound validation checks cluster UUID length.
Entity catalog construction
internal/exporter/collector/identity.go, internal/exporter/collector/collector.go, internal/exporter/collector/identity_test.go
The collector builds an EntityCatalog from machine inventory and DCGM indexes. Cached machine information initializes when identity-dependent exports are enabled.
OTLP identity and inventory conversion
internal/exporter/converter/otlp.go, internal/exporter/converter/otlp_test.go
The converter uses physical entity identity, deterministic labels, uptime metrics, inventory metrics, and normalized host.name attributes. Events and incidents exclude metric-only identity labels.

Estimated code review effort: 4 (Complex) | ~60 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Collector
  participant MachineInfoProvider
  participant EntityCatalog
  participant OTLPConverter
  Collector->>MachineInfoProvider: collect cached machine information
  Collector->>EntityCatalog: build host and GPU identities
  Collector->>OTLPConverter: pass HealthData with EntityCatalog
  OTLPConverter->>OTLPConverter: enrich resources, metrics, events, and incidents
Loading

Possibly related PRs

Suggested reviewers: mukilsh

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 8.70% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the primary change: enriching OTLP telemetry with identity information.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch agent/otlp-identity-inventory-metrics

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 181ee8ca01

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread internal/exporter/converter/otlp.go Outdated

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (3)
third_party/fleet-intelligence-sdk/pkg/machine-info/machine_info.go (1)

409-409: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Hoist the fabric-support probe out of the device loop.

platformInfoSupported is computed once at line 343. FabricStateSupported() is a node-level capability, but it is queried for every device. Compute it once before the loop for symmetry and to avoid repeated probes.

♻️ Proposed refactor
 	platformInfoSupported := nvidianvml.PlatformInfoSupported()
+	fabricStateSupported := nvmlInstance.FabricStateSupported()
-		if nvmlInstance.FabricStateSupported() {
+		if fabricStateSupported {
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@third_party/fleet-intelligence-sdk/pkg/machine-info/machine_info.go` at line
409, Move the FabricStateSupported probe out of the per-device loop and compute
it once alongside the existing platformInfoSupported initialization. Reuse that
single node-level capability value in the loop’s fabric-state logic instead of
calling nvmlInstance.FabricStateSupported() for each device.
third_party/fleet-intelligence-sdk/pkg/machine-info/machine_info_test.go (1)

581-590: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Add a model-name success assertion.

fabricIdentityMockGPUDevice embeds partialFailureMockGPUDevice, whose GetName returns ERROR_GPU_IS_LOST. No test asserts a populated ModelName. Override GetName on this mock and assert the value so the new ModelName field has success-path coverage.

💚 Proposed test addition
+func (d *fabricIdentityMockGPUDevice) GetName() (string, nvml.Return) {
+	return "NVIDIA H100", nvml.SUCCESS
+}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@third_party/fleet-intelligence-sdk/pkg/machine-info/machine_info_test.go`
around lines 581 - 590, Update fabricIdentityMockGPUDevice by overriding GetName
to return a valid model name instead of the embedded ERROR_GPU_IS_LOST value,
then extend the associated fabric-identity test to assert that ModelName
contains that value. Keep the existing fabric-state assertions and failure-path
behavior unchanged.
internal/exporter/converter/otlp.go (1)

385-400: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Collapse the duplicated attribute builder.

convertLabelsToOTLPAttributes repeats the sort-and-build loop that labelsToOTLPAttributes already implements at lines 655-667. Delegate to it.

♻️ Proposed refactor
 func (c *otlpConverter) convertLabelsToOTLPAttributes(labels map[string]string, identity identityContext) []*commonv1.KeyValue {
-	enriched := identity.enrichLabels(labels)
-	keys := make([]string, 0, len(enriched))
-	for key := range enriched {
-		keys = append(keys, key)
-	}
-	sort.Strings(keys)
-
-	attributes := make([]*commonv1.KeyValue, 0, len(keys))
-	for _, key := range keys {
-		attributes = append(attributes, &commonv1.KeyValue{Key: key, Value: stringAnyValue(enriched[key])})
-	}
-	return attributes
+	return labelsToOTLPAttributes(identity.enrichLabels(labels))
 }
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@internal/exporter/converter/otlp.go` around lines 385 - 400, Update
convertLabelsToOTLPAttributes to enrich the labels with identity.enrichLabels,
then delegate sorting and OTLP attribute construction to the existing
labelsToOTLPAttributes helper instead of duplicating its key collection and
build loop.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@internal/exporter/collector/collector.go`:
- Around line 155-160: Update the machine-information collection flow around
collectMachineInfo to avoid waiting initialMachineInfoWait when the configured
health export interval is shorter than that wait, including metrics-only and
events-only deployments; alternatively enforce a minimum
HealthExporterConfig.Interval of 5 seconds during validation. Preserve the
existing machine-info behavior for valid intervals.

In `@internal/exporter/converter/otlp.go`:
- Around line 693-712: Update identityFromEntityID to validate the trimmed
identifier as numeric before returning index-based labels; reject non-numeric
values, while recognizing a full UUID remainder, returning the uuid label and
preserving the original entity ID unchanged. Extend TestIdentityFromEntityID
with non-numeric and full-UUID cases covering these outcomes.

---

Nitpick comments:
In `@internal/exporter/converter/otlp.go`:
- Around line 385-400: Update convertLabelsToOTLPAttributes to enrich the labels
with identity.enrichLabels, then delegate sorting and OTLP attribute
construction to the existing labelsToOTLPAttributes helper instead of
duplicating its key collection and build loop.

In `@third_party/fleet-intelligence-sdk/pkg/machine-info/machine_info_test.go`:
- Around line 581-590: Update fabricIdentityMockGPUDevice by overriding GetName
to return a valid model name instead of the embedded ERROR_GPU_IS_LOST value,
then extend the associated fabric-identity test to assert that ModelName
contains that value. Keep the existing fabric-state assertions and failure-path
behavior unchanged.

In `@third_party/fleet-intelligence-sdk/pkg/machine-info/machine_info.go`:
- Line 409: Move the FabricStateSupported probe out of the per-device loop and
compute it once alongside the existing platformInfoSupported initialization.
Reuse that single node-level capability value in the loop’s fabric-state logic
instead of calling nvmlInstance.FabricStateSupported() for each device.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: bdcadbf9-8d28-4d3f-99ad-641278439e01

📥 Commits

Reviewing files that changed from the base of the PR and between 0d95213 and 181ee8c.

📒 Files selected for processing (8)
  • internal/exporter/collector/collector.go
  • internal/exporter/collector/identity.go
  • internal/exporter/collector/identity_test.go
  • internal/exporter/converter/otlp.go
  • internal/exporter/converter/otlp_test.go
  • third_party/fleet-intelligence-sdk/api/v1/types.go
  • third_party/fleet-intelligence-sdk/pkg/machine-info/machine_info.go
  • third_party/fleet-intelligence-sdk/pkg/machine-info/machine_info_test.go

Comment thread internal/exporter/collector/collector.go
Comment thread internal/exporter/converter/otlp.go Outdated
Signed-off-by: Jingxiang Zhang <jingzhang@nvidia.com>
Signed-off-by: Jingxiang Zhang <jingzhang@nvidia.com>
Signed-off-by: Jingxiang Zhang <jingzhang@nvidia.com>
@jingxiang-z
jingxiang-z force-pushed the agent/otlp-identity-inventory-metrics branch from a6b7732 to cab4214 Compare August 4, 2026 18:14

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
internal/inventory/hash_test.go (1)

130-137: 🗄️ Data Integrity & Integration | 🔵 Trivial | ⚡ Quick win

Test ClusterUUID and CliqueID independently.

The current case changes both fields before one hash comparison. It passes if ComputeHash includes only one field. Use separate cases so each identity field is proven to invalidate the inventory hash.

Proposed test split
-	changedNVLinkDomain := *base
-	changedNVLinkDomain.Resources.GPUInfo.GPUs = append([]GPUDevice(nil), base.Resources.GPUInfo.GPUs...)
-	cliqueID := uint32(0)
-	changedNVLinkDomain.Resources.GPUInfo.GPUs[0].ClusterUUID = "11111111-2222-3333-4444-555555555555"
-	changedNVLinkDomain.Resources.GPUInfo.GPUs[0].CliqueID = &cliqueID
-	changedNVLinkDomainHash, err := ComputeHash(&changedNVLinkDomain)
+	changedClusterUUID := *base
+	changedClusterUUID.Resources.GPUInfo.GPUs = append([]GPUDevice(nil), base.Resources.GPUInfo.GPUs...)
+	changedClusterUUID.Resources.GPUInfo.GPUs[0].ClusterUUID = "11111111-2222-3333-4444-555555555555"
+	changedClusterUUIDHash, err := ComputeHash(&changedClusterUUID)
 	require.NoError(t, err)
-	require.NotEqual(t, baseHash, changedNVLinkDomainHash)
+	require.NotEqual(t, baseHash, changedClusterUUIDHash)
+
+	changedCliqueID := *base
+	changedCliqueID.Resources.GPUInfo.GPUs = append([]GPUDevice(nil), base.Resources.GPUInfo.GPUs...)
+	cliqueID := uint32(0)
+	changedCliqueID.Resources.GPUInfo.GPUs[0].CliqueID = &cliqueID
+	changedCliqueIDHash, err := ComputeHash(&changedCliqueID)
+	require.NoError(t, err)
+	require.NotEqual(t, baseHash, changedCliqueIDHash)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@internal/inventory/hash_test.go` around lines 130 - 137, Split the
changedNVLinkDomain hash test into independent cases: create one variant that
changes only ClusterUUID and another that changes only CliqueID, computing and
comparing each hash against baseHash. Keep the existing GPU slice copy and
ComputeHash error assertions for both cases so each field independently proves
inventory hash invalidation.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@internal/inventory/hash_test.go`:
- Around line 130-137: Split the changedNVLinkDomain hash test into independent
cases: create one variant that changes only ClusterUUID and another that changes
only CliqueID, computing and comparing each hash against baseHash. Keep the
existing GPU slice copy and ComputeHash error assertions for both cases so each
field independently proves inventory hash invalidation.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: f1d458b2-9c8a-419b-983b-1fc1b95e0f79

📥 Commits

Reviewing files that changed from the base of the PR and between a6b7732 and cab4214.

📒 Files selected for processing (16)
  • internal/backendclient/types.go
  • internal/exporter/collector/collector.go
  • internal/exporter/collector/identity.go
  • internal/exporter/collector/identity_test.go
  • internal/exporter/converter/otlp.go
  • internal/exporter/converter/otlp_test.go
  • internal/inventory/hash_test.go
  • internal/inventory/mapper/backend.go
  • internal/inventory/mapper/backend_test.go
  • internal/inventory/source/source.go
  • internal/inventory/source/source_test.go
  • internal/inventory/types.go
  • internal/validation/outbound/validator.go
  • third_party/fleet-intelligence-sdk/api/v1/types.go
  • third_party/fleet-intelligence-sdk/pkg/machine-info/machine_info.go
  • third_party/fleet-intelligence-sdk/pkg/machine-info/machine_info_test.go
🚧 Files skipped from review as they are similar to previous changes (8)
  • internal/exporter/collector/identity.go
  • third_party/fleet-intelligence-sdk/pkg/machine-info/machine_info.go
  • internal/exporter/collector/collector.go
  • third_party/fleet-intelligence-sdk/pkg/machine-info/machine_info_test.go
  • internal/exporter/collector/identity_test.go
  • internal/exporter/converter/otlp.go
  • third_party/fleet-intelligence-sdk/api/v1/types.go
  • internal/exporter/converter/otlp_test.go

Signed-off-by: Jingxiang Zhang <jingzhang@nvidia.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
internal/exporter/collector/identity.go (1)

76-77: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

Add a non-UTC boot-time regression test.

identity_test.go currently supplies an already-UTC timestamp. Add a timestamp with a non-UTC location and assert that catalog.BootTime equals info.Uptime.UTC(). This verifies the behavior changed on Line 77.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@internal/exporter/collector/identity.go` around lines 76 - 77, Add a
regression test in identity_test.go covering the boot-time assignment in the
collector identity path: construct info.Uptime with a non-UTC location, invoke
the relevant collection logic, and assert catalog.BootTime equals
info.Uptime.UTC(). Keep the existing zero-time behavior and current UTC test
coverage intact.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@internal/exporter/collector/identity.go`:
- Around line 76-77: Add a regression test in identity_test.go covering the
boot-time assignment in the collector identity path: construct info.Uptime with
a non-UTC location, invoke the relevant collection logic, and assert
catalog.BootTime equals info.Uptime.UTC(). Keep the existing zero-time behavior
and current UTC test coverage intact.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 8e3cb88a-1e71-4ab4-a4a6-e40d04ec6873

📥 Commits

Reviewing files that changed from the base of the PR and between cab4214 and 039ac47.

📒 Files selected for processing (1)
  • internal/exporter/collector/identity.go

@jingxiang-z
jingxiang-z requested a review from mukilsh August 4, 2026 18:45
@jingxiang-z jingxiang-z self-assigned this Aug 4, 2026
Signed-off-by: Jingxiang Zhang <jingzhang@nvidia.com>
@jingxiang-z
jingxiang-z merged commit 97ee4af into main Aug 7, 2026
9 checks passed
@jingxiang-z
jingxiang-z deleted the agent/otlp-identity-inventory-metrics branch August 7, 2026 21:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants