Event Deltas: Field filtering and priority classification

[alex-fedotyev]

Summary

Automatically hide noisy/low-value fields (IDs, timestamp arrays, high-cardinality fields) from the primary view and show them in a separate "Lower-priority fields" section with proper pagination.

Problem

The Event Deltas grid shows all flattened span attributes including TraceId, SpanId, ParentSpanId, Events.Timestamp[N], and high-cardinality fields like service.instance.id — which are noise for outlier detection. These fields push genuinely useful attributes off-screen onto later pages.

Changes

Field classification functions (in deltaChartUtils.ts)

• isIdField(key, columnMeta) — detects structural ID fields: top-level String columns or Array(String) elements ending in "Id"/"ID" (TraceId, SpanId, Links.TraceId[N], etc.)
• isTimestampArrayField(key, columnMeta) — detects per-index timestamp array elements from Array(DateTime64(...)) columns
• isDenylisted(key, columnMeta) — combined denylist: ID fields + timestamp arrays
• isHighCardinality(key, outlierValues, inlierValues, outlierProps, inlierProps) — detects fields where min(outlierUniqueness, inlierUniqueness) > 0.9 with combined sample > 20. Uses min so if either group clusters, the field stays visible

Two-section layout with row-based pagination

• Properties split into visibleProperties (primary) and hiddenProperties (denylisted + high-cardinality)
• Rendered in two separate CSS grids with a Divider labeled "Lower-priority fields (N)" between them
• Row-based pagination: calculates rows for each section independently (visibleRows + hiddenRows), ensures hidden fields always start from column 1 (not filling incomplete visible rows)
• Divider appears when both sections are on the same page; header appears when only hidden fields are on a page

Helper utilities

• stripTypeWrappers(type) — removes LowCardinality(...) and Nullable(...) wrappers from ClickHouse type strings (exported for reuse)
• flattenData(data) — recursive object/array flattener (moved from DBDeltaChart inline)
• getPropertyStatistics(data) — computes property/value percentages relative to total rows
• mergeValueStatisticsMaps(outlier, inlier) — merges two value maps into combined array

Files

• packages/app/src/components/deltaChartUtils.ts (isIdField, isTimestampArrayField, isDenylisted, isHighCardinality, stripTypeWrappers, flattenData, getPropertyStatistics, mergeValueStatisticsMaps)
• packages/app/src/components/DBDeltaChart.tsx (visible/hidden split, two-grid layout, row-based pagination)
• packages/app/src/components/__tests__/DBDeltaChart.test.ts (tests for all classification functions + integration test)

Dependencies

None — this is a standalone utility + layout improvement.

Test plan

• TraceId, SpanId, ParentSpanId appear in "Lower-priority fields" section
• Events.Timestamp[N] elements appear in lower-priority section
• High-cardinality fields (>90% unique values, sample > 20) appear in lower-priority section
• Low-cardinality fields like http.method (2 values) stay in primary section
• Hidden fields start from column 1 on their grid rows (not filling incomplete primary rows)
• Pagination correctly accounts for both sections' row counts
• Small sample sizes (≤ 20 combined) keep fields visible regardless of uniqueness

Context

Part of the Event Deltas improvement series. Reference implementation in PR #1797.