v2.9.2

Honeycomb recently updated its event ingest limit to 1MB per event instead of the former 100K per event.
This release enables Refinery to conform to the same limit and now supports sending events up to 1MB.
It also improves Refinery's internal metrics by introducing a new metric, incoming_router_otlp, to track the amount of OTLP requests Refinery has received.

Fixes

fix: only load config and rules once (#1470) | Tyler Helmuth
fix: add incoming/peer_router_otlp metrics for otlp traffic (#1474) | Yingrong Zhao

Maintenance

maint(deps): bump the minor-patch group across 1 directory with 14 updates (#1475) | dependabot
feat: update libhoney to 1.25.0 (#1473) | Yingrong Zhao
maint: bump golang.org/x/net to v0.33.0 (#1466) | Tyler Helmuth

v2.9.1

This is a bug fix release.

Fixes

• register OTEL metrics if it's missing (#1456) | Yingrong Zhao

Maintenance

• call out cache changes and add missing changelog entry (#1454) | Yingrong Zhao
• update Husky to 0.34.0 (#1459) | Mike Goldsmith

v2.9.0

This release introduces a variety of enhancements and bug fixes. It has two major features: one that improves memory consumption reporting, and one experimental feature for configuring trace locality mode.
See full details in the Release Notes.

Features

• feat: rename DisableTraceLocality to TraceCache (#1450) | Yingrong Zhao
• feat: Add 'unpublished' flag to configs (#1446) | Kent Quirk
• feat: Rename EnableTraceLocality to DisableTraceLocality (#1442) | Mike Goldsmith
• feat: add a limit to queue draining logic (#1441) | Yingrong Zhao
• feat: Try to drain incoming and peer queues for an amount of time (#1440) | Mike Goldsmith
• feat: ignore trace decision messages produced by the publishers (#1437) | Yingrong Zhao
• feat: Add basic telemetry to event, batch and OTLP endpoints (#1431) | Mike Goldsmith
• feat: compress kept trace decision message (#1430) | Yingrong Zhao
• feat: publish instanceID during peer comms (#1420) | Kent Quirk
• feat: increase KeptDecisionSendInterval default value to 1s (#1421) | Yingrong Zhao
• feat: batch kept decisions (#1419) | Yingrong Zhao
• feat: set a better version for dev builds (#1415) | Robb Kidd
• feat: only enalbe stress relief for the entire cluster together (#1413) | Yingrong Zhao
• feat: send kept trace decision in a separate goroutine (#1412) | Yingrong Zhao
• feat: only redistribute traces when its ownership has changed (#1411) | Yingrong Zhao
• feat: Add a way to specify the team key for config fetches (experimental) (#1410) | Kent Quirk
• feat: send drop decisions in batch (#1402) | Yingrong Zhao
• feat: Update in-memory trace cache to use LRU instead of ring buffer (#1359) | Mike Goldsmith
• feat: Log response bodies when sending events to Honeycomb (#1386) | Mike Goldsmith
• feat: make collector health check timeout configurable (#1371) | Yingrong Zhao
• feat: Record original user agent for spans and logs (#1358) | Mike Goldsmith
• feat: forward decision span through peer endpoint (#1342) | Yingrong Zhao
• feat: extract decision span from full span (#1338) | Yingrong Zhao
• feat(doc): separate table for metrics contains prefix (#1354) | Yingrong Zhao
• feat: generate metrics documentation (#1351) | Yingrong Zhao
• feat: Improve shutdown logic (#1347) | Kent Quirk
• feat: Update Honeycomb logger to use EMAThroughput sampler (#1328) | Mike Goldsmith
• feat: Improve log messages to be more informative (#1322) | Mike Goldsmith
• feat: extract key fields from rules config (#1327) | Yingrong Zhao

Fixes

• fix: documentation bug (#1449) | Kent Quirk
• fix: missing read lock in MapWithTTL (#1445) | Yingrong Zhao
• fix: revert draining logic for incoming and peer queue (#1443) | Yingrong Zhao
• fix: only ignore messages that are coming from the node itself (#1438) | Yingrong Zhao
• fix: Update flaky test (#1436) | Mike Goldsmith
• fix: send all traffic through deterministic sampler during stress relief activated (#1433) | Yingrong Zhao
• fix: more reliable dev version tagging (#1424) | Robb Kidd
• fix: do not use trace object during processTraceDecisions (#1423) | Yingrong Zhao
• fix: count the number of IDs in drop decision messages (#1416) | Mike Goldsmith
• fix: replace api key with SendKey before transmission (#1404) | Yingrong Zhao
• fix: deal with orphan traces and expired traces (#1408) | Yingrong Zhao
• fix: reset redistribution delay on peer membership change (#1403) | Yingrong Zhao
• fix: explictly assign float64 type for trace cache metrics (#1406) | Yingrong Zhao
• fix: deal with orphan traces in trace cache (#1405) | Yingrong Zhao
• fix: use current node address as default peer list (#1388) | Yingrong Zhao
• fix: Only set incoming user agent if not already present (#1366) | Mike Goldsmith
• fix: Put a limit on the size of sampler keys (#1364) | Kent Quirk
• fix: set 0 for otel metrics during registration (#1352) | Yingrong Zhao
• fix: remove InMemoryCollector from liveness check on shutdown (#1349) | Yingrong Zhao
• fix: ConvertNumeric now handles bools (#1336) | Kent Quirk
• fix: remove unnecessary assertion to any (#1333) | Yingrong Zhao
• fix: Use peer transmission during redistribute and shutdown events (#1332) | Mike Goldsmith

Maintenance

• maint: remove unused collect_cache metrics (#1452) | Yingrong Zhao
• maint(deps): bump the minor-patch group with 6 updates (#1451) | dependabot
• maint: update metrics doc (#1448) | Yingrong Zhao
• maint: remove trace cache metrics (#1447) | Yingrong Zhao
• maint: clean up sampler log entry (#1444) | Yingrong Zhao
• maint(deps): bump the minor-patch group with 11 updates (#1428) | dependabot
• maint: Add missing LICENSE file (#1429) | Kent Quirk
• maint: build fixes (#1427) | Kent Quirk
• maint: Update log level for making a trace decision to debug (#1425) | Mike Goldsmith
  Goldsmith](https://github.com/MikeGoldsmith)
• maint: Add missing LICENSE file (#1429) | Kent Quirk
• maint: build fixes (#1427) | Kent Quirk
• maint: Update log level for making a trace decision to debug (#1425) | Mike Goldsmith
• docs: update config docs for compatability of using DryRun and EnableTraceLocality together (#1418) | Mike Goldsmith
• maint: add comments about an edge case in SendKey (#1387) | Yingrong Zhao
• maint: Update OTel dependencies (#1409) | Mike Goldsmith
  lmuth](https://github.com/vinozzZ)
• docs: Document the ability to use prefix in dynamic sampler FieldList (#1396) | Irving Popovetsky
  (demo_metrics_doc)- maint: cherry pick v2.8.4 commits into main. (#1383) | Tyler Helmuth
• maint: Update main documentation with 2.8.3 release (#1374) | Tyler Helmuth
• docs: Update configMeta.yaml with capitalization fixes (#1373) | Mary J.
• maint: add collector_redistribute_traces_duration_ms metric (#1368) | Yingrong Zhao
• maint(deps): bump the minor-patch group with 13 updates (#1357) | dependabot
• maint: Refactor metrics registration to streamline declaration and enable easier documentation generation (#1350) | Yingrong Zhao
• maint: rename sent_reason_cache to kept_reason_cache (#1346) | Yingrong Zhao

v2.8.4

2.8.4 2024-10-11

This is a bugfix release

See full details in the Release Notes.

Fixes

• fix: Switch collector_collect_loop_duration_ms metric to be a histogram (#1381) | Tyler Helmuth

v2.8.3

2.8.3 2024-10-08

This is a bugfix update

See full details in the Release Notes.

Changes

• fix: Only set incoming user agent if not already present (#1366) | Mike Goldsmith
• fix: Put a limit on the size of sampler keys (#1364) | Kent Quirk
• feat: Record original user agent for spans and logs (#1358) | Mike Goldsmith
• feat: Make collector health check timeout configurable (#1371) | Yingrong Zhao
• fix: ConvertNumeric now handles bools (#1336) | Kent Quirk

v2.8.2

This is a bug fix release.

In particular, the feature of Throughput Samplers known as UseClusterSize did not work, leading to odd results. This release fixes that problem.

It also allows the new SendKey feature to be controlled with environment variables and the command line.

2.8.2 2024-09-13

Changes

• fix: assertion for ClusterSizer (#1329) | Yingrong Zhao
• feat: Allow SendKey to be set via command line and env var (#1323) | Mike Goldsmith
• feat: add SendKey envvar config metadata (#1331) | Yingrong Zhao
• maint: run smoke test in CI (#1316) | Yingrong Zhao

v2.8.1

2.8.1 2024-09-05

This is a bugfix update; it replaces v2.8.0, which has been withdrawn.

These release notes incorporate all of the 2.8.0 updates as well.

This release has many features designed to help manage and operate Refinery at scale.
It also includes some features to help in writing sampling rules (in and not-in operators, root.)
See full details in the Release Notes.

Features

• feat: add IN operator (#1302) | Kent Quirk
• feat: support layered (multiple) configuration files (#1301) | Kent Quirk
• feat: Add a cache to the cache (#1296) | Kent Quirk
• feat: support configure refinery to use redis in cluster mode (#1294) | Yingrong Zhao
• feat: allow root. in field list for dynamic sampler (#1275) | Yingrong Zhao
• feat: redistribute traces on peer membership changes (#1268) | Yingrong Zhao
• feat: Add SpanLimit (includes some config changes) (#1266) | Kent Quirk
• feat: redistribute remaining traces during shutdown (#1261) | Yingrong Zhao
• feat: Allow more complex key behavior (#1263) | Kent Quirk
• feat: unregister peer asap on shutdown (#1260) | Yingrong Zhao

Fixes

• fix: load peer list in sharder once manually on startup (#1314) | Yingrong Zhao*
• fix: periodically clean up recent_dropped_traces cache (#1312) | Yingrong Zhao
• fix: revert the revert -- that wasn't the problem (#1311) | Kent Quirk
• fix: revert "Use HTTP/2 for all upstream and peer-to-peer connections… (#1310) | Kent Quirk
• fix: join peer list only after refinery is ready to accept traffic (#1309) | Yingrong Zhao
• fix: use float histogram for otel metrics (#1303) | Kent Quirk
• fix: escape use input in debug route (#1299) | Tyler Helmuth
• fix: use trace.DescendantCount for span limit (#1297) | Yingrong Zhao
• fix: support TLS connections to Redis (#1285) | Yingrong Zhao
• fix: only set send reason to span limit if it's configured (#1290) | Yingrong Zhao
• fix: record previous value of sampler counter metrics so they report correctly (#1281) | Kent Quirk
• fix: set up tls for redis when it's enabled | Yingrong Zhao
• fix: don't read more than max bytes from a request (#1282) | Kent Quirk
• fix: allow draining traces even if only 1 peer left (#1278) | Yingrong Zhao
• fix: record sample rate in decision cache during stress relief (#1273) | Yingrong Zhao
• fix: SpanLimit shouldn't add SendDelay (#1272) | Kent Quirk
• fix: Use HTTP/2 for all upstream and peer-to-peer connections (#1269) | Irving Popovetsky

Maintenance

• maint: Add some extra logging to pubsub systems (#1308) | Kent Quirk
• maint: Add warning about cli flags (#1293) | Tyler Helmuth
• maint: Delete unused Dockerfile (#1292) | Tyler Helmuth
• maint: add a docker'd Redis TLS local setup (#1291) | Robb Kidd
• maint: change default for MaxSendMsgSize and MaxRcvMsgSize. (#1289) | Kent Quirk
• maint: use non-forked cuckoofilter again (#1287) | Kent Quirk
• maint(deps): bump the minor-patch group with 13 updates (#1304) | dependabot[bot]
• maint(deps): bump the minor-patch group with 4 updates (#1262) | dependabot[bot]
• refactor: Remove error returns from config functions, fix tests. (#1259) | Kent Quirk
• docs: fix CacheCapacity documentation (#1267) | Kent Quirk

Full Changelog: v2.7.0...v2.8.1

v2.7.1

2.7.1 2024-08-15

This is a bug fix release.
Sending compressed event or batch payloads that are larger than 5MB will now result in a HTTP 400 "failed to parse event" error response to clients.

Fixes

• fix: don't read more than max bytes from a request (#1276) | Kent Quirk

v2.7.0

2.7.0 2024-07-29

This release incorporates a new publish/subscribe (pubsub) system for faster and cleaner communication between Refinery nodes.
In particular, the way Refinery uses Redis has changed.
See full details in the Release Notes.

Features

• feat: Add metrics to pubsub and peers (#1226) | Kent Quirk
• feat: add otel tracing support for Refinery internal operations (#1218) | Yingrong Zhao
• feat: Add some useful generics (#1206) | Kent Quirk
• feat: gossip config reload information (#1241) | Kent Quirk
• feat: Health/Ready system imported from R3 (#1231) | Kent Quirk
• feat: peer management on pubsub via callbacks (#1220) | Kent Quirk
• feat: track config hash on config reload (#1212) | Yingrong Zhao
• feat: use pub/sub for stress relief (#1221) | Yingrong Zhao
• feat: Working, tested, but unused pubsub system (#1205) | Kent Quirk

Fixes

• fix: add injection tags for configwatcher (#1246) | Yingrong Zhao
• fix: add peer logging, add debug log of peers (#1239) | Kent Quirk
• fix: allow a single node to activate stress relief mode during significant load increase (#1256) | Yingrong Zhao
• fix: allow sending otel tracing to non honeycomb backend (#1219) | Yingrong Zhao
• fix: Change pubsub interface to use callbacks. (#1217) | Kent Quirk
• fix: clean up a print line (#1250) | Yingrong Zhao
• fix: FilePeers implies no Redis (#1251) | Kent Quirk
• fix: make sure stress relief pub/sub topic is consistent (#1245) | Yingrong Zhao
• fix: make sure to inject Health object as a pointer (#1237) | Yingrong Zhao
• fix: Record hashes at startup in metrics (#1252) | Kent Quirk
• fix: reduce pub/sub messages from stress relief (#1248) | Yingrong Zhao
• fix: remove otel-config-go as a dependency (#1240) | Yingrong Zhao
• fix: remove personal api keys (#1253) | Kent Quirk
• fix: Root spans must have a non-empty parent ID field (#1236) | Mike Goldsmith
• fix: sharder should use peer identity from Peers package (#1249) | Yingrong Zhao

Maintenance

• docs: Tweak docs for reload (#1247) | Kent Quirk
• docs: update vulnerability reporting process (#1224) | [Robb Kidd](https://github.com/Robb Kidd)
• maint: add instrumentation for GoRedisPubSub (#1229) | Yingrong Zhao
• maint: Add jitter to peer traffic, fix startup (#1227) | Kent Quirk
• maint: change targeted arch to arm for local development Dockerfile (#1228) | Yingrong Zhao
• maint: last changes before the final release prep (#1254) | Kent Quirk
• maint: update doc based on config changes (#1243) | Yingrong Zhao
• maint: Update licenses (#1244) | [Tyler Helmuth](https://github.com/Tyler Helmuth)
• maint(deps): bump google.golang.org/grpc from 1.64.0 to 1.64.1 (#1223) | dependabot[bot]
• maint(deps): bump the minor-patch group across 1 directory with 9 updates (#1232) | dependabot[bot]

v2.6.1

What's Changed

🐛 Fixes

• fix: Don’t consider log events as root spans by @MikeGoldsmith in #1208

🛠 Maintenance

• maint(deps): bump the minor-patch group with 9 updates by @dependabot in #1198

Full Changelog: v2.6.0...v2.6.1