-
Notifications
You must be signed in to change notification settings - Fork 3
feat(perf): Add performance measuring api #117
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
TatianaFomina
wants to merge
38
commits into
master
Choose a base branch
from
feat/perf
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
38 commits
Select commit
Hold shift + click to select a range
c022264
Implement perf events
TatianaFomina 0bc0093
feat(performance): enhance performance monitoring with batch sending …
TatianaFomina 767592b
Lint
TatianaFomina ab1ab01
Upd
TatianaFomina 751a048
Upd
TatianaFomina 5daeb4e
Upd api
TatianaFomina b5d71c4
Remove traceId
TatianaFomina d708ebb
feat(performance): add performance monitoring
TatianaFomina 1c7a123
feat(performance): enhance performance monitoring configuration and s…
TatianaFomina 2cabdaa
feat(performance): add performance monitoring settings demo with tran…
TatianaFomina a0d56bb
refactor(performance): simplify transaction queuing and remove unused…
TatianaFomina 933f7f9
refactor(performance): update PerformancePayload interface to include…
TatianaFomina 6c544bd
Lint
TatianaFomina 8309ae3
Fix
TatianaFomina d611f1d
Review
TatianaFomina 2459f5d
feat(performance): introduce batch sending configuration and enhance …
TatianaFomina c1639ca
feat(performance): add batch interval configuration and update UI for…
TatianaFomina c8b37bc
style: clean up code formatting and remove unnecessary whitespace in …
TatianaFomina 1e57084
chore(performance): remove debugger statement from startTransaction m…
TatianaFomina 8e8d0f7
Split
TatianaFomina efa12f9
Review
TatianaFomina 677fdaf
feat(performance): update HawkCatcher initialization and enhance perf…
TatianaFomina 1f1e758
fix(performance): correct transaction length reference and update bat…
TatianaFomina 93fd56c
Add doc
TatianaFomina 846e864
Update performance-monitoring.md
neSpecc e510e2a
Update performance-monitoring.md
neSpecc cefde4d
Update performance-monitoring.md
neSpecc e6885ca
Update performance-monitoring.md
neSpecc 366ca89
Update respectively to docs
TatianaFomina 936da97
Upd
TatianaFomina 6ffcf16
Update readme
TatianaFomina e007868
Refactor HawkCatcher initialization in monitoring.html to use a const…
TatianaFomina bbc9cea
Merge branch 'master' into feat/perf
TatianaFomina f11bdd1
Enhance performance monitoring by adding critical duration threshold …
TatianaFomina c4f0094
Refactor performance monitoring code by adding missing commas for con…
TatianaFomina e5a32ca
Add retries
TatianaFomina 17c44f5
Enhance performance monitoring by adding detailed documentation for S…
TatianaFomina 61649a9
Refactor consoleCatcher and performance monitoring code for improved …
TatianaFomina File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,169 @@ | ||
# Performance Monitoring | ||
|
||
## Optimizations | ||
|
||
Sending all transactions without filtering and aggregation would create an heavy load on the service. We must carefully balance data completeness with efficient storage and processing to ensure optimal performance monitoring. | ||
|
||
### 1. Transaction Selection Criteria | ||
|
||
#### Problem: Too Much Data in case of full sending | ||
|
||
Sending every transaction without filtering creates several critical issues: | ||
|
||
- Significantly increased server load, particularly during high traffic periods | ||
- Large amounts of redundant data that provide minimal analytical value | ||
- "Infinite loops" in client code may generate endless transactions. | ||
|
||
#### Solution: Smart Sampling and Grouping | ||
|
||
Instead of collecting every transaction, we focus on gathering a representative sample that provides meaningful insights while minimizing data volume. | ||
|
||
### 2. Data Flow Optimization Strategies | ||
#### Optimization #1: Sampling (Sending Part of the Data Randomly) | ||
|
||
To ensure we capture important but infrequent transactions: | ||
|
||
– Slow transactions are always sent. Transaction is considered slow if its duration is greater than criticalDurationThresholdMs parameter. | ||
- Errors (`status` == 'failure') are always sent. | ||
|
||
See [Sampling](#sampling) for details. | ||
|
||
#### Optimization #2: Aggregation of Identical Transactions Before Sending | ||
|
||
Throttling + transaction batches → instead of 1000 separate messages, send 1 `AggregatedTransaction`. | ||
|
||
##### Combine transactions with the same name (e.g., GET /api/users) and time window (e.g., 3 seconds). | ||
|
||
Instead of 1000 transactions, send one with count = 1000 and average metrics. | ||
|
||
##### How to choose the time? | ||
|
||
- Store P50, P95, P100 (percentiles). | ||
- Save min(startTime) and max(endTime) to see the interval boundaries and calculate Transactions Per Minute. | ||
|
||
**What do we lose ?** | ||
- The detail of each specific transaction. | ||
- Exact startTime and endTime for each transaction. | ||
|
||
**What do we gain?** | ||
- A sharp reduction in load on the Collector and DB (10-100 times fewer records). | ||
- All necessary metrics (P50, P95, P100, avg) remain. | ||
- You can continue to build graphs and calculate metrics, but with less load. | ||
|
||
See [Transaction Aggregation](#transaction-aggregation) for details on how transactions are aggregated. | ||
|
||
#### Optimization #3: Filtering "Garbage" | ||
|
||
Transactions with duration < `thresholdMs` will not be sent, as they are not critical. | ||
|
||
|
||
## Data types | ||
|
||
### Transaction | ||
| Field | Type | Description | | ||
|-------|------|-------------| | ||
| id | string | Unique identifier of the transaction | | ||
| severity | string | Type of transaction sampling ('default' or 'critical'). See [Sampling](#sampling) for details | | ||
| name | string | Name of the transaction | | ||
| startTime | number | Timestamp when transaction started | | ||
| endTime | number | Timestamp when transaction ended | | ||
| duration | number | Total duration of transaction in milliseconds | | ||
| status | string | Status when transaction finished. 'success' (default) or 'failure'. See [Transaction Completion](#2-transaction-completion) | | ||
| spans | Span[] | Array of [spans](#span) associated with this transaction | | ||
|
||
### AggregatedTransaction | ||
| Field | Type | Description | | ||
|-------|------|-------------| | ||
| aggregationId | string | Identifier of the aggregation | | ||
| name | string | Name of the transaction | | ||
| avgStartTime | number | Average timestamp when transaction started | | ||
| minStartTime | number | Minimum timestamp when transaction started | | ||
| maxEndTime | number | Maximum timestamp when transaction ended | | ||
| p50duration | number | 50th percentile (median) duration of transaction in milliseconds | | ||
| p95duration | number | 95th percentile duration of transaction in milliseconds | | ||
| maxDuration | number | Maximum duration of transaction in milliseconds | | ||
| count | number | how many transactions aggregated | | ||
| failureRate | number | percentage of transactions with status 'failure' | | ||
| aggregatedSpans | AggregatedSpan[] | List of spans in transactions | | ||
|
||
|
||
### Span | ||
| Field | Type | Description | | ||
|-------|------|-------------| | ||
| id | string | Unique identifier of the span | | ||
| name | string | Name of the span | | ||
| startTime | number | Timestamp when span started | | ||
| endTime | number | Timestamp when span ended | | ||
| duration | number | Total duration of span in milliseconds | | ||
| status | string | Status when span finished. 'success' (default) or 'failure' | | ||
|
||
### AggregatedSpan | ||
See [Transaction Aggregation](#transaction-aggregation) for details on how spans are aggregated. | ||
|
||
| Field | Type | Description | | ||
|-------|------|-------------| | ||
| aggregationId | string | Unique identifier of the span aggregation | | ||
| name | string | Name of the span | | ||
| minStartTime | number | Minimum timestamp when span started | | ||
| maxEndTime | number | Maximum timestamp when span ended | | ||
| p50duration | number | 50th percentile (median) duration of span in milliseconds | | ||
| p95duration | number | 95th percentile duration of span in milliseconds | | ||
| maxDuration | number | Maximum duration of span in milliseconds | | ||
| failureRate | number | percentage of spans with status 'failure' | | ||
|
||
## Transaction Lifecycle | ||
|
||
### 1. Transaction Creation | ||
|
||
When creating a transaction, you can specify its type: | ||
|
||
- 'critical' - important transactions that are always sent to the server | ||
- 'default' - regular transactions that go through the [sampling process](#sampling) | ||
|
||
### 2. Transaction Completion | ||
|
||
When completing a transaction: | ||
|
||
1. A finish status is specified (`status`): | ||
- 'success' (default) - successful completion | ||
- 'failure' - completion with error (such transactions are always sent to the server) | ||
|
||
2. The transaction duration is checked: | ||
- If `thresholdMs` parameter is specified and the transaction duration is less than this value, the transaction is discarded | ||
- Default `thresholdMs` is 20ms | ||
- `status` "failure" has a priority over `thresholdMs` | ||
- Otherwise, the transaction goes through the [sampling process](#sampling) | ||
|
||
3. After successful sampling, the transaction is added to the list for sending | ||
|
||
### 3. Sending Transactions | ||
|
||
- When the first transaction is added to the list, a timer starts | ||
- When the timer expires: | ||
1. All collected transactions are [aggregated](#transaction-aggregation) | ||
2. Aggregated data is sent to the server | ||
3. The transaction list is cleared | ||
|
||
## Sampling | ||
|
||
- The probability of sending transactions is configured through the `performance.sampleRate` parameter (value from 0 to 1) | ||
- Only transactions of type 'default' with finish status 'success' are subject to sampling | ||
- Sampling process: | ||
1. A random number between 0 and 1 is generated for each transaction | ||
2. If the number is less than or equal to sampleRate, the transaction is sent | ||
|
||
## Transaction Aggregation | ||
|
||
1. [Transactions](#transaction) are grouped by name (name field) | ||
2. For each group, statistical indicators are calculated: | ||
- minStartTime - earliest start time | ||
- maxEndTime - latest end time | ||
- p50duration - median duration (50th percentile) | ||
- p95duration - 95th percentile duration | ||
- maxDuration - maximum duration | ||
|
||
3. Based on this data, [AggregatedTransaction](#aggregatedtransaction) objects are created | ||
4. For each aggregated transaction: | ||
- [Spans](#span) are grouped by name | ||
- Their own statistical indicators (see [AggregatedSpan](#aggregatedspan)) are calculated for each span group | ||
- [AggregatedSpan](#aggregatedspan) objects are created |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.