Skip to content

Commit f8e6d8b

Browse files
Rperry2174knylander-grafanagithub-actions[bot]kolesnikovae
authored
Add span profiles documentation (#2982)
* Add span profiles documentation * Remove warning * Add moe docs * Add span profiles * Apply suggestions from code review Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Update docs/sources/configure-client/go-span-profiles.md Co-authored-by: Anton Kolesnikov <[email protected]> * Update docs/sources/configure-client/go-span-profiles.md Co-authored-by: Kim Nylander <[email protected]> * Update docs/sources/configure-client/go-span-profiles.md * Update docs/sources/configure-client/go-span-profiles.md Co-authored-by: Kim Nylander <[email protected]> * Update docs/sources/configure-client/go-span-profiles.md * Update docs/sources/configure-client/go-span-profiles.md * Update docs/sources/configure-client/go-span-profiles.md Co-authored-by: Kim Nylander <[email protected]> * Apply suggestions from code review --------- Co-authored-by: Kim Nylander <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Anton Kolesnikov <[email protected]>
1 parent 7d318f4 commit f8e6d8b

File tree

2 files changed

+110
-1
lines changed

2 files changed

+110
-1
lines changed
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,109 @@
1+
---
2+
title: Span profiles with Traces to profiles for Go
3+
menuTitle: Span profiles with Traces to profiles (Go)
4+
description: Learn about and configure Span profiles with Traces to profiles in Grafana for the Go language.
5+
weight: 100
6+
---
7+
8+
# Span profiles with Traces to profiles for Go
9+
10+
![span-profiles screenshot](https://grafana.com/static/img/docs/tempo/profiles/tempo-profiles-Span-link-profile-data-source.png)
11+
12+
## Before you begin
13+
14+
Your applications must be instrumented for profiling and tracing before you can use span profiles.
15+
16+
* Profiling: Your application must be instrumented with Pyroscopes Go SDK. If you haven't done this yet, please refer to the [Go (push mode)]({{< relref "../configure-client/language-sdks/go_push" >}}) guide.
17+
* Tracing: Your application must be instrumented with OpenTelemetry traces. If you haven't done this yet, please refer to the [OpenTelemetry](https://opentelemetry.io/docs/go/getting-started/) guide.
18+
19+
## OpenTelemetry support
20+
21+
Pyroscope can integrate with distributed tracing systems supporting [**OpenTelemetry**](https://opentelemetry.io/docs/instrumentation/go/getting-started/) standard which allows you to
22+
link traces with the profiling data, and find resource usage for specific lines of code for your trace spans
23+
24+
:::note
25+
* Only CPU profiling is supported at the moment.
26+
* Because of how sampling profilers work, spans shorter than the sample interval may not be captured. Go CPU profiler probes stack traces 100 times per second, meaning that spans shorter than 10ms may not be captured.
27+
:::
28+
29+
30+
## Configure the otel-profiling-go package
31+
32+
To start collecting Span Profiles for your Go application, you need to include [otel-profiling-go](https://github.com/pyroscope-io/otel-profiling-go) in your code.
33+
34+
This package is a `TracerProvider` implementation, that labels profiling data with span IDs which makes it possible to query for span-specific profiling data in Grafana Tempo UI.
35+
36+
```shell
37+
# Make sure you also upgrade pyroscope server to version 0.14.0 or higher.
38+
go get github.com/grafana/otel-profiling-go
39+
```
40+
41+
Next you need to create and configure the tracer provider:
42+
```go
43+
package main
44+
45+
import (
46+
otelpyroscope "github.com/grafana/otel-profiling-go"
47+
"github.com/grafana/pyroscope-go"
48+
)
49+
50+
func main() {
51+
// Initialize your tracer provider as usual.
52+
tp := initTracer()
53+
54+
// Wrap it with otelpyroscope tracer provider.
55+
otel.SetTracerProvider(otelpyroscope.NewTracerProvider(tp))
56+
57+
// If you're using Pyroscope Go SDK, initialize pyroscope profiler.
58+
_, _ = pyroscope.Start(pyroscope.Config{
59+
ApplicationName: "my-service",
60+
ServerAddress: "http://localhost:4040",
61+
})
62+
63+
// Your code goes here.
64+
}
65+
```
66+
67+
Now that we set up the tracer, we can create a new trace from anywhere and the profiler will automatically capture profiles for it.
68+
```go
69+
ctx, span := otel.Tracer("tracerName").Start(ctx, "ExampleSpan")
70+
defer span.End()
71+
72+
// Your code goes here.
73+
```
74+
75+
## View the span profiles in Grafana Tempo
76+
77+
To view the span profiles in Grafana Tempo, you need to have a Grafana instance running and a data source configured to link trace spans and profiles.
78+
79+
Refer to the [data source configuration documentation](/docs/grafana/datasources/tempo/configure-tempo-data-source) to see how to configure the visualization to link trace spans with profiles.
80+
81+
To use a simple configuration, follow these steps:
82+
83+
1. Select a Pyroscope data source from the Data source drop-down.
84+
85+
2. Optional: Choose any tags to use in the query. If left blank, the default values of service.name and service.namespace are used.
86+
87+
The tags you configure must be present in the spans attributes or resources for a trace to profiles span link to appear. You can optionally configure a new name for the tag. This is useful for example if the tag has dots in the name and the target data source doesn’t allow using dots in labels. In that case you can for example remap service.name to service_name.
88+
89+
3. Select one or more profile types to use in the query. Select the drop-down and choose options from the menu.
90+
91+
The profile type or app must be selected for the query to be valid. Grafana doesn’t show any data if the profile type or app isn’t selected when a query runs.
92+
93+
![span-profiles configuration](https://grafana.com/static/img/docs/tempo/profiles/Tempo-data-source-profiles-Settings.png)
94+
95+
## Examples
96+
97+
Check out the [examples](https://github.com/grafana/pyroscope/tree/main/examples/tracing/tempo) directory in our repository to
98+
find a complete example application that demonstrates tracing integration features.
99+
100+
<!-- ## Using tracing exemplars manually
101+
102+
If you're not using open telemetry integration you can still use exemplars storage to store profiles associated with some execution context (e.g individual HTTP / GRPC request). To create exemplars you need to tag specific parts of your code with a special `profile_id` tag, for example, in golang you could do this:
103+
```golang
104+
pprof.Do(ctx, pprof.Labels("profile_id", "8474e98b95013e4f"), func(ctx context.Context) {
105+
slowRequest()
106+
})
107+
```
108+
109+
`"8474e98b95013e4f"` can be any ID that you use to identify execution contexts (individual HTTP / GRPC requests). -->

examples/tracing/tempo/README.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
### This is a pre-release demo project for internal use only
1+
# Span Profiles with Grafana Tempo and Pyroscope
22

33
The docker compose consists of:
44
- Ride share demo application

0 commit comments

Comments
 (0)