Sentry Structured Logging for .NET

[AbhiPrasad]

https://develop.sentry.dev/sdk/telemetry/logs/

Sentry is adding support for structured logging. Let's add it to the .NET SDK!

1. Define the logs protocol and log envelope item in the SDK
2. Add the Public API (SDK options and methods) as per the docs
3. Make sure the the SDK follows the documented behavior
4. Attach default attributes to the SDK as per docs
5. Instrument popular logging libraries to send logs to Sentry. SDK maintainers can best decide what they should support, but we should definitely support the same integrations that we do in the SDK (NLog, Serilog, etc.)
6. Create a GH discussion that contains instructions for setting up the SDK in your repo. Eventually this will be moved into the primary docs. See the JS SDK's GH discussion for inspiration.

[bruno-garcia]

The idea is to:

1. Ship the barebone Sentry API into the core Sentry package
2. Make this work from Microsoft.Extensions.Logging
3. Make this work from Serilog
4. Make this work from NLog
5. Make this work from log4net

I'll dare to say that no one in .NET will use "Sentry's logging API". I know I wouldn't. But that's not the point.
The point is to have the API in the main SDK, so that the logging libraries integrations only pipe data to it.

The order above allows us to try/test things incrementally.
Once we get to n2, we can add it to nuget-trends and symbol-collector for dogfooding.

All our current logging integrations do:

1. Check level, if higher than MininumEventLevel it records an event
2. Check level, if higher than MinimumBreadcrumbLevel it adds a breadcrumb

e.g:

sentry-dotnet/src/Sentry.Extensions.Logging/SentryLogger.cs

Lines 39 to 86 in 084e3b4

	public void Log<TState>(
	LogLevel logLevel,
	EventId eventId,
	TState state,
	Exception? exception,
	Func<TState, Exception?, string>? formatter)
	{
	if (!IsEnabled(logLevel))
	{
	return;
	}
	var message = formatter?.Invoke(state, exception);
	if (ShouldCaptureEvent(logLevel, eventId, exception))
	{
	var @event = CreateEvent(logLevel, eventId, state, exception, message, CategoryName);
	_ = _hub.CaptureEvent(@event);
	// Capturing exception events adds a breadcrumb automatically... we don't want to add another one
	if (exception != null)
	{
	return;
	}
	}
	if (ShouldAddBreadcrumb(logLevel, eventId, exception))
	{
	var data = eventId.ToDictionaryOrNull();
	if (exception != null && message != null)
	{
	// Exception.Message won't be used as Breadcrumb message
	// Avoid losing it by adding as data:
	data ??= new Dictionary<string, string>();
	data.Add("exception_message", exception.Message);
	}
	_hub.AddBreadcrumb(
	_clock,
	(message ?? exception?.Message)!,
	CategoryName,
	null,
	data,
	logLevel.ToBreadcrumbLevel());
	}
	}

The crumb goes after, so that in a subsequent event, it will include the breadcrumb of the previous event.

This works well for error monitoring. But with the new logging integration, we'll send all logs to Sentry.
The new logging could be a new logger class, that's added in of logging is enabled. Since little is shared from the current logger described above, and the new one.

[Flash0ver]

Sentry Structured Logging for .NET

• Add Sentry-Logger APIs
  • See https://develop.sentry.dev/sdk/telemetry/logs/
• Integration: Sentry.Extensions.Logging
  • public CaptureLog on IHub and ISentryClient
  • and/or internal Extension-Methods receiving IHub and ISentryClient
  • ensure SDK-Name/Version is set correctly, test via Sentry.AspNetCore and Sentry.Maui
  • see also feat(logs): initial API for Sentry Logs #4158 (comment)
• more buffering for Sentry.Extensions.Logging
  • add support for the new Microsoft.Extensions.Logging.Abstractions.IBufferedLogger interface
  • next to the original Microsoft.Extensions.Logging.ILogger
• Integration: Sentry.AspNetCore
• Integration: Sentry.AspNetCore.Blazor.WebAssembly
• Integration: Sentry.Azure.Functions.Worker
• Integration: Sentry.Maui
• Integration: Sentry.Serilog
• Integration: Sentry.NLog
• Integration: Sentry.Log4Net
• Internal APIs for Sentry.Unity
  • via internal Extension-Methods receiving IHub and ISentryClient
• Processing: Batching / Priority
  • see also feat(logs): initial API for Sentry Logs #4158 (comment)
  • see also feat(logs): initial API for Sentry Logs #4158 (comment)
• Sentry-Logger-API: Accelerator overloads for convenience
  • see also feat(logs): initial API for Sentry Logs #4158 (comment)
• Sentry-Logger-API: Generic overloads
  • 3 overloads (with 1, 2 and 3 generic arguments) per Log{Level} method
  • to avoid both boxing and array allocations when EnableLogs is false
  • see also feat(logs): initial API for Sentry Logs #4158 (comment)
• Sentry-Logger-API: [InterpolatedStringHandler]
• Documentation
  • see also feat(logs): initial API for Sentry Logs #4158 (comment)

[wondertalik]

Hey,

Thanks for this feature that are you working on. Any plans to adapt opentelemetry for logs as you did for tracing?

[AbhiPrasad]

Hey @wondertalik - thanks for your interest! At the current moment we are focusing on adding logs support via the Sentry SDKs. We'll evaluate OpenTelemetry support in the future. Our logs schema is a superset of the OpenTelemetry schema, so we are already compatible with OpenTelemetry.

[jamescrosswell]

There aren't yet/currently any docs for OpenTelemetry logging but it looks like even if you were using it, the instrumentation would still be done via ILogger<T> (i.e. the same way you'd instrument an app using Micosoft.Extensions.Logging). Anything logged to ILogger<T> can be captured by both Sentry (using the MEL integration) and OpenTelemetry.

I'm not sure if/why we'd ever want to add OpenTelemetry as a middle man in the process of piping logs from ILogger<T> to Sentry. About the only reason I can think of to do this would be so that any OpenTelemetry Resources could be shared by Logging and Tracing (assuming you're using OpenTelemetry for tracing as well)... I think Resources basically define a set of tags that would be applied to all traces and logs captured by an application.

@wondertalik was your main interest in ensuring logs you captured via ILogger<T> ended up in Sentry (something that can be done with the MEL integration already) or was there something else you needed from OpenTelemetry logging?

[wondertalik]

@jamescrosswell basically is unify setup when use sentry and opentelemetry together. Opentelemetry becomes a standart across all the clouds. The main benefit is posibility to send all telemetry to otel collector and via exporters resend where do i need trasparenty for all dev teams (services). I'm doing right now with tracing

For example

• for dev in my laptop i use otel collector + seq + aspire dashboard + jaeger + prometheus.
• for prod - otel collector, sentry, signoz

[jamescrosswell]

I see. That's not how Sentry's OTel integration for tracing works either then... For tracing we have a SpanProcessor - so we capture OTel spans in real time and send these to Sentry (there's no exporter involved).

So basically you can instrument your application with OTel for logging and tracing and have the logs/traces captured by Sentry, but OTel exporters aren't used at any point for either of those things.

[Syed-RI]

Is this going to count towards the quota? For reference, AppCenter didnt have cap or quota for the number log messages but had a retention policy of these logs for 30 days(IIRC). Apologies if its already documented somewhere and I have missed it.

[AbhiPrasad]

@Syed-RI check out this GH discussion for details about pricing/quota: getsentry/sentry#86804 (comment). Feel free to comment on that thread if you have any follow up concerns.