-
Notifications
You must be signed in to change notification settings - Fork 1.1k
feat: Add native semantic logging support with property extraction #7933
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: Add native semantic logging support with property extraction #7933
Conversation
98e185a to
af0cb33
Compare
|
Set the semantic log formatter to be the new default so I we can find faults with it throughout the entire test suite. Still have some work to do to ensure this stuff plays nice with Serilog / NLog / msft.ext.logging |
Implements semantic/structured logging with support for both positional ({0})
and named ({PropertyName}) message templates, enabling structured property
extraction for external logging frameworks.
Key Features:
- MessageTemplateParser with ThreadStatic LRU cache for template parsing
- LogMessage enhanced with PropertyNames and GetProperties() APIs
- SemanticLogMessageFormatter for Serilog-style template formatting
- LogEventExtensions helper methods for easy property extraction
- StandardOutLogger updated to display semantic properties
- Zero new dependencies - pure BCL implementation
- Full backward compatibility maintained
Performance Optimizations:
- ThreadStatic caching avoids lock contention
- Lazy property evaluation (zero cost if not used)
- FrozenDictionary on .NET 8+ for optimal read performance
- LRU eviction prevents unbounded cache growth
Testing:
- 25 new unit tests covering template parsing, property extraction, and formatting
- All 79 existing logger tests pass (full backward compatibility)
- Tests validate positional templates, named templates, edge cases, and caching
This enables external logger plugins (Serilog, NLog, MEL) to easily extract
structured properties using logEvent.TryGetProperties() for integration with
their native structured logging capabilities.
Addresses akkadotnet#7932
Implemented Priority 1 performance optimizations to reduce GC pressure in semantic logging operations. Changes: - LogMessage.GetProperties(): Avoid ToArray() when Parameters() returns IReadOnlyList<object> (LogValues<T> structs), saving ~200-300 bytes - SemanticLogMessageFormatter.Format(): Check args type before conversion, use IReadOnlyList directly for named templates, only convert to array when required by string.Format(), saving ~500-800 bytes - SemanticLoggingBenchmarks: Add comprehensive benchmark suite (34 benchmarks) and fix GlobalSetup to include GetProperties benchmarks Performance Results: - Full E2E pipeline: 1592B → 400B (75% reduction) 🎯 - Format 3 params: 1248B → 680B (45% reduction) - GetProperties access: 526ns → 1.7ns (99.7% faster) - Template cache hits: 70ns → 47ns (33% faster) - E2E semantic logging: 1.34μs → 284ns (79% faster) All 79 unit tests passing. Benchmarks confirm optimizations maintain correctness while achieving target allocation reductions. Addresses akkadotnet#7932
Changed the default logger formatter from DefaultLogMessageFormatter to SemanticLogMessageFormatter to enable semantic logging support by default. This allows both positional {0} and named {PropertyName} templates to work out of the box.
Changes:
- Updated akka.conf to use SemanticLogMessageFormatter as default
- Added special case handling in Settings.cs for SemanticLogMessageFormatter singleton instance
All 62 existing logger tests pass, confirming backward compatibility with positional templates while enabling new semantic logging capabilities.
Enables EventFilter to match against semantic logging templates in unit tests, resolving the core issue from GitHub akkadotnet#7932 where EventFilter.Info("BetId:{BetId}") would fail to match log messages using named property syntax. Changes: - Modified EventFilterBase.InternalDoMatch to check LogMessage.Format template before falling back to formatted output - Allows matching against both template patterns ("{UserId}") and formatted values ("12345") - Added comprehensive tests for EventFilter with semantic templates (exact match, contains, starts with) - Removed FormatException catching for positional templates to maintain backward compatibility with DefaultLogMessageFormatter All 66 logger tests pass, including 4 new EventFilter semantic logging tests and existing backward compatibility tests.
cbb90d0 to
69c15e5
Compare
Added 8 comprehensive tests verifying that log filtering works correctly with semantic logging templates. Tests cover:
- Filtering by formatted message content with named properties
- Filtering by property values (e.g., {AlertLevel} = "CRITICAL")
- Multiple properties in single log message
- Positional templates with filtering (backward compatibility)
- Source filtering combined with semantic logging
- Format specifiers in templates (e.g., {Amount:N2})
- Messages that should pass through filters
All 25 log filter tests pass (17 existing + 8 new), confirming semantic logging integrates seamlessly with the log filtering system introduced in v1.5.21.
…s default Updated the configuration validation test to expect SemanticLogMessageFormatter instead of DefaultLogMessageFormatter as the default logger formatter, matching the change made in commit f9a2d2c. All 4 configuration tests pass.
- Added #nullable enable directive - Marked 'properties' out parameter as nullable in TryGetProperties - Ensures proper null safety for the semantic logging API
Aaronontheweb
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Detailed my changes - this PR isn't as scary as it looks and with these changes, literally every test in the suite hits the new code if it does any sort of logging.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Big battery of benchmarks that covers:
- Performance of the template cache
- Performance of the template formatter / property name extractor
- Performance of the parameter value extractor too
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also, validates that we haven't significantly impacted the performance of our old string.Format calls either
| settings.LogDeadLettersDuringShutdown.ShouldBeFalse(); | ||
| settings.LogDeadLettersSuspendDuration.ShouldBe(TimeSpan.FromMinutes(5)); | ||
| settings.LogFormatter.Should().BeOfType<DefaultLogMessageFormatter>(); | ||
| settings.LogFormatter.Should().BeOfType<SemanticLogMessageFormatter>(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we now use the semantic log message formatter by default
| /// <summary> | ||
| /// Tests that log filtering works correctly with semantic logging templates | ||
| /// </summary> | ||
| public class SemanticLoggingFilterCases |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Make sure log filtering functionality works with semantic logging - should help us move forward with akkadotnet/Akka.Logger.Serilog#289 hopefully
| ")) | ||
| { | ||
| } | ||
| [Fact(DisplayName = "MessageTemplateParser should parse positional templates correctly")] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Test to ensure that we've preserved backwards compat with the current string.Format style that we've supported since inception
| } | ||
|
|
||
| [Fact(DisplayName = "MessageTemplateParser should parse named templates correctly")] | ||
| public void MessageTemplateParser_should_parse_named_templates() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Basic named parameter parsing
| } | ||
|
|
||
| [Fact(DisplayName = "EventFilter should match semantic logging templates with named properties")] | ||
| public void EventFilter_should_match_semantic_templates() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Validate that the EventFilter works on semantic templates too
| # Specifies the formatter used to format log messages. Can be customized | ||
| # to use a different logging implementation, such as Serilog. | ||
| logger-formatter = "Akka.Event.DefaultLogMessageFormatter, Akka" | ||
| logger-formatter = "Akka.Event.SemanticLogMessageFormatter, Akka" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Changes the default formatter to the semantic log message formatter in the built-in HOCON
| get | ||
| { | ||
| if (_propertyNames == null) | ||
| _propertyNames = MessageTemplateParser.GetPropertyNames(Format); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Returns array.Empty if there are none, so we don't hit the slow path more than once here.
|
|
||
| #if NET8_0_OR_GREATER | ||
| // Use FrozenDictionary for optimal read performance on .NET 8+ | ||
| return System.Collections.Frozen.FrozenDictionary.ToFrozenDictionary(dict); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Won't do anything until we upgrade to use .NET 8/10 in v1.6, but I thought this optimization was worthwhile anyway
| /// <summary> | ||
| /// Parses a message template to extract property names. | ||
| /// </summary> | ||
| private static IReadOnlyList<string> ParseTemplate(string template) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ugly template parsing code, but it works and has been covered with benchmarks
|
I still need to do API approvals on a Windows machine in order to get the .NET Framework stuff to pass |
| propertyNames[0].Should().Be("Value"); | ||
| } | ||
|
|
||
| [Fact(DisplayName = "MessageTemplateParser should handle format specifiers")] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should add a Verify spec for the log output when we do format specification or alignment specification
- Added ShouldHandleSemanticLogEdgeCases test to DefaultLogFormatSpec - Tests named properties, positional properties, mixed types, null values, special characters, booleans, dates, and formatting alignment - Reuses existing sanitization methods from DefaultLogFormatSpec - Verifies semantic logging formatter output for various edge cases
…kka.net into feature/semantic-logging
Message Templates Specification ReferenceThis implementation follows the Message Templates specification, which is the language-neutral standard used by Serilog, Microsoft.Extensions.Logging, NLog, and other structured logging frameworks. Supported Syntax
Not Supported
Edge Cases Under Review@Arkatufus added failing edge case tests in commit 1c58a6b that reveal some parser bugs with escaped brace handling:
These are legitimate bugs per the Message Templates spec and should be fixed. The |
…tter - Added link to https://messagetemplates.org/ specification - Documented supported syntax (named/positional properties, format specifiers, alignment, escaped braces) - Documented unsupported syntax (destructuring operators, empty property names)
…emplates spec
Parser fixes:
- Removed incorrect }} check after placeholder closing brace
- Parser now correctly extracts {UserId} from "{UserId}}" and "{{{UserId}}}"
Formatter fixes:
- Rewrote FormatNamedTemplate to handle }} in literal text correctly
- Added UnescapeBraces helper for templates with no placeholders
- "Use {{ and }}" now correctly produces "Use { and }"
Test updates:
- Updated {:N2} test to document as invalid per Message Templates spec
- Invalid templates have "garbage in, garbage out" behavior (not crashing)
Fixes edge cases reported in commit 1c58a6b.
All 34 semantic logging tests now pass.
The ShouldHandleSemanticLogEdgeCases verify test was failing on CI due
to locale differences:
- {Amount:C} produces $123.45 on US locale but ¤123.45 on invariant
- DateTime.ToString() produces different formats per locale
Changed to culture-independent formats:
- Use ${Amount:F2} (literal $ + fixed-point number) instead of {Amount:C}
- Use {JoinDate:yyyy-MM-dd} (ISO 8601) for dates
- Added benchmark category for escaped brace handling to track performance of edge case fixes - Added .Net.verified.txt baseline for .NET Framework 4.8 CI runs
Performance Benchmarks After Escaped Brace Edge Case FixesRan benchmarks to verify the escaped brace edge case fixes (commit 9b73de9) did not introduce performance regressions. Comparison: Before vs After Edge Case Fixes
New Escaped Brace BenchmarksAdded dedicated benchmarks for the edge cases we fixed:
Summary✅ No performance regression from the edge case fixes. Most benchmarks actually improved, likely due to the more efficient character-by-character processing in The fast path ( |
…emanticLogMessageFormatter
- Add support for alignment specifiers in named templates per Message Templates spec
- Parse {Name,alignment:format} syntax correctly
- Apply PadLeft() for positive alignment (right-align)
- Apply PadRight() for negative alignment (left-align)
- Fix null handling when ToString() returns null
- Check ToString() result before attempting format operations
- Return "null" string instead of empty string for null ToString() results
- Handles both plain and formatted property cases
- Fix test bug: missing '>' character in alignment test format string
These changes ensure the semantic logging formatter correctly implements the
Message Templates specification for alignment and handles defensive edge cases.
Arkatufus
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Fixes #7932
Changes
This PR adds native semantic logging support to Akka.NET, enabling structured logging with property extraction for better integration with modern logging platforms (Serilog, Microsoft.Extensions.Logging, etc.).
Companion PRs for Logger Plugins
This feature requires updates to external logger plugins to take advantage of the new semantic logging APIs:
These PRs are marked as drafts and will be updated to production-ready status once this PR is merged.
Key Features
{0}) and named ({PropertyName}) templates with Serilog-style syntaxPropertyNamesandGetProperties()APIs onLogMessagefor accessing structured log dataSemanticLogMessageFormatterprovides Serilog-compatible formatting with format specifier supportLogEventExtensionshelpers for external logger integrationImplementation Details
New Files:
src/core/Akka/Event/MessageTemplateParser.cs- Template parsing with ThreadStatic LRU cachingsrc/core/Akka/Event/SemanticLogMessageFormatter.cs- Serilog-style message formattersrc/core/Akka/Event/LogEventExtensions.cs- Helper extension methodssrc/benchmark/Akka.Benchmarks/Logging/SemanticLoggingBenchmarks.cs- Comprehensive benchmark suite (34 benchmarks)Modified Files:
src/core/Akka/Event/LogMessage.cs- Added PropertyNames and GetProperties() APIssrc/core/Akka/Event/StandardOutLogger.cs- Display semantic propertiesTests:
SemanticLoggingSpecs.csPerformance Optimizations
Two rounds of optimization were performed:
Priority 1 Fixes:
ToArray()inLogMessage.GetProperties()- saves ~200-300 bytesToArray()inSemanticLogMessageFormatter.Format()- saves ~500-800 bytesResults:
Checklist
Latest
devBenchmarks (Baseline - Before Optimizations)Issues Identified:
This PR's Benchmarks (Optimized)
Allocation Improvements:
Performance Improvements:
Testing