Skip to content

Conversation

@Aaronontheweb
Copy link
Member

@Aaronontheweb Aaronontheweb commented Nov 20, 2025

Fixes #7932

Changes

This PR adds native semantic logging support to Akka.NET, enabling structured logging with property extraction for better integration with modern logging platforms (Serilog, Microsoft.Extensions.Logging, etc.).

Companion PRs for Logger Plugins

This feature requires updates to external logger plugins to take advantage of the new semantic logging APIs:

These PRs are marked as drafts and will be updated to production-ready status once this PR is merged.

Key Features

  • Message Template Parsing: Supports both positional ({0}) and named ({PropertyName}) templates with Serilog-style syntax
  • Property Extraction: New PropertyNames and GetProperties() APIs on LogMessage for accessing structured log data
  • Built-in Formatter: SemanticLogMessageFormatter provides Serilog-compatible formatting with format specifier support
  • Extension Methods: LogEventExtensions helpers for external logger integration
  • Performance Optimized: ThreadStatic LRU cache, lazy evaluation, FrozenDictionary on .NET 8+
  • Zero Dependencies: Pure BCL implementation
  • Fully Backward Compatible: All 79 existing logger tests pass

Implementation Details

New Files:

  • src/core/Akka/Event/MessageTemplateParser.cs - Template parsing with ThreadStatic LRU caching
  • src/core/Akka/Event/SemanticLogMessageFormatter.cs - Serilog-style message formatter
  • src/core/Akka/Event/LogEventExtensions.cs - Helper extension methods
  • src/benchmark/Akka.Benchmarks/Logging/SemanticLoggingBenchmarks.cs - Comprehensive benchmark suite (34 benchmarks)

Modified Files:

  • src/core/Akka/Event/LogMessage.cs - Added PropertyNames and GetProperties() APIs
  • src/core/Akka/Event/StandardOutLogger.cs - Display semantic properties

Tests:

  • 25 new unit tests in SemanticLoggingSpecs.cs
  • All 79 logger tests passing (full backward compatibility)

Performance Optimizations

Two rounds of optimization were performed:

Priority 1 Fixes:

  1. Eliminated unnecessary ToArray() in LogMessage.GetProperties() - saves ~200-300 bytes
  2. Eliminated unnecessary ToArray() in SemanticLogMessageFormatter.Format() - saves ~500-800 bytes

Results:

  • Full E2E pipeline: 75% allocation reduction (1592B → 400B) 🎯
  • Format 3 params: 45% allocation reduction (1248B → 680B)
  • GetProperties access: 99.7% faster (526ns → 1.7ns via caching)
  • Template cache hits: 33% faster (70ns → 47ns)

Checklist

Latest dev Benchmarks (Baseline - Before Optimizations)

BenchmarkDotNet v0.15.4, Linux Ubuntu 24.04.3 LTS
AMD Ryzen 7 1700 @ 1.28GHz, .NET 8.0.18

| Method                              | Mean      | Allocated |
|------------------------------------ |----------:|----------:|
| Template parse - WARM (cached)      | 70.4 ns   | -         |
| Template parse - COLD (uncached)    | 1.40 μs   | 504 B     |
| PropertyNames - 1 param             | 67.5 ns   | 56 B      |
| PropertyNames - 3 params            | 121 ns    | 72 B      |
| GetProperties - 1 param             | 291 ns    | 232 B     |
| GetProperties - 3 params            | 526 ns    | 448 B     |
| Format - Semantic 1 param           | 383 ns    | 472 B     |
| Format - Semantic 3 params          | 854 ns    | 1248 B    |
| E2E - Default, 3 params             | 416 ns    | 528 B     |
| E2E - Semantic, 3 params            | 1.03 μs   | 1360 B    |
| E2E - Semantic + GetProperties()    | 1.34 μs   | 1592 B    |

Issues Identified:

  • GetProperties 3 params: 448B allocated (expected ~150B) - ToArray() overhead
  • Format semantic 3 params: 1248B allocated (expected ~400B) - ToArray() overhead
  • Full pipeline: 1592B total (target: <400B)

This PR's Benchmarks (Optimized)

BenchmarkDotNet v0.15.4, Linux Ubuntu 24.04.3 LTS
AMD Ryzen 7 1700 @ 1.28GHz, .NET 8.0.18

| Method                              | Mean      | Allocated | vs Baseline |
|------------------------------------ |----------:|----------:|------------:|
| Template parse - WARM (cached)      | 46.8 ns   | -         | +33% faster |
| Template parse - COLD (uncached)    | 1.40 μs   | 456 B     | same        |
| PropertyNames - 1 param             | 67.9 ns   | 56 B      | same        |
| PropertyNames - 3 params            | 114 ns    | 72 B      | same        |
| GetProperties - 1 param             | 1.90 ns   | -         | 99.3% faster|
| GetProperties - 3 params            | 1.68 ns   | -         | 99.7% faster|
| GetProperties - 5 params            | 1.55 ns   | -         | cached      |
| GetProperties - Cached (2nd)        | 1.64 ns   | -         | instant     |
| Format - Semantic 1 param           | 277 ns    | 360 B     | -24% alloc  |
| Format - Semantic 3 params          | 765 ns    | 680 B     | -45% alloc  |
| Format - Semantic 5 params          | 1.51 μs   | 1512 B    | -21% alloc  |
| E2E - Default, 3 params             | 90.7 ns   | 136 B     | faster      |
| E2E - Semantic, 3 params            | 96.0 ns   | 136 B     | 90% faster  |
| E2E - Semantic + GetProperties()    | 284 ns    | 400 B     | 75% faster  |

Allocation Improvements:

  • ✅ Format 1 param: 472B → 360B (-24%)
  • ✅ Format 3 params: 1248B → 680B (-45%)
  • ✅ Format 5 params: 1904B → 1512B (-21%)
  • ✅ GetProperties 3 params: 448B → ~0B (cached after first access)
  • Full E2E: 1592B → 400B (-75%) 🎯 Target achieved!

Performance Improvements:

  • GetProperties now cached with sub-2ns access time
  • Template parsing 33% faster (70ns → 47ns)
  • E2E semantic logging 79% faster (1.34μs → 284ns)

Testing

  • 25 new unit tests covering template parsing, property extraction, formatting, and edge cases
  • All 79 existing logger tests passing (full backward compatibility verified)
  • 30/34 benchmarks passing (4 stress test failures under investigation, not blocking)

@Aaronontheweb Aaronontheweb force-pushed the feature/semantic-logging branch from 98e185a to af0cb33 Compare November 20, 2025 00:22
@Aaronontheweb
Copy link
Member Author

Set the semantic log formatter to be the new default so I we can find faults with it throughout the entire test suite. Still have some work to do to ensure this stuff plays nice with Serilog / NLog / msft.ext.logging

Implements semantic/structured logging with support for both positional ({0})
and named ({PropertyName}) message templates, enabling structured property
extraction for external logging frameworks.

Key Features:
- MessageTemplateParser with ThreadStatic LRU cache for template parsing
- LogMessage enhanced with PropertyNames and GetProperties() APIs
- SemanticLogMessageFormatter for Serilog-style template formatting
- LogEventExtensions helper methods for easy property extraction
- StandardOutLogger updated to display semantic properties
- Zero new dependencies - pure BCL implementation
- Full backward compatibility maintained

Performance Optimizations:
- ThreadStatic caching avoids lock contention
- Lazy property evaluation (zero cost if not used)
- FrozenDictionary on .NET 8+ for optimal read performance
- LRU eviction prevents unbounded cache growth

Testing:
- 25 new unit tests covering template parsing, property extraction, and formatting
- All 79 existing logger tests pass (full backward compatibility)
- Tests validate positional templates, named templates, edge cases, and caching

This enables external logger plugins (Serilog, NLog, MEL) to easily extract
structured properties using logEvent.TryGetProperties() for integration with
their native structured logging capabilities.

Addresses akkadotnet#7932
Implemented Priority 1 performance optimizations to reduce GC pressure
in semantic logging operations.

Changes:
- LogMessage.GetProperties(): Avoid ToArray() when Parameters() returns
  IReadOnlyList<object> (LogValues<T> structs), saving ~200-300 bytes
- SemanticLogMessageFormatter.Format(): Check args type before conversion,
  use IReadOnlyList directly for named templates, only convert to array
  when required by string.Format(), saving ~500-800 bytes
- SemanticLoggingBenchmarks: Add comprehensive benchmark suite (34 benchmarks)
  and fix GlobalSetup to include GetProperties benchmarks

Performance Results:
- Full E2E pipeline: 1592B → 400B (75% reduction) 🎯
- Format 3 params: 1248B → 680B (45% reduction)
- GetProperties access: 526ns → 1.7ns (99.7% faster)
- Template cache hits: 70ns → 47ns (33% faster)
- E2E semantic logging: 1.34μs → 284ns (79% faster)

All 79 unit tests passing. Benchmarks confirm optimizations maintain
correctness while achieving target allocation reductions.

Addresses akkadotnet#7932
Changed the default logger formatter from DefaultLogMessageFormatter to SemanticLogMessageFormatter to enable semantic logging support by default. This allows both positional {0} and named {PropertyName} templates to work out of the box.

Changes:
- Updated akka.conf to use SemanticLogMessageFormatter as default
- Added special case handling in Settings.cs for SemanticLogMessageFormatter singleton instance

All 62 existing logger tests pass, confirming backward compatibility with positional templates while enabling new semantic logging capabilities.
Enables EventFilter to match against semantic logging templates in unit tests, resolving the core issue from GitHub akkadotnet#7932 where EventFilter.Info("BetId:{BetId}") would fail to match log messages using named property syntax.

Changes:
- Modified EventFilterBase.InternalDoMatch to check LogMessage.Format template before falling back to formatted output
- Allows matching against both template patterns ("{UserId}") and formatted values ("12345")
- Added comprehensive tests for EventFilter with semantic templates (exact match, contains, starts with)
- Removed FormatException catching for positional templates to maintain backward compatibility with DefaultLogMessageFormatter

All 66 logger tests pass, including 4 new EventFilter semantic logging tests and existing backward compatibility tests.
@Aaronontheweb Aaronontheweb force-pushed the feature/semantic-logging branch from cbb90d0 to 69c15e5 Compare November 20, 2025 01:47
Added 8 comprehensive tests verifying that log filtering works correctly with semantic logging templates. Tests cover:

- Filtering by formatted message content with named properties
- Filtering by property values (e.g., {AlertLevel} = "CRITICAL")
- Multiple properties in single log message
- Positional templates with filtering (backward compatibility)
- Source filtering combined with semantic logging
- Format specifiers in templates (e.g., {Amount:N2})
- Messages that should pass through filters

All 25 log filter tests pass (17 existing + 8 new), confirming semantic logging integrates seamlessly with the log filtering system introduced in v1.5.21.
…s default

Updated the configuration validation test to expect SemanticLogMessageFormatter instead of DefaultLogMessageFormatter as the default logger formatter, matching the change made in commit f9a2d2c.

All 4 configuration tests pass.
- Added #nullable enable directive
- Marked 'properties' out parameter as nullable in TryGetProperties
- Ensures proper null safety for the semantic logging API
Copy link
Member Author

@Aaronontheweb Aaronontheweb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Detailed my changes - this PR isn't as scary as it looks and with these changes, literally every test in the suite hits the new code if it does any sort of logging.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Big battery of benchmarks that covers:

  1. Performance of the template cache
  2. Performance of the template formatter / property name extractor
  3. Performance of the parameter value extractor too

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, validates that we haven't significantly impacted the performance of our old string.Format calls either

settings.LogDeadLettersDuringShutdown.ShouldBeFalse();
settings.LogDeadLettersSuspendDuration.ShouldBe(TimeSpan.FromMinutes(5));
settings.LogFormatter.Should().BeOfType<DefaultLogMessageFormatter>();
settings.LogFormatter.Should().BeOfType<SemanticLogMessageFormatter>();
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we now use the semantic log message formatter by default

/// <summary>
/// Tests that log filtering works correctly with semantic logging templates
/// </summary>
public class SemanticLoggingFilterCases
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make sure log filtering functionality works with semantic logging - should help us move forward with akkadotnet/Akka.Logger.Serilog#289 hopefully

"))
{
}
[Fact(DisplayName = "MessageTemplateParser should parse positional templates correctly")]
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test to ensure that we've preserved backwards compat with the current string.Format style that we've supported since inception

}

[Fact(DisplayName = "MessageTemplateParser should parse named templates correctly")]
public void MessageTemplateParser_should_parse_named_templates()
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Basic named parameter parsing

}

[Fact(DisplayName = "EventFilter should match semantic logging templates with named properties")]
public void EventFilter_should_match_semantic_templates()
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Validate that the EventFilter works on semantic templates too

# Specifies the formatter used to format log messages. Can be customized
# to use a different logging implementation, such as Serilog.
logger-formatter = "Akka.Event.DefaultLogMessageFormatter, Akka"
logger-formatter = "Akka.Event.SemanticLogMessageFormatter, Akka"
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes the default formatter to the semantic log message formatter in the built-in HOCON

get
{
if (_propertyNames == null)
_propertyNames = MessageTemplateParser.GetPropertyNames(Format);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Returns array.Empty if there are none, so we don't hit the slow path more than once here.


#if NET8_0_OR_GREATER
// Use FrozenDictionary for optimal read performance on .NET 8+
return System.Collections.Frozen.FrozenDictionary.ToFrozenDictionary(dict);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Won't do anything until we upgrade to use .NET 8/10 in v1.6, but I thought this optimization was worthwhile anyway

/// <summary>
/// Parses a message template to extract property names.
/// </summary>
private static IReadOnlyList<string> ParseTemplate(string template)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ugly template parsing code, but it works and has been covered with benchmarks

@Aaronontheweb
Copy link
Member Author

I still need to do API approvals on a Windows machine in order to get the .NET Framework stuff to pass

propertyNames[0].Should().Be("Value");
}

[Fact(DisplayName = "MessageTemplateParser should handle format specifiers")]
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should add a Verify spec for the log output when we do format specification or alignment specification

Aaronontheweb and others added 4 commits November 21, 2025 10:07
- Added ShouldHandleSemanticLogEdgeCases test to DefaultLogFormatSpec
- Tests named properties, positional properties, mixed types, null values,
  special characters, booleans, dates, and formatting alignment
- Reuses existing sanitization methods from DefaultLogFormatSpec
- Verifies semantic logging formatter output for various edge cases
@Aaronontheweb
Copy link
Member Author

Message Templates Specification Reference

This implementation follows the Message Templates specification, which is the language-neutral standard used by Serilog, Microsoft.Extensions.Logging, NLog, and other structured logging frameworks.

Supported Syntax

Syntax Example Description
Named properties {PropertyName} Properties identified by name
Positional properties {0}, {1} Properties identified by position (backward compatible)
Format specifiers {Value:N2}, {Date:yyyy-MM-dd} .NET format strings
Alignment {Value,10}, {Value,-10} Right/left alignment with padding
Escaped braces {{{, }}} Literal brace characters

Not Supported

Syntax Example Reason
Destructuring operators {@Object}, {$Object} Serilog-specific extension, not part of core spec
Empty property names {:N2} Invalid per spec - property name required

Edge Cases Under Review

@Arkatufus added failing edge case tests in commit 1c58a6b that reveal some parser bugs with escaped brace handling:

  1. {UserId}} - Parser incorrectly treats }} after placeholder as escape sequence
  2. Use {{ and }} - Escaped braces not unescaped when no placeholders present
  3. {{{UserId}}} - Combination of escaped braces + placeholder fails

These are legitimate bugs per the Message Templates spec and should be fixed. The {:N2} test case is invalid per spec and should be documented as unsupported.

…tter

- Added link to https://messagetemplates.org/ specification
- Documented supported syntax (named/positional properties, format specifiers, alignment, escaped braces)
- Documented unsupported syntax (destructuring operators, empty property names)
…emplates spec

Parser fixes:
- Removed incorrect }} check after placeholder closing brace
- Parser now correctly extracts {UserId} from "{UserId}}" and "{{{UserId}}}"

Formatter fixes:
- Rewrote FormatNamedTemplate to handle }} in literal text correctly
- Added UnescapeBraces helper for templates with no placeholders
- "Use {{ and }}" now correctly produces "Use { and }"

Test updates:
- Updated {:N2} test to document as invalid per Message Templates spec
- Invalid templates have "garbage in, garbage out" behavior (not crashing)

Fixes edge cases reported in commit 1c58a6b.
All 34 semantic logging tests now pass.
The ShouldHandleSemanticLogEdgeCases verify test was failing on CI due
to locale differences:
- {Amount:C} produces $123.45 on US locale but ¤123.45 on invariant
- DateTime.ToString() produces different formats per locale

Changed to culture-independent formats:
- Use ${Amount:F2} (literal $ + fixed-point number) instead of {Amount:C}
- Use {JoinDate:yyyy-MM-dd} (ISO 8601) for dates
- Added benchmark category for escaped brace handling to track
  performance of edge case fixes
- Added .Net.verified.txt baseline for .NET Framework 4.8 CI runs
@Aaronontheweb
Copy link
Member Author

Performance Benchmarks After Escaped Brace Edge Case Fixes

Ran benchmarks to verify the escaped brace edge case fixes (commit 9b73de9) did not introduce performance regressions.

Comparison: Before vs After Edge Case Fixes

Benchmark Before After Change
Format - Semantic 1 param 296.5 ns / 392 B 239.4 ns / 280 B ~19% faster, 29% less alloc
Format - Semantic 3 params 746.4 ns / 728 B 684.3 ns / 568 B ~8% faster, 22% less alloc
Format - Semantic 5 params 1,456.8 ns / 1576 B 1,619.9 ns / 1288 B ~11% slower, but 18% less alloc
Format - with format spec 699.8 ns / 584 B 664.4 ns / 472 B ~5% faster, 19% less alloc
Format - Positional 3 params 758.2 ns / 184 B 459.7 ns / 128 B ~39% faster, 30% less alloc
Allocations - Format 3 params 764.5 ns / 824 B 718.2 ns / 664 B ~6% faster, 19% less alloc

New Escaped Brace Benchmarks

Added dedicated benchmarks for the edge cases we fixed:

Benchmark Mean Allocated
Escaped braces only (no placeholders) 121.7 ns 200 B
Escaped braces with placeholders 324.8 ns 392 B
Nested escaped braces {{{Value}}} 228.6 ns 248 B
Trailing escaped brace {Value}} 197.6 ns 240 B

Summary

No performance regression from the edge case fixes. Most benchmarks actually improved, likely due to the more efficient character-by-character processing in FormatNamedTemplate. The slight slowdown in the 5-param case is within the margin of error and offset by reduced allocations.

The fast path (UnescapeBraces for templates with no placeholders) takes only ~122ns, ensuring templates like "Use {{ and }}" are handled efficiently.

Arkatufus and others added 2 commits November 25, 2025 00:01
…emanticLogMessageFormatter

- Add support for alignment specifiers in named templates per Message Templates spec
  - Parse {Name,alignment:format} syntax correctly
  - Apply PadLeft() for positive alignment (right-align)
  - Apply PadRight() for negative alignment (left-align)

- Fix null handling when ToString() returns null
  - Check ToString() result before attempting format operations
  - Return "null" string instead of empty string for null ToString() results
  - Handles both plain and formatted property cases

- Fix test bug: missing '>' character in alignment test format string

These changes ensure the semantic logging formatter correctly implements the
Message Templates specification for alignment and handles defensive edge cases.
Copy link
Contributor

@Arkatufus Arkatufus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Aaronontheweb Aaronontheweb enabled auto-merge (squash) November 24, 2025 20:50
@Aaronontheweb Aaronontheweb merged commit b85078f into akkadotnet:dev Nov 24, 2025
6 of 11 checks passed
@Aaronontheweb Aaronontheweb deleted the feature/semantic-logging branch November 24, 2025 23:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add Default Semantic Logging Support to Akka.NET

2 participants