[RORDEV-1567] ECS serializer and improved configurable serializer #1165
Conversation
mgoworko
changed the title
📝 WalkthroughWalkthroughThe change adds ECS v1 audit log serialization. It introduces EcsV1AuditLogSerializer with a predefined ECS field map, including nested fields, and integrates it into AuditingSettingsDecoder via a new "ecs" serializer type and version parsing (V1). AuditFieldValueDescriptor is extended with ProcessingDurationNanos, BooleanValue, NumericValue, and Nested; AuditSerializationHelper resolves these, including recursive nested object construction. AuditFieldValueDescriptorParser recognizes PROCESSING_DURATION_NANOS. AuditingSettingsDecoder now decodes nested JSON into Nested descriptors. Tests are expanded to cover ECS serializer outputs, nested fields, and deterministic contexts, with utilities to normalize timestamps. Sequence Diagram(s)sequenceDiagram
autonumber
participant Config as Config (YAML/JSON)
participant Decoder as AuditingSettingsDecoder
participant ECS as EcsV1AuditLogSerializer
participant Helper as AuditSerializationHelper
participant Resp as AuditResponseContext
Note over Config,Decoder: Build serializer from configuration
Config->>Decoder: audit.serializer = "ecs", version = "v1"
Decoder->>Decoder: Parse serializerType ("ecs") and version (V1)
Decoder->>ECS: Construct EcsV1AuditLogSerializer(allowedEventMode)
Note over Resp,Helper: Serialize an audit response using ECS v1 mapping
Resp->>ECS: onResponse(responseContext)
ECS->>Helper: serialize(responseContext, fields, allowedEventMode)
Helper->>Helper: Resolve descriptors<br/>- primitives (string/number/boolean)<br/>- ProcessingDurationMillis/Nanos<br/>- Nested (recursive map -> JSONObject)
Helper-->>ECS: JSONObject
ECS-->>Resp: Option[JSONObject]
sequenceDiagram
autonumber
participant Conf as Serializer Config (fields)
participant Decoder as AuditingSettingsDecoder
participant Parser as AuditFieldValueDescriptorParser
Note over Conf,Decoder: Decode field descriptors (configurable path)
Conf->>Decoder: fields: { "a": "PROCESSING_DURATION_NANOS", "b": { "x": "...", "y": true } }
alt scalar string
Decoder->>Parser: "PROCESSING_DURATION_NANOS"
Parser-->>Decoder: ProcessingDurationNanos
else object (nested)
Decoder->>Decoder: recursively decode object
Decoder-->>Decoder: Nested( "x" -> ..., "y" -> BooleanValue(true) )
end
Estimated code review effort🎯 4 (Complex) | ⏱️ ~75 minutes Pre-merge checks and finishing touches❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
core/src/main/scala/tech/beshu/ror/accesscontrol/factory/decoders/AuditingSettingsDecoder.scala (1)
285-291: Fix invalid serializer type message (include 'ecs').Error message still lists “[static, configurable]”. Add “ecs”.
- AuditingSettingsCreationError(Message(s"Invalid serializer type '$other', allowed values [static, configurable]")) + AuditingSettingsCreationError(Message(s"Invalid serializer type '$other', allowed values [static, configurable, ecs]"))
🧹 Nitpick comments (5)
core/src/main/scala/tech/beshu/ror/accesscontrol/audit/configurable/AuditFieldValueDescriptorParser.scala (1)
29-31: Avoid double uppercasing in placeholder parsing.
placeholderuppercases the key, anddeserializerAuditFieldValueDescriptoruppercases again. Drop one for clarity.- private val placeholder: P[Either[String, AuditFieldValueDescriptor]] = - (lbrace *> key <* rbrace).map(k => deserializerAuditFieldValueDescriptor(k.trim.toUpperCase).toRight(k)) + private val placeholder: P[Either[String, AuditFieldValueDescriptor]] = + (lbrace *> key <* rbrace).map(k => deserializerAuditFieldValueDescriptor(k.trim).toRight(k))audit/src/main/scala/tech/beshu/ror/audit/utils/AuditSerializationHelper.scala (1)
239-242: Preserve numeric fidelity (avoid 123 → 123.0 and float rounding).Representing numbers as
Doublecan introduce rounding and formatting (e.g., 123.0). PreferBigDecimaland passjava.math.BigDecimaltoJSONObject.- final case class NumericValue(value: Double) extends AuditFieldValueDescriptor + final case class NumericValue(value: BigDecimal) extends AuditFieldValueDescriptorAnd in the resolver:
- case AuditFieldValueDescriptor.NumericValue(value) => value + case AuditFieldValueDescriptor.NumericValue(value) => value.bigDecimalNote: This also requires decoding numbers as
BigDecimalin AuditingSettingsDecoder (see comment there).Also applies to: 123-125
core/src/main/scala/tech/beshu/ror/accesscontrol/audit/ecs/EcsV1AuditLogSerializer.scala (1)
35-36: Drop redundant @timestamp in fields map.
createEntryalready injects@timestamp. Keeping it here duplicates a put; safe but unnecessary.- AuditFieldName("@timestamp") -> AuditFieldValueDescriptor.Timestamp,core/src/main/scala/tech/beshu/ror/accesscontrol/factory/decoders/AuditingSettingsDecoder.scala (2)
376-393: Decode numbers as BigDecimal to avoid precision/format issues.Current decoding uses
toDouble. PreferBigDecimalto preserve exact values and avoid 123 → 123.0.- jsonNumber = n => Right(AuditFieldValueDescriptor.NumericValue(n.toDouble)), + jsonNumber = n => + n.toBigDecimal + .map(bd => AuditFieldValueDescriptor.NumericValue(bd)) + .toRight(DecodingFailure("Cannot decode number", cursor.history)),Note: This pairs with changing
NumericValuetoBigDecimaland resolver to putbigDecimal(see AuditSerializationHelper comment).
309-313: Nit: rename given to ecsSerializerVersionDecoder.Minor typo (“ccs”). Improves readability.
- private given ccsSerializerVersionDecoder: Decoder[EcsSerializerVersion] = Decoder.decodeString.map(_.toLowerCase).emap { + private given ecsSerializerVersionDecoder: Decoder[EcsSerializerVersion] = Decoder.decodeString.map(_.toLowerCase).emap {
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
Disabled knowledge base sources:
- Jira integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (5)
audit/src/main/scala/tech/beshu/ror/audit/utils/AuditSerializationHelper.scala(4 hunks)core/src/main/scala/tech/beshu/ror/accesscontrol/audit/configurable/AuditFieldValueDescriptorParser.scala(1 hunks)core/src/main/scala/tech/beshu/ror/accesscontrol/audit/ecs/EcsV1AuditLogSerializer.scala(1 hunks)core/src/main/scala/tech/beshu/ror/accesscontrol/factory/decoders/AuditingSettingsDecoder.scala(5 hunks)core/src/test/scala/tech/beshu/ror/unit/acl/factory/AuditSettingsTests.scala(7 hunks)
🧰 Additional context used
🧬 Code graph analysis (4)
core/src/main/scala/tech/beshu/ror/accesscontrol/audit/configurable/AuditFieldValueDescriptorParser.scala (1)
audit/src/main/scala/tech/beshu/ror/audit/utils/AuditSerializationHelper.scala (2)
AuditFieldValueDescriptor(171-251)ProcessingDurationNanos(192-192)
core/src/main/scala/tech/beshu/ror/accesscontrol/audit/ecs/EcsV1AuditLogSerializer.scala (1)
audit/src/main/scala/tech/beshu/ror/audit/utils/AuditSerializationHelper.scala (31)
ror(28-299)AllowedEventMode(161-165)AuditFieldName(167-167)AuditFieldValueDescriptor(171-251)serialize(32-45)serialize(47-66)Timestamp(195-195)Nested(245-245)Nested(247-249)CorrelationId(199-199)HttpPath(215-215)RemoteAddress(219-219)LocalAddress(221-221)HttpMethod(211-211)Content(223-223)ContentLengthInBytes(225-225)User(180-180)ImpersonatedByUser(182-182)Id(197-197)Action(184-184)Type(208-208)FinalState(176-176)ProcessingDurationNanos(192-192)ErrorType(204-204)ErrorMessage(206-206)EsClusterName(233-233)EsNodeName(231-231)TaskId(201-201)InvolvedIndices(186-186)AclHistory(188-188)XForwardedForHttpHeader(217-217)
core/src/main/scala/tech/beshu/ror/accesscontrol/factory/decoders/AuditingSettingsDecoder.scala (4)
audit/src/main/scala/tech/beshu/ror/audit/utils/AuditSerializationHelper.scala (9)
ror(28-299)Reason(178-178)AllowedEventMode(161-165)AuditFieldName(167-167)AuditFieldValueDescriptor(171-251)BooleanValue(239-239)NumericValue(241-241)Nested(245-245)Nested(247-249)core/src/main/scala/tech/beshu/ror/accesscontrol/audit/configurable/AuditFieldValueDescriptorParser.scala (2)
AuditFieldValueDescriptorParser(23-92)parse(41-57)core/src/main/scala/tech/beshu/ror/accesscontrol/audit/configurable/ConfigurableAuditLogSerializer.scala (1)
ConfigurableAuditLogSerializer(24-30)core/src/main/scala/tech/beshu/ror/accesscontrol/audit/ecs/EcsV1AuditLogSerializer.scala (2)
EcsV1AuditLogSerializer(25-31)EcsV1AuditLogSerializer(33-83)
core/src/test/scala/tech/beshu/ror/unit/acl/factory/AuditSettingsTests.scala (2)
audit/src/main/scala/tech/beshu/ror/audit/AuditResponseContext.scala (2)
AuditResponseContext(28-55)Forbidden(40-41)audit/src/test/scala/tech/beshu/ror/audit/SerializerTest.scala (19)
DummyAuditRequestContext(295-345)timestamp(296-296)id(298-298)correlationId(300-300)indices(302-302)action(304-304)headers(306-306)requestHeaders(308-308)uriPath(310-310)history(312-312)content(314-314)contentLength(316-316)remoteAddress(318-318)localAddress(320-320)type(322-322)taskId(324-324)httpMethod(326-326)loggedInUserName(328-328)impersonatedByUserName(330-330)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (16)
- GitHub Check: ror (Run all tests IT_es67x)
- GitHub Check: ror (Run all tests IT_es70x)
- GitHub Check: ror (Run all tests IT_es80x)
- GitHub Check: ror (Run all tests IT_es710x)
- GitHub Check: ror (Required checks AUDIT_BUILD_CHECK)
- GitHub Check: ror (Run all tests IT_es810x)
- GitHub Check: ror (Run all tests IT_es717x)
- GitHub Check: ror (Required checks COMPILE_CODEBASE_CHECK)
- GitHub Check: ror (Run all tests WIN_IT_es818x)
- GitHub Check: ror (Run all tests IT_es816x)
- GitHub Check: ror (Run all tests WIN_IT_es91x)
- GitHub Check: ror (Run all tests IT_es818x)
- GitHub Check: ror (Required checks LICENSE_CHECK)
- GitHub Check: ror (Run all tests Unit tests)
- GitHub Check: ror (Run all tests IT_es91x)
- GitHub Check: ror (Optional checks CVE_CHECK)
🔇 Additional comments (2)
core/src/main/scala/tech/beshu/ror/accesscontrol/audit/configurable/AuditFieldValueDescriptorParser.scala (1)
68-69: New nanos placeholder mapping looks correct.
PROCESSING_DURATION_NANOS→ProcessingDurationNanosis consistent with resolver changes.audit/src/main/scala/tech/beshu/ror/audit/utils/AuditSerializationHelper.scala (1)
103-104: Nanos support LGTM.
ProcessingDurationNanoscorrectly resolves viaduration.toNanos.
| .left.map(withAuditingSettingsCreationErrorMessage(msg => s"Configurable serializer is used, but the 'verbosity_level_serialization_mode' setting is invalid: $msg")) | ||
| serializer = version match { |
There was a problem hiding this comment.
Correct error context for ECS verbosity mode.
Message says “Configurable serializer…”, but this path is for ECS.
- .left.map(withAuditingSettingsCreationErrorMessage(msg => s"Configurable serializer is used, but the 'verbosity_level_serialization_mode' setting is invalid: $msg"))
+ .left.map(withAuditingSettingsCreationErrorMessage(msg => s"ECS serializer is used, but the 'verbosity_level_serialization_mode' setting is invalid: $msg"))📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| .left.map(withAuditingSettingsCreationErrorMessage(msg => s"Configurable serializer is used, but the 'verbosity_level_serialization_mode' setting is invalid: $msg")) | |
| serializer = version match { | |
| .left.map(withAuditingSettingsCreationErrorMessage(msg => s"ECS serializer is used, but the 'verbosity_level_serialization_mode' setting is invalid: $msg")) | |
| serializer = version match { |
🤖 Prompt for AI Agents
In
core/src/main/scala/tech/beshu/ror/accesscontrol/factory/decoders/AuditingSettingsDecoder.scala
around lines 231-232, the error message incorrectly refers to "Configurable
serializer" although this path is validating the ECS verbosity mode; update the
mapped error message to reference ECS verbosity_mode (or
verbosity_level_serialization_mode for ECS) so it accurately describes the
context — e.g. change the string to indicate the ECS verbosity mode setting is
invalid and include the original msg; keep the same left.map wrapping and
interpolation style.
1 participant
🚀New (ES) Audit log serializer compliant with Elastic Common Schema (ECS)
🧐Enhancement (ES) Enable nested field definitions in the configurable audit log serializer for more flexible audit logging