Add filters/replace_dots.lua

[ShelbyZ]

Summary

• Add filter functionality similar to golang plugins
• include at /fluent-bit/filters

Provide a workaround to allow replace_dots functionality from golang plugins via lua script and not need core fluent-bit changes to c plugins.

https://github.com/aws/amazon-kinesis-firehose-for-fluent-bit

replace_dots: Replace dot characters in key names with the value of this option. For example, if you add replace_dots _ in your config then all occurrences of . will be replaced with an underscore. By default, dots will not be replaced.

Testing

Ran sample tests with replace dots via conf/lua:

Conf file

[SERVICE]
    Flush        1
    Log_Level    info
    Daemon       off

# Generate test data with dots in field names
[INPUT]
    Name dummy
    Dummy {"field.with.dots": "value1", "nested.object": {"inner.field": "value2", "another.one": 123}, "normal_field": "value3"}
    Tag test.logs

# Apply the Lua filter
[FILTER]
    Name lua
    Match *
    Script replace_dots.lua
    Call filter

# Output results to console
[OUTPUT]
    Name stdout
    Match *
    Format json_lines

With

docker run -ti -v ./test_replace_dots.conf:/fluent-bit/etc/fluent-bit.conf -v ./replace_dots.lua:/fluent-bit/etc/replace_dots.lua cr.fluentbit.io/fluent/fluent-bit 
Fluent Bit v4.1.2
* Copyright (C) 2015-2025 The Fluent Bit Authors
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io

______ _                  _    ______ _ _             ___   __  
|  ___| |                | |   | ___ (_) |           /   | /  | 
| |_  | |_   _  ___ _ __ | |_  | |_/ /_| |_  __   __/ /| | `| | 
|  _| | | | | |/ _ \ '_ \| __| | ___ \ | __| \ \ / / /_| |  | | 
| |   | | |_| |  __/ | | | |_  | |_/ / | |_   \ V /\___  |__| |_
\_|   |_|\__,_|\___|_| |_|\__| \____/|_|\__|   \_/     |_(_)___/


[2025/12/11 20:45:34.892421908] [ info] [fluent bit] version=4.1.2, commit=ad534bebfb, pid=1
[2025/12/11 20:45:34.892473838] [ info] [storage] ver=1.5.3, type=memory, sync=normal, checksum=off, max_chunks_up=128
[2025/12/11 20:45:34.892478708] [ info] [simd    ] SSE2
[2025/12/11 20:45:34.892481228] [ info] [cmetrics] version=1.0.5
[2025/12/11 20:45:34.892482968] [ info] [ctraces ] version=0.6.6
[2025/12/11 20:45:34.892510478] [ info] [input:dummy:dummy.0] initializing
[2025/12/11 20:45:34.892514028] [ info] [input:dummy:dummy.0] storage_strategy='memory' (memory only)
[2025/12/11 20:45:34.892897449] [ info] [sp] stream processor started
[2025/12/11 20:45:34.892932690] [ info] [engine] Shutdown Grace Period=5, Shutdown Input Grace Period=2
[2025/12/11 20:45:34.893026140] [ info] [output:stdout:stdout.0] worker #0 started
{"date":1765485935.141493,"field_with_dots":"value1","nested_object":{"another_one":123,"inner_field":"value2"},"normal_field":"value3"}
{"date":1765485936.141494,"field_with_dots":"value1","nested_object":{"another_one":123,"inner_field":"value2"},"normal_field":"value3"}

Without

Fluent Bit v4.1.2
* Copyright (C) 2015-2025 The Fluent Bit Authors
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io

______ _                  _    ______ _ _             ___   __  
|  ___| |                | |   | ___ (_) |           /   | /  | 
| |_  | |_   _  ___ _ __ | |_  | |_/ /_| |_  __   __/ /| | `| | 
|  _| | | | | |/ _ \ '_ \| __| | ___ \ | __| \ \ / / /_| |  | | 
| |   | | |_| |  __/ | | | |_  | |_/ / | |_   \ V /\___  |__| |_
\_|   |_|\__,_|\___|_| |_|\__| \____/|_|\__|   \_/     |_(_)___/


[2025/12/11 20:49:53.631140103] [ info] [fluent bit] version=4.1.2, commit=ad534bebfb, pid=1
[2025/12/11 20:49:53.631205253] [ info] [storage] ver=1.5.3, type=memory, sync=normal, checksum=off, max_chunks_up=128
[2025/12/11 20:49:53.631208083] [ info] [simd    ] SSE2
[2025/12/11 20:49:53.631209623] [ info] [cmetrics] version=1.0.5
[2025/12/11 20:49:53.631210973] [ info] [ctraces ] version=0.6.6
[2025/12/11 20:49:53.631244954] [ info] [input:dummy:dummy.0] initializing
[2025/12/11 20:49:53.631246474] [ info] [input:dummy:dummy.0] storage_strategy='memory' (memory only)
[2025/12/11 20:49:53.631331554] [ info] [sp] stream processor started
[2025/12/11 20:49:53.631360784] [ info] [engine] Shutdown Grace Period=5, Shutdown Input Grace Period=2
[2025/12/11 20:49:53.631457614] [ info] [output:stdout:stdout.0] worker #0 started
{"date":1765486194.141524,"field.with.dots":"value1","nested.object":{"inner.field":"value2","another.one":123},"normal_field":"value3"}
{"date":1765486195.141495,"field.with.dots":"value1","nested.object":{"inner.field":"value2","another.one":123},"normal_field":"value3"}

Description for the changelog

Enhancement: Add filters/replace_dots.lua to enable replacing dots in records

Licensing

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

[amogh09]

Why do we want to convert the key to string type? How about just leaving non-string keys untouched?

[amogh09]

Can we have unit tests for the filter?