Skip to content

Conversation

mrForza
Copy link
Contributor

@mrForza mrForza commented Sep 24, 2025

The safe_uri function is used in vshard to get a purged uri. Before #11810 patch sensitive ssl data may leaked into logs as uri.format function didn't handle ssl params of uri.

This patch explicitly sets write_sensitive parameter of uri.format into false in safe_uri function and add some tests which check that sensitive ssl data doesn't leak into logs of vshard storage and router.

NO_DOC=bugfix

Closes #593
Closes #597

Copy link
Collaborator

@Serpentian Serpentian left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the patch! Great work, just some nits

P.S. I checked the tests manually, they seem ok, not flaky on my laptop. We don't have CI for EE, and I'm not sure, we can do that politically

@Serpentian Serpentian assigned mrForza and unassigned Serpentian Sep 30, 2025
@mrForza mrForza force-pushed the mrforza/ssl_security_fixes branch 4 times, most recently from 3c0d105 to f6149a6 Compare October 7, 2025 09:23
@mrForza mrForza assigned Serpentian and unassigned mrForza Oct 7, 2025
@mrForza mrForza force-pushed the mrforza/ssl_security_fixes branch from f6149a6 to 22db719 Compare October 7, 2025 12:18
@mrForza mrForza requested a review from Serpentian October 7, 2025 12:35
Copy link
Collaborator

@Serpentian Serpentian left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The patch already looks great and I don't have any new comments regarding the code, let's just properly split the commits and call it a day. Good work

@Serpentian Serpentian assigned mrForza and unassigned Serpentian Oct 8, 2025
@mrForza mrForza force-pushed the mrforza/ssl_security_fixes branch 2 times, most recently from 02fb16a to 5241202 Compare October 8, 2025 14:43
@mrForza mrForza requested a review from Serpentian October 8, 2025 15:02
@mrForza mrForza assigned Serpentian and unassigned mrForza Oct 8, 2025
Copy link
Collaborator

@Serpentian Serpentian left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the patches. Looks good!

@Serpentian Serpentian requested a review from Gerold103 October 8, 2025 15:07
@Serpentian Serpentian assigned Gerold103 and mrForza and unassigned Serpentian Oct 8, 2025
@mrForza mrForza force-pushed the mrforza/ssl_security_fixes branch from 5241202 to f1a5e46 Compare October 8, 2025 15:17
Copy link
Collaborator

@Gerold103 Gerold103 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks 💪! Top work 🔥! I like how you've used this ticket as a chance to do a bit of cleanup around the related code.

@@ -0,0 +1,149 @@
local t = require('luatest')
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This part of the commit message:

Since ssl
decided not to backport it. Vshard will work safely with ssl only on
related problems occurred in large numbers of tarantool verions, we
3.5 or above versions.

I somehow couldn't parse it at all. Could you rephrase?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I can't parse it too) Something went wrong 🍸 🍾

Comment on lines +31 to +43
t.run_only_if(vutil.feature.ssl)
vtest.cluster_new(g, global_cfg)
vtest.cluster_bootstrap(g, global_cfg)
vtest.cluster_rebalancer_disable(g)

local new_cfg_template = table.deepcopy(cfg_template)
new_cfg_template.sharding[1].is_ssl = true
local new_global_cfg = vtest.config_new(new_cfg_template)

g.router = vtest.router_new(g, 'router', new_global_cfg)
vtest.cluster_cfg(g, new_global_cfg)
vtest.router_cfg(g.router, new_global_cfg)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do you need to start it without SSL and then configure it with SSL right away? Can you update the original cfg_template to simply already contain all the SSL params?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems that we cannot simply pass is_ssl = true into cfg_template. The storages cannot be configured correctly.

Part of the test:

local g = t.group('router_ssl')
local cfg_template = {
    sharding = {
        {
            replicas = {
                replica_1_a = {
                    master = true,
                },
                replica_1_b = {},
            },
            is_ssl = true,
        },
        {
            replicas = {
                replica_2_a = {
                    master = true,
                },
                replica_2_b = {},
            },
            is_ssl = true,
        },
    },
    bucket_count = 100,
}
local global_cfg = vtest.config_new(cfg_template)

g.before_all(function()
    t.run_only_if(vutil.feature.ssl)
    vtest.cluster_new(g, global_cfg)
    vtest.cluster_bootstrap(g, global_cfg)
    vtest.cluster_rebalancer_disable(g)

    g.router = vtest.router_new(g, 'router', global_cfg)
    vtest.router_cfg(g.router, global_cfg)
    vtest.cluster_wait_fullsync(g)
end)

The logs:

replica_1_b | Incorrect value for option 'replication': Invalid URI table: expected {uri = string, params = table} or {string, params = table} {"type":"ClientError","code":59,"name":"CFG","o
ption":"replication","details":"Invalid URI table: expected {uri = string, params = table} or {string, params = table}","trace":[{"file":"[C]","line":4294967295}]}
replica_1_b | fatal error, exiting the event loop
replica_1_a | Incorrect value for option 'replication': Invalid URI table: expected {uri = string, params = table} or {string, params = table} {"type":"ClientError","code":59,"name":"CFG","o
ption":"replication","details":"Invalid URI table: expected {uri = string, params = table} or {string, params = table}","trace":[{"file":"[C]","line":4294967295}]}
replica_1_a | fatal error, exiting the event loop
replica_2_b | Incorrect value for option 'replication': Invalid URI table: expected {uri = string, params = table} or {string, params = table} {"type":"ClientError","code":59,"name":"CFG","o
ption":"replication","details":"Invalid URI table: expected {uri = string, params = table} or {string, params = table}","trace":[{"file":"[C]","line":4294967295}]}
replica_2_b | fatal error, exiting the event loop
replica_2_a | Incorrect value for option 'replication': Invalid URI table: expected {uri = string, params = table} or {string, params = table} {"type":"ClientError","code":59,"name":"CFG","o
ption":"replication","details":"Invalid URI table: expected {uri = string, params = table} or {string, params = table}","trace":[{"file":"[C]","line":4294967295}]}
replica_2_a | fatal error, exiting the event loop
not ok 1    router_ssl.test_no_ssl_sensitive_data_in_storage_logs
#   fiber is cancelled
#   stack traceback:
#       ...e/mrforza/Desktop/vshard/test/luatest_helpers/server.lua:107: in function 'wait_for_readiness'
#       /home/mrforza/Desktop/vshard/test/luatest_helpers/vtest.lua:228: in function 'cluster_new'
#       ...sktop/vshard/test/router-luatest/router_2_2_ssl_test.lua:34: in function <...sktop/vshard/test/router-luatest/router_2_2_ssl_test.lua:32>
#       ...
#       [C]: in function 'xpcall'
# Ran 1 tests in 15.300 seconds, 0 succeeded, 1 errored

AFAIU, we firstly should configure the cluster without ssl and then reconfigure it with ssl options

t.assert_not(is_nop)
t.assert_equals(rs.master.uuid, g.replica_1_c:instance_uuid())

g.replica_1_b:thaw()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My bad, but the first commit actually closes the #597 and is needed for #593

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

Before this patch outdating the replicasets led to broken replicas
in logs. It happened because the outdated replicaset and replica
didn't have the correct tostring method. As a result, all replica's
json table is printed into logs which means that some sensitive
data could leak into it (e.g. ssl data, uri password and e.t.c.).

Now, we fix this issue by introducing custom tostring functions for
replicasets and replicas.

Closes tarantool#597
Needed for tarantool#593

NO_DOC=bugfix
This patch adds `(outdated)` prefix into tostring of outdated replicasets
and replicas in order to inform user about outdatedness. Also we change
the format of replica's tostring in order for it be consistent with
replicaset tostring.

Since the `failover/failover.test.lua` and
`misc/check_uuid_on_connect.test.lua` tests didn't expect another format
of replica's tostring, we accordingly changed them.

NO_DOC=minor
The `safe_uri` function is used in vshard to get a purged uri. Before
tarantool/tarantool#11810 patch sensitive ssl data may leaked into
logs as `uri.format` function didn't handle ssl params of uri. As a
result we don't fix this issue in vshard for those tarantool-ee versions
where `uri.format` works incorrectly.

This patch explicitly set `write_sensitive` option of `uri.format` into
`false` in `replica_safe_uri` and add some tests which check that
sensitive ssl data doesn't leak into logs of vshard storage and router.
Also we move all ssl related testcases into the `router_2_2_ssl_test`.

Closes tarantool#593

NO_DOC=bugfix
@mrForza mrForza force-pushed the mrforza/ssl_security_fixes branch from f1a5e46 to bf615e1 Compare October 10, 2025 10:25
@mrForza mrForza assigned Gerold103 and unassigned Gerold103 Oct 10, 2025
@mrForza mrForza requested a review from Gerold103 October 10, 2025 12:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Outdating the replicasets leads to broken replicas in logs Vshard leaks SSL password and options in logs

3 participants