Skip to content

Add filter state receive_before_connect to tcp_proxy #38189

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 17 commits into from
Feb 20, 2025

Conversation

akshita31
Copy link
Contributor

@akshita31 akshita31 commented Jan 25, 2025

Commit Message:
This PR picks up the work done in #25804. It can help to fix #9023 by using the receive_before_connect filter state.

  1. It adds a receive_before_connect filter state which can be read by TCP_PROXY and if set TCP_PROXY will not disable downstream reads until the upstream connection is established. This can hence be used by filters before TCP_PROXY to set the filter state in initializeReadFilterCallbacks, and then StopIteration in onNewConnection and onData, until they have read the required amount of data before upstream connection establishment.
  2. Based on feedback in tcp_proxy: Add filter state envoy.tcp_proxy.receive_before_connect #25804 , this PR adds a flow control to prevent early_data_buffer from overflowing, by disabling downstream reads on early data until the upstream connection establishment is finished.
  3. Adds integration and unit tests to cover the newly added functionality.

Additional Description:
Risk Level: Low
Testing: Added integration and unit tests
Docs Changes: Done
Release Notes:
Platform Specific Features:
[Optional Runtime guard:]
[Optional Fixes #Issue]
[Optional Fixes commit #PR or SHA]
[Optional Deprecated:]
[Optional API Considerations:]

Copy link

Hi @akshita31, welcome and thank you for your contribution.

We will try to review your Pull Request as quickly as possible.

In the meantime, please take a look at the contribution guidelines if you have not done so already.

🐱

Caused by: #38189 was opened by akshita31.

see: more, trace.

@akshita31
Copy link
Contributor Author

akshita31 commented Jan 25, 2025

cc @hbhasker @dwj300 Opened another PR as the #36581 was closed out due to being stale.

Copy link
Member

@ggreenway ggreenway left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/wait

@cccsss01
Copy link

Can we get this added please, believe this immensely important.

@akshita31
Copy link
Contributor Author

@cccsss01 I am working on it and will try to get it merged as soon as possible, it is a really helpful feature for us as well.

@ggreenway
Copy link
Member

@akshita31 still waiting on comment #38189 (comment)

/wait

Add filter state object bool proxy_receive_before_connect that tcp_proxy
filter checks for and when set to 'true' will not set readDisable on the
downstream connection before upstream connection is established.

This change allows advanced cases where a network read filter needs to
receive (and possibly send) downstream data that may change the
destination of the upstream TCP connection, e.g. via metadata set based
on the received data. If this is the case, then the read filter preceding
the tcp_proxy filter must return StopIteration from its onNewConnection()
call, causing tcp_proxy filter to postpone its upstream connection
establishment until onData() returns Continue.

Any data reaching the tcp_proxy filter before the upstream connection is
established is buffered so that the downstream filters do not see the
same data again which would be the case if it would remaining in the buffer
and more data is received. This also allows downstream filters inject
data before the upstream connection is established; such injected data
would be lost if tcp_proxy would not buffer it while connection
establishment is still ongoing.

An existing dynamic metadata integration test is modified to use
proxy_receive_before_connect=true. This makes the use case less
reliant on the tcp_proxy internal detail, such as balancing and timing of
the readDisable() calls on the downstream connection.

Signed-off-by: Jarno Rajahalme <[email protected]>
Signed-off-by: Jarno Rajahalme <[email protected]>
Signed-off-by: Akshita Agarwal <[email protected]>
@ggreenway
Copy link
Member

Please do not rebase or force-push your PR:

* Once your PR is under review, *please do not rebase it*. If you rebase, you will need to force push to

Copy link
Member

@ggreenway ggreenway left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/wait

Akshita Agarwal added 2 commits February 10, 2025 18:15
Signed-off-by: Akshita Agarwal <[email protected]>
Signed-off-by: Akshita Agarwal <[email protected]>
@ggreenway
Copy link
Member

Please do not rebase or force-push your PR:

* Once your PR is under review, *please do not rebase it*. If you rebase, you will need to force push to

Please stop force-pushing to the PR! It makes it very difficult to review.

/wait-any

@akshita31
Copy link
Contributor Author

@ggreenway apology for the issue, I didn't remember the contributing guide well previously. But after your comment, the last 2 commits I pushed preserve the commit history. I hadn't signed the last 2 commit so I had to sign and force push as per the DCO instructions.

Apology for the inconvenience, I will make sure to sign and not force push now. I just have last few feedback comments to resolve. Thanks a lot for your patience and understanding.

Akshita Agarwal added 3 commits February 11, 2025 00:16
Signed-off-by: Akshita Agarwal <[email protected]>
Signed-off-by: Akshita Agarwal <[email protected]>
@ggreenway
Copy link
Member

/wait

Akshita Agarwal added 3 commits February 12, 2025 11:42
Signed-off-by: Akshita Agarwal <[email protected]>
Signed-off-by: Akshita Agarwal <[email protected]>
Signed-off-by: Akshita Agarwal <[email protected]>
@akshita31
Copy link
Contributor Author

/wait

Signed-off-by: Akshita Agarwal <[email protected]>
Copy link
Member

@ggreenway ggreenway left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good overall. If you're getting useful information from the new metric, I'm fine leaving it in. It's also useful for tests, although if we wanted to remove it you could add a trace log in the code and have the test check for the log message.

You need to document the new metric at https://github.com/envoyproxy/envoy/blob/main/docs/root/configuration/listeners/network_filters/tcp_proxy_filter.rst?plain=1#L75 if you want to keep it.

/wait

Signed-off-by: Akshita Agarwal <[email protected]>
@ggreenway ggreenway merged commit bcaa95f into envoyproxy:main Feb 20, 2025
24 checks passed
@akshita31 akshita31 deleted the akshita-tcp-proxy branch March 12, 2025 18:34
agrawroh pushed a commit to agrawroh/envoy that referenced this pull request Apr 9, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
5 participants