[BUG][TRANSPORT]: SSE generate_response uses client-facing URL for self-referencing RPC call, fails behind proxy/mesh

[scwhaley]

🐞 Bug Summary

I am using Hybrid Cloud Mesh to connect a MCP client in Fyre (Environment A) to a ContextForge instance in AWS (Environment B). SSE messages fail to generate a response due to ContextForge leveraging the URL used by the client's original SSE connection when calling it's own /rpc endpoint. This is because the client URL is only resolvable/reachable from the client's Fyre environment.

The setup:

• MCP Client is in environment A, ContextForge in environment B. Neither the client or ContextForge have a public endpoint.
• Mesh gateways in both environments, where Mesh creates a local endpoint in environment A that proxies traffic to ContextForge in environment B.
• This local endpoint only makes sense in environment A. For example, environment A is a 192.168.x.x private network, environment B is a 10.x.x.x private network.
• MCP Client reaches ContextForge at that local endpoint (i.e. 192.168.0.10).

The behavior:

1. Client connects to ContextForge at http://$ENV_A_ENDPOINT/servers/$SERVER_ID/sse , the SSE stream and session_id is set up correctly
2. Client sends a POST request to ContextForge at http://$ENV_A_ENDPOINT/servers/$SERVER_ID/message?session_id=$SESSION_ID
3. The /message handler in main.py > message_endpoint() broadcasts the message to the corresponding session using session_registry.broadcast()
4. To actually respond to the client, ContextForge handles the broadcasted message in session_registry.py > respond() > generate_response()
5. In generate_response(), ContextForge attempts to make a RPC call to ContextForge/itself using the URL that the original client used when setting up the SSE stream
6. The issue: Because the client reached ContextForge at a URL that only makes sense in environment A, the RPC call fails (since there is no corresponding endpoint in environment B)

---

🧩 Affected Component

Select the area of the project impacted:

• mcpgateway - API
• mcpgateway - UI (admin panel)
• mcpgateway.wrapper - stdio wrapper
• Federation or Transports
• CLI, Makefiles, or shell scripts
• Container setup (Docker/Podman/Compose)
• Other (explain below)

---

🔁 Steps to Reproduce

Steps are described above in Bug Summary

---

🤔 Expected Behavior

What should have happened instead?

ContextForge should not rely on the URL used by the client because it may not be resolvable/reachable in ContextForge's environment. It should handle the /rpc call internally rather than going back out over the network to simply reach itself.

---

📓 Logs / Error Output

Paste any relevant stack traces or logs here.
⚠️ Do not paste secrets, credentials, or tokens.

---

🧠 Environment Info

You can retrieve most of this from the /version endpoint.

Key	Value
Version or commit	1.0.0-RC1
Platform / OS	Ubuntu 24.04
Container	Kubernetes (microk8s)

---

🧩 Additional Context (optional)

Add any configuration details, flags, or related issues.

[crivetimihai]

Thanks for the thorough report and clear reproduction steps, @scwhaley — confirmed this is a real issue.

Root cause: session_registry.py:generate_response() (lines 1964-1981) constructs a self-referencing /rpc URL from the client-facing base_url (derived from request.base_url). When ContextForge is behind a proxy/mesh, this URL is only reachable from the client's network.

Short-term fix: Replace the 14-line URL parsing block with the internal loopback pattern already used in 5 other places in the codebase:

rpc_url = f"http://127.0.0.1:{settings.port}{settings.app_root_path}/rpc"

Longer-term: There are 5 other self-call sites (mcp_session_pool.py ×2, streamablehttp_transport.py ×2, main.py ×1) that all hardcode this URL independently — some inconsistently (missing app_root_path, mixed localhost/127.0.0.1). We'll consolidate these behind a single settings.internal_base_url property, which will also handle SSL mode and allow override via INTERNAL_BASE_URL for exotic network setups. That will be a separate chore.

[jonpspri]

Would calling handle_rpc() as suggested elsewhere work, and avoid an unnecessary engagement of the network layer? Calling the local URL is going to re-execute all middleware and authentication processes -- that might be a bad thing, or it might be a good thing.

[kevalmahajan]

@jonpspri Yes, i agree, calling an internal RPC handler directly is preferable. Hitting /rpc re-enters the full HTTP stack (middleware, auth, routing), adding unnecessary overhead. Direct invocation avoids the network layer and redundant processing.

Verifying both solutions, will select the appropriate one after considering all the aspects

[kevalmahajan]

Design Decision: HTTP Internal Call vs Internal Function Call

Why continuing with existing HTTP internal calls Instead of Direct Function Calls?

We chose to maintain HTTP-based internal RPC calls rather than refactoring to direct function calls for critical architectural reasons.



NOTE: Middleware reason and Security reason(since already call will be authenticated) could be bypassed and ignored for internal function calls but all the other reasons could not be ignored

Comparison

Aspect	HTTP Internal Call (Chosen)	Direct Function Call ( Rejected)
Multi-worker support	Works across Gunicorn workers	Process-local only
Session affinity	Routes to correct worker via headers	Cannot route between workers
Middleware execution	Auth, RBAC, rate limiting applied	Bypasses security middleware
Observability	Full tracing, metrics, logs	Invisible to monitoring
Horizontal scaling	Supports distributed deployment	Single-process limitation
Testing	Easy to mock and verify	Complex mocking required
Performance	~1ms overhead (localhost)	~0.01ms (negligible difference)
Code complexity	Single code path	Duplicate logic needed

Key Reasons

1. Multi-Worker Architecture: Production runs with multiple Gunicorn workers. HTTP calls work across workers; function calls don't.

2. Security: HTTP ensures all requests go through authentication and RBAC middleware. Direct calls would bypass security. Since the call is already authenticated even this can be ignored for internal function call.

3. Session Affinity: The x-mcp-session-id header routes requests to the worker holding the session. Function calls can't route.

4. Observability: HTTP calls are traced, logged, and monitored. Function calls are invisible to observability stack.

5. Scalability: HTTP supports distributed deployments and service mesh. Function calls limit to single-process.

Performance Trade-off

• HTTP overhead: ~1ms per call (localhost, no network)
• Benefit: Maintains architecture, security, and scalability
• Verdict: 1ms overhead is negligible compared to architectural benefits

Conclusion

HTTP internal calls are the correct architectural choice for a production-grade gateway that must support multi-worker deployment, enforce security policies, maintain observability, and scale horizontally. The minimal performance cost is vastly outweighed by these critical capabilities.