server: add rendezvous mode

Signed-off-by: hexian000 <[email protected]>

README.md

@@ -14,17 +14,19 @@ Status: **Stable**
 - [Introduction](#introduction)
 - [Features](#features)
 - [Security](#security)
-	- [Encryption](#encryption)
-	- [Obfuscation](#obfuscation)
+  - [Encryption](#encryption)
+  - [Obfuscation](#obfuscation)
 - [Compatibility](#compatibility)
-	- [System](#system)
-	- [Version Compatibility](#version-compatibility)
+  - [System](#system)
+  - [Version Compatibility](#version-compatibility)
 - [Build](#build)
-	- [Dependencies](#dependencies)
-	- [Build on Unix-like systems](#build-on-unix-like-systems)
+  - [Dependencies](#dependencies)
+  - [Build on Unix-like systems](#build-on-unix-like-systems)
 - [Runtime](#runtime)
-	- [Dependencies](#dependencies-1)
-	- [Configurations](#configurations)
+  - [Dependencies](#dependencies-1)
+  - [Configurations](#configurations)
+    - [Basic Usage](#basic-usage)
+    - [Rendezvous Mode](#rendezvous-mode)
 - [Tunables](#tunables)
 - [Observability](#observability)
 - [Credits](#credits)
@@ -48,6 +50,8 @@ network access -> proxy client -> kcptun-libev client ->
 -> kcptun-libev server -> proxy server -> stable network
 ```
 
+Less typically, reliable UDP can help users connect to TCP services behind NAT, see [rendezvous mode](#rendezvous-mode).
+
 Since KCP retransmits packets more aggressively. It is recommended to enable proper QoS at the NIC level when running on a public network.
 
 Read more about [KCP](https://github.com/skywind3000/kcp/blob/master/README.en.md)
@@ -59,7 +63,8 @@ Read more about [KCP](https://github.com/skywind3000/kcp/blob/master/README.en.m
 - Proper: KCP will be flushed on demand, no mechanistic lag introduced.
 - Simple: Do one thing well. kcptun-libev only acts as a layer 4 forwarder.
 - Morden: Full IPv6 support.
-- DDNS aware: Dynamic IP addresses are automatically resolved.
+- DDNS aware: Dynamic IP addresses can be automatically resolved.
+- NAT traversal: Server behind certain types of NAT can be connected directly with the help of a well-known rendezvous server.
 - Configurable: If you plan to use with another encryption implementation (such as udp2raw, wireguard, etc.), encryption can be completely disabled or even excluded from build.
 - Portable: Compliant with ISO C standard. Support both GNU/Linux and POSIX APIs.
 
@@ -186,14 +191,15 @@ opkg install libev libsodium
 ```
 
 ### Configurations
+#### Basic Usage
 
 Generate a random key for encryption:
 
 ```sh
 ./kcptun-libev --genpsk xchacha20poly1305_ietf
 ```
 
-Create a server.json file and fill in the options:
+Create a `server.json` file and fill in the options:
 
 ```json
 {
@@ -210,7 +216,7 @@ Start the server:
 ./kcptun-libev -c server.json
 ```
 
-Create a client.json file and fill in the options:
+Create a `client.json` file and fill in the options:
 
 ```json
 {
@@ -235,7 +241,47 @@ Let's explain some common fields in server.json/client.json:
 - The client side "listen" TCP ports and send data to "kcp_connect".
 - The server side receive data from "kcp_bind" and forward the connections to "connect".
 - Set a "password" or "psk" is strongly suggested when using in public networks.
-- "loglevel": 0-7 are Silence, Fatal, Error, Warning, Notice, Info, Debug, Verbose respectively. The default is 4 (NOTICE). High log levels can affect performance.
+- "loglevel": 0-7 are Silence, Fatal, Error, Warning, Notice, Info, Debug, Verbose respectively. The default is 4 (Notice). High log levels can affect performance.
+
+#### Rendezvous Mode
+
+Rendezvous mode may be useful for accessing servers behind NAT. The rendezvous server only helps establish the connection, the traffic goes directly between client and server.
+
+*The method is non-standard and may not work with all NAT implementations.*
+
+`rendezvous_server.json`: The rendezvous server should have an address which is reachable by both client and server.
+
+```json
+{
+    "kcp_bind": "0.0.0.0:12345",
+    "method": "xchacha20poly1305_ietf",
+    "psk": "// your key here"
+}
+```
+
+`server.json`: The server may be behind one or more levels of NAT.
+
+```json
+{
+    "connect": "127.0.0.1:1080",
+    "rendezvous_server": "203.0.113.1:12345",
+    "method": "xchacha20poly1305_ietf",
+    "psk": "// your key here"
+}
+```
+
+`client.json`: The client may be behind one or more levels of NAT, which may or may not be the same ones as the server.
+
+```json
+{
+    "listen": "127.0.0.1:1080",
+    "rendezvous_server": "203.0.113.1:12345",
+    "method": "xchacha20poly1305_ietf",
+    "psk": "// your key here"
+}
+```
+
+rendezvous_server : server : client = 1 : 1 : n
 
 ## Tunables
 
@@ -244,13 +290,13 @@ Let's explain some common fields in server.json/client.json:
 Some tunables are the same as [KCP](https://github.com/skywind3000/kcp), read their docs for full explaination. Here are some hints:
 
 - "kcp.sndwnd", "kcp.rcvwnd":
-	1. Should be tuned according to RTT.
-	2. For enthusiasts, you can start an idle client with loglevel >= 5 and wait 1 minute to check the theoretical bandwidth of current window values.
-	3. On systems with very little memory, you may need to reduce it to save memory.
+  1. Should be tuned according to RTT.
+  2. For enthusiasts, you can start an idle client with loglevel >= 5 and wait 1 minute to check the theoretical bandwidth of current window values.
+  3. On systems with very little memory, you may need to reduce it to save memory.
 - "kcp.nodelay": Enabled by default. Note that this is not an equivalent to `TCP_NODELAY`.
 - "kcp.interval":
-	1. Since we run KCP differently, the recommended value is longer than the previous implementation. This will save some CPU power.
-	2. This option is not intended for [traffic shaping](https://en.wikipedia.org/wiki/Traffic_shaping). For Linux, check out [sqm-scripts](https://github.com/tohojo/sqm-scripts) for it. Read more about [CAKE](https://man7.org/linux/man-pages/man8/CAKE.8.html).
+  1. Since we run KCP differently, the recommended value is longer than the previous implementation. This will save some CPU power.
+  2. This option is not intended for [traffic shaping](https://en.wikipedia.org/wiki/Traffic_shaping). For Linux, check out [sqm-scripts](https://github.com/tohojo/sqm-scripts) for it. Read more about [CAKE](https://man7.org/linux/man-pages/man8/CAKE.8.html).
 - "kcp.resend": Disabled by default.
 - "kcp.nc": Enabled by default.
 - "kcp.mtu": Specifies the final IP packet size, including all overhead.
@@ -259,9 +305,9 @@ Again, there is some kcptun-libev specific options:
 
 - "kcp.flush": 0 - periodic only, 1 - flush after sending, 2 - also flush acks (for benchmarking)
 - "tcp.sndbuf", "tcp.rcvbuf", "udp.sndbuf", "udp.rcvbuf": Socket options, see your OS manual for further information.
-	1. Normally, default value just works.
-	2. Usually setting the udp buffers relatively large (e.g. 1048576) gives performance benefits. But since kcptun-libev handles packets efficiently, a receive buffer that is too large doesn't make sense.
-	3. All buffers should not be too small, otherwise you may experience performance degradation.
+  1. Normally, default value just works.
+  2. Usually setting the udp buffers relatively large (e.g. 1048576) gives performance benefits. But since kcptun-libev handles packets efficiently, a receive buffer that is too large doesn't make sense.
+  3. All buffers should not be too small, otherwise you may experience performance degradation.
 - "user": if running as root, switch to this user to drop privileges, e.g. "nobody"
 
 ## Observability

m.sh

@@ -125,13 +125,14 @@ case "$1" in
         -S "." -B "build"
     find contrib src -name '*.c' | while read -r FILE; do
         echo "#include \"${FILE}\""
-    done | gcc -pipe -O2 -g -DNDEBUG -D_GNU_SOURCE -pedantic -Wall -Wextra -std=gnu11 \
-        -Icontrib/csnippets -Icontrib -Isrc -include build/src/config.h \
+    done | gcc -pipe -O3 -s -DNDEBUG -D_GNU_SOURCE -pedantic -Wall -Wextra -std=c11 \
+        -Icontrib/csnippets -Icontrib/json -Icontrib/kcp -Icontrib/libbloom -Isrc \
+	-include build/src/config.h \
         -o "build/src/kcptun-libev" -xc - -lev -lsodium -lm
     ls -lh "build/src/kcptun-libev"
     ;;
 "d")
-    find src -name '*.c' -o -name '*.h' | xargs clang-format -i
+    find src -type f -regex '.*\.[hc]' -exec clang-format -i {} +
     # debug
     rm -rf "build" && mkdir -p "build"
     cmake -G "${GENERATOR}" \

src/conf.c

@@ -2,11 +2,11 @@
  * This code is licensed under MIT license (see LICENSE for details) */
 
 #include "conf.h"
+#include "utils/debug.h"
 #include "utils/slog.h"
 #include "util.h"
 #include "jsonutil.h"
 
-#include <assert.h>
 #include <stdbool.h>
 #include <stddef.h>
 #include <stdio.h>
@@ -154,6 +154,9 @@ static bool main_scope_cb(
 	} else if (NAME_EQUAL("kcp_connect")) {
 		conf->kcp_connect = jutil_strdup(value);
 		return conf->kcp_connect != NULL;
+	} else if (NAME_EQUAL("rendezvous_server")) {
+		conf->rendezvous_server = jutil_strdup(value);
+		return conf->rendezvous_server != NULL;
 	} else if (NAME_EQUAL("http_listen")) {
 		conf->http_listen = jutil_strdup(value);
 		return conf->http_listen != NULL;
@@ -199,15 +202,15 @@ static bool main_scope_cb(
 
 #undef NAME_EQUAL
 
-const char *runmode_str(const int mode)
+const char *conf_modestr(const struct config *restrict conf)
 {
-	static const char *str[] = {
-		[MODE_SERVER] = "server",
-		[MODE_CLIENT] = "client",
-	};
-	assert(mode >= 0);
-	assert((size_t)mode < ARRAY_SIZE(str));
-	return str[mode];
+	if (conf->mode & MODE_SERVER) {
+		return "server";
+	}
+	if (conf->mode & MODE_CLIENT) {
+		return "client";
+	}
+	return "rendezvous server";
 }
 
 static struct config conf_default(void)
@@ -249,13 +252,23 @@ static bool conf_check(struct config *restrict conf)
 {
 	/* 1. network address check */
 	int mode = 0;
-	if (conf->kcp_bind != NULL && conf->connect != NULL) {
+	if (conf->connect != NULL) {
 		mode |= MODE_SERVER;
 	}
-	if (conf->listen != NULL && conf->kcp_connect != NULL) {
+	if (conf->listen != NULL) {
 		mode |= MODE_CLIENT;
 	}
-	if (mode != MODE_SERVER && mode != MODE_CLIENT) {
+	if (conf->rendezvous_server != NULL) {
+		mode |= MODE_RENDEZVOUS;
+		if (conf->keepalive <= 0) {
+			LOGE("config: keepalive can't be disabled in rendezvous mode");
+			return false;
+		}
+	}
+	if (((mode & (MODE_RENDEZVOUS | MODE_SERVER)) == MODE_SERVER &&
+	     conf->kcp_bind == NULL) ||
+	    ((mode & (MODE_RENDEZVOUS | MODE_CLIENT)) == MODE_CLIENT &&
+	     conf->kcp_connect == NULL)) {
 		LOGE("config: no forward could be provided (are you missing some address field?)");
 		return false;
 	}
@@ -333,6 +346,7 @@ void conf_free(struct config *conf)
 	UTIL_SAFE_FREE(conf->connect);
 	UTIL_SAFE_FREE(conf->kcp_bind);
 	UTIL_SAFE_FREE(conf->kcp_connect);
+	UTIL_SAFE_FREE(conf->rendezvous_server);
 	UTIL_SAFE_FREE(conf->http_listen);
 	UTIL_SAFE_FREE(conf->netdev);
 	UTIL_SAFE_FREE(conf->user);

src/conf.h

@@ -12,17 +12,17 @@ struct sockaddr;
 enum runmode {
 	MODE_SERVER = 1 << 0,
 	MODE_CLIENT = 1 << 1,
+	MODE_RENDEZVOUS = 1 << 2,
 };
 
-const char *runmode_str(int mode);
-
 struct config {
-	const char *netdev;
 	const char *listen;
 	const char *connect;
 	const char *kcp_bind;
 	const char *kcp_connect;
+	const char *rendezvous_server;
 	const char *http_listen;
+	const char *netdev;
 
 	int mode;
 	int kcp_mtu, kcp_sndwnd, kcp_rcvwnd;
@@ -50,6 +50,7 @@ struct config {
 };
 
 struct config *conf_read(const char *path);
+const char *conf_modestr(const struct config *conf);
 void conf_free(struct config *conf);
 
 #endif /* CONF_H */

src/event.h

@@ -30,7 +30,7 @@ void resolve_cb(struct ev_loop *loop, struct ev_timer *watcher, int revents);
 void timeout_cb(struct ev_loop *loop, struct ev_timer *watcher, int revents);
 void http_accept_cb(struct ev_loop *loop, struct ev_io *watcher, int revents);
 
-int udp_output(const char *buf, int len, struct IKCPCB *kcp, void *user);
+int kcp_output(const char *buf, int len, struct IKCPCB *kcp, void *user);
 bool kcp_sendmsg(struct session *ss, uint16_t msg);
 bool kcp_push(struct session *ss);
 void kcp_recv(struct session *ss);

src/event_http.c

@@ -14,7 +14,6 @@
 #include "obfs.h"
 
 #include <ev.h>
-
 #include <unistd.h>
 #include <sys/socket.h>
 
@@ -66,7 +65,7 @@ void http_accept_cb(struct ev_loop *loop, struct ev_io *watcher, int revents)
 	CHECK_EV_ERROR(revents);
 
 	struct server *restrict s = watcher->data;
-	sockaddr_max_t addr;
+	union sockaddr_max addr;
 	socklen_t addrlen = sizeof(addr);
 	const int fd = accept(watcher->fd, &addr.sa, &addrlen);
 	if (fd < 0) {

src/event_impl.h

@@ -4,6 +4,8 @@
 #ifndef EVENT_IMPL_H
 #define EVENT_IMPL_H
 
+#include "utils/slog.h"
+
 #include <ev.h>
 
 #include <errno.h>

src/event_kcp.c

@@ -24,7 +24,7 @@
 #include <stdint.h>
 #include <string.h>
 
-int udp_output(const char *buf, int len, ikcpcb *kcp, void *user)
+int kcp_output(const char *buf, int len, ikcpcb *kcp, void *user)
 {
 	UNUSED(kcp);
 	assert(len > 0 && len < MAX_PACKET_SIZE);
@@ -35,9 +35,9 @@ int udp_output(const char *buf, int len, ikcpcb *kcp, void *user)
 		LOGOOM();
 		return -1;
 	}
-	memcpy(&msg->addr.sa, &ss->raddr.sa, getsocklen(&ss->raddr.sa));
-	unsigned char *kcp_packet = msg->buf + msg->off;
-	memcpy(kcp_packet, buf, len);
+	msg->addr = ss->raddr;
+	unsigned char *dst = msg->buf + msg->off;
+	memcpy(dst, buf, len);
 	msg->len = len;
 	s->stats.kcp_tx += len;
 	ss->stats.kcp_tx += len;

src/event_tcp.c

@@ -16,9 +16,9 @@
 #include <stdbool.h>
 #include <sys/socket.h>
 #include <unistd.h>
-#include <errno.h>
 
 #include <assert.h>
+#include <errno.h>
 #include <stddef.h>
 #include <stdint.h>
 #include <inttypes.h>
@@ -93,7 +93,7 @@ void tcp_accept_cb(struct ev_loop *loop, struct ev_io *watcher, int revents)
 	const struct config *restrict conf = s->conf;
 
 	for (;;) {
-		sockaddr_max_t addr;
+		union sockaddr_max addr;
 		socklen_t addrlen = sizeof(addr);
 		/* accept client request */
 		const int fd = accept(watcher->fd, &addr.sa, &addrlen);
@@ -128,6 +128,11 @@ void tcp_accept_cb(struct ev_loop *loop, struct ev_io *watcher, int revents)
 		socket_set_tcp(fd, conf->tcp_nodelay, conf->tcp_keepalive);
 		socket_set_buffer(fd, conf->tcp_sndbuf, conf->tcp_rcvbuf);
 
+		if (!s->pkt.connected) {
+			LOGE("packet connection is not ready, refusing");
+			CLOSE_FD(fd);
+			return;
+		}
 		accept_one(s, fd, &addr.sa);
 	}
 }
@@ -322,7 +327,7 @@ void tcp_socket_cb(struct ev_loop *loop, struct ev_io *watcher, int revents)
 	UNUSED(loop);
 	CHECK_EV_ERROR(revents);
 
-	LOGD_F("io fd=%d revents=0x%x", watcher->fd, revents);
+	LOGV_F("io fd=%d revents=0x%x", watcher->fd, revents);
 	struct session *restrict ss = watcher->data;
 	if (ss->tcp_state == STATE_CONNECT) {
 		connected_cb(ss);