tcpdrop_example.txt

Demonstrations of tcpdrop, the Linux BPF/bcc version.


tcpdrop prints details of TCP packets or segments that were dropped by the
kernel, including the kernel stack trace that led to the drop:

# ./tcpdrop.py
TIME     PID     IP SADDR:SPORT          > DADDR:DPORT          STATE (FLAGS)        REASON (CODE)
14:46:49 0       4  39.156.66.10:80      > 10.211.55.10:33280   SYN_SENT (SYN|ACK)   NETFILTER_DROP (12)
	b'__traceiter_kfree_skb+0x90'
	b'__traceiter_kfree_skb+0x90'
	b'sk_skb_reason_drop+0x1e4'
	b'nft_do_chain+0x93c'
	b'nft_do_chain_ipv4+0x16c'
	b'nf_hook_slow+0xb0'
	b'ip_local_deliver+0x244'
	b'ip_sublist_rcv_finish+0xec'
	b'ip_sublist_rcv+0x32c'
	b'ip_list_rcv+0x210'
	b'__netif_receive_skb_list_core+0x348'
	b'netif_receive_skb_list_internal+0x498'
	b'napi_complete_done+0x190'
	b'virtnet_poll+0x10a8'
	b'__napi_poll+0xa4'
	b'net_rx_action+0x460'
	b'handle_softirqs+0x304'
	b'__do_softirq+0x20'
	b'____do_softirq+0x1c'
	b'call_on_irq_stack+0x3c'
	b'do_softirq_own_stack+0x28'
	b'__irq_exit_rcu+0x384'
	b'irq_exit_rcu+0x1c'
	b'el1_interrupt+0x4c'
	b'el1h_64_irq_handler+0x1c'
	b'el1h_64_irq+0x84'
	b'do_idle+0x31c'
	b'cpu_startup_entry+0x6c'
	b'rest_init+0x170'
	b'start_kernel+0x314'
	b'__primary_switched+0x94'

[...]

The last four columns show the state of the TCP session, the TCP flags, the
reason for the packet drop, and the corresponding reason code. In this
example, a packet arriving for a session in the `SYN_SENT` state with
`SYN|ACK` flags was dropped by the kernel due to the reason `NETFILTER_DROP`.

This tool is useful for debugging high rates of drops, which can cause the
remote end to do timer-based retransmits, hurting performance.


USAGE:

# ./tcpdrop.py -h
usage: tcpdrop.py [-4 | -6] [-h]

Trace TCP drops by the kernel

optional arguments:
  -4, --ipv4  trace IPv4 family only
  -6, --ipv6  trace IPv6 family only
  -h, --help  show this help message and exit

examples:
    ./tcpdrop           # trace kernel TCP drops
    ./tcpdrop -4        # trace IPv4 family only
    ./tcpdrop -6        # trace IPv6 family only