How to run netdev selftests CI style

netdev selftest runner

This page describes how we run selftests in netdev CI. It should be helpful for reproducing CI failures, but it's not the one and only way the tests can be run!

Netdev CI: https://netdev.bots.linux.dev/status.html

"Flakes" view: https://netdev.bots.linux.dev/flakes.html

vng

We run tests in virtme-ng: https://github.com/arighi/virtme-ng virtme-ng builds the kernel and runs the tests.

Runner types

There are 3 runner types which can be distinguished by looking at the "Remote":

• runners called metal-${name} are netdev runners running with a normal / full performance kernel;
• runners called metal-${name}-dbg are netdev runners running with debug kernels;
• runners without metal- in the name are external, run by other teams / companies and reported to the system.

How to build

Kernels are built with just the relevant options enabled, for instance for net selftests:

vng --build  --config tools/testing/selftests/net/config

and for forwarding selftests we'd use the forwarding config:

vng --build  --config tools/testing/selftests/net/forwarding/config

The "-dbg" runners get extra debug options:

vng --build  \
        --config tools/testing/selftests/net/forwarding/config \
        --config kernel/configs/debug.config \
        --config kernel/configs/x86_debug.config

Tests are built separately, e.g. for forwarding:

make -C tools/testing/selftests/ TARGETS=net/forwarding

How to run

Executor runs tests one by one:

vng -v --run . --user root --cpus 4 -- \
        make -C tools/testing/selftests TARGETS=net TEST_PROGS=pmtu.sh TEST_GEN_PROGS="" run_tests

Tips

We found that following changes increase test flakiness and can help reproduce rare failures:

• Increase the number of CPUs with --cpus $number.
• Disable KVM support by passing --disable-kvm to vng.

Things not working:

• Skip the -v option if you don't want to see kernel logs.
• If QEMU fails to start try using --disable-microvm.
• some tests try to write into the PWD and virtme-ng gives by default RO access to the host filesystem, to override that add --rwdir tools/testing/selftests/net/