Benchmark CMSIS Conv & DW Conv and improve operator selection

[rascani]

Here we actually have the benefit of choosing between a regular and DW conv. It is likely but not certain that the un-optimized CMSIS-NN DW conv or the one without any SIMD is less efficient that the corresponding CMSIS-NN conv. We don't know exactly until we measure. We could then add something like this for now with a TODO comment:
optimal_dw_conv_constraints = (
in_channels == out_channels and dilation == [1, 1]
) or in_channels == 1

Originally posted by @mansnils in #16233 (comment)