Simplify the usual run-tests workflow

[wingo]

The README documents a relatively involved test runner invocation:

python3 test-runner/wasi_test_runner.py                                                  \
   -t ./tests/assemblyscript/testsuite/ `# path to folders containing .wasm test files` \
      ./tests/c/testsuite/wasm32-wasip1                                                 \
      ./tests/rust/testsuite/                                                           \
   -r adapters/wasmtime.py # path to a runtime adapter

OK, it needs to be updated for the new paths. And to add wasip3 for Rust; and somehow to invoke the "adapter" such that it can apply different command-line flags to different versions (wasip1 vs wasip3).

It occurs to me that we should try to be more simple for the default run. I basically want to be able to just ./run-tests. Some ideas to get us there:

• New front-end script at the top level; uses the existing code tho
• Instead of specifying paths to test suite dirs manually, just traverse subdirs looking for manifest.json files; those are the available tests
• Start a test run by looking for all adapters (WASI implementations), and querying them for which wasi versions they support; run them all.
• Add a simple regex-based "includelist" to the command-line, as in the JS engine's test runners

Thoughts? @pchickey @loganek @alexcrichton ?

[alexcrichton]

I'm not familiar enough with the setup in this repo to comment too much, but I can echo the sentiment that awhile back when I wanted to add a new test and run it I found it pretty complicated to do so. I was looking for "put the file here then run this one command" but couldn't quite figure out the invocation and had to string together a really big shell "one-liner".

I like what you're describing here. IMO test-running should be pretty automatic with, as you say, ./run-tests doing everything effectively including test discovery, building tests, loading runtimes, etc. That can have all sorts of flags to configure behavior but by default trying to build/run everything I think would be good.

[wingo]

A start on this is at #149 and #152. Still missing:

• Query wasm runtimes for which versions of wasi they support; don't assume that a wasi 0.1 impl will support wasi 0.3
• Includelist not implemented
• Need to allow wasm runtimes to pass different flags for different wasi versions
• Make it easier to add tests
• Add jco adapter

[wingo]

An update: I merged in the implementation of the simpler run-tests command in #149 and #152. In #162 and #163 I make it so that we skip tests if a runtime doesn't support that WASI version, and arrange to pass appropriate command-line args to tell the runtime which version we are testing.

Includelist not implemented, though now run-tests prints out command lines that can be used to re-run failures; going to punt on include filters for the time being.

I will write up a doc on adding tests.

I looked at getting a jco adapter but I seem to be holding it wrong: #wasi > jco and wasip3. Anyway it's pretty clear what needs doing there.

When I get the test-adding doc I will close this bug report.

[wingo]

All merged, except the jco adapter over at #165. Thanks for feedback!