Add support for cpputestTestAdapter.preLaunchTask setting

[malsyned]

The configured task will be run at the beginning of load(), run(), and debug(). This can be used to rebuild the test executable.

Addresses issue #42

One minor issue I ran into is that when first opening a CMake project with "cmake.configureOnOpen": true, there's a race that leads to an error popup, "Configuration is already in progress". The only way to solve this that I have found is to set cmake.configureOnOpen to false, and to make the CMake Build task in tasks.json depend on the CMake Configure task. It works, but it creates a little bit of latency in various operations and requires creating custom tasks instead of being able to rely on the default templates from CMake Tools.

If the task process returns a nonzero error code, that will be reported by emitting a TestLoadFinishedEvent event with an errorMessage instead of a suite.

This required some changes to how run() decides which tests to run. If you're interested, I can submit another PR that uses the errorMessage feature to report test runner failures as well instead of CreateTestSuiteError(), if you're interested. Conversely, if you'd rather I mimic CreateTestSuiteError() instead for this PR, I'd be happy to make that change.

[bneumann]

Anyway we can fix this race condition? It should only happen when all tests configure in parallel right? So maybe there is something in Cmake that let's us check if we need the configuration to run beforehand?

[bneumann]

Nice! That's a good feature

[bneumann]

Nicely done. If you want you could make it more "promisey" e.g. in the updateTests() throw an error to propagate it up and in the calling methods do something like:

this.updateTests
.then(success => {
...
})
.catch(err => this.testsEmitter...)

But I am fine with it as it is

[malsyned]

Any reason to use .then().catch() instead of ordinary try/catch blocks? All the callers of updateTests() are already async methods, doesn't that result in the same behavior under the hood? I am not as fluent in TypeScript's async/Promises as I am in, e.g., Python's, but I thought those were equivalent ways of saying the same thing.

[bneumann]

Technically it is almost the same. It is more a style thing, but I was just perusing the rest of the code and I remember that I wanted to make it more functional but didn't go through all of it anyway. So it's fine as it is. I will merge it and maybe refactor "later" :)

[malsyned]

Anyway we can fix this race condition? It should only happen when all tests configure in parallel right? So maybe there is something in Cmake that let's us check if we need the configuration to run beforehand?

I did some digging about this for a while and didn't find anything that looked immediately promising. I'll say a bit more about how it's all working, maybe you will see something I'm missing or know some tricks that I don't.

This feature doesn't know anything about CMake specifically. It is designed to work with any build system by using tasks. These can either be tasks provided by an extension, e.g. "build" from CMake Tools, or a custom task configured in tasks.json. In my case, I'm using a tasks.json task with "type": "cmake", "command": "build", which is what gets created when you use CMake Tools' "Configure Task" command.

When the CMake Tools extension is running a configure at project open time because of cmake.configureOnOpen, it doesn't use the Tasks subsystem to do it, it just runs its own cmake.configure command. So it doesn't show up in vscode.tasks.taskExecutions or anything.

So the issue is that when a new folder is opening, CMake Tools cmake.configure is running concurrently with CppUTest-Adapter's adapter.load(), and I don't see a way to detect that situation. I read through the CMake Tools extension's public API but nothing useful leapt out to me immediately. Even if it did offer something, that would add an additional dependency on CMake Tools, which we don't currently depend on directly.

You could make the case that the "right" place to fix this is in the CMake Tools extension, by making configure and build commands wait for previous commands to complete rather than throwing an error.

As I was researching to make sure I had all my ducks in a row for this explanation, I did come across another potential workaround from the one I mentioned in a comment on #42. I don't know if it will work in practice, but if it does the tradeoffs feel better to me than the ones I'm making now.

• Modify updateTasks() to wait for vscode.tasks.taskExecutions to be empty. I'd do the "waiting" portion using an onDidEndTask event handler so that it didn't hang up the UI.
• set "cmake.configureOnOpen": false in .vscode/settings.json
• Configure an explicit task in tasks.json for CMake configure, with "runOptions": { "runOn": "folderOpen" }

Downsides to this approach are:

• You can't use the automatically-created default CMake Tools tasks, you have to create the task in tasks.json, so the "simplest possible project" is a bit less simple
• You have to have "Allow Automatic Tasks" configured in VSCode, which could be considered a security risk (and I bet is why CMake Tools has its own otherwise-redundant setting)
• Even if it works, it still may suffer from a race condition where the task that runs on folder open starts second and fails to execute, so the build task still has to depend unconditionally on the configure task, which adds latency (~500ms on my machine).

The longer I look at this, the more I don't see a way out aside from patching the CMake Tools extension. Do you see something I'm overlooking?

[malsyned]

I have a couple of additional small quality-of-life changes I may make to this PR. It's mergable as-is but you may want to hold off for a day or two while I tweak it if you want to avoid PR churn.

[bneumann]

Wow, that was a very concise explanation of the problem. Thanks for that and no, I don't see a way either except maybe checking running process on the OS level to detect cmake doing something. But then again you can't be sure it's not your own process that is running.