propose build_dir in data.yaml as only relevant for GitData(StaticData)

[vandome]

Was considering putting build_dir as a top-level option, e.g. in config.yaml, and then pass it through the DatatStore instantiation as argument alongside data_dir. However, this requires lots of additions of build_dir=None as keyword argument in multiple levels of class instantiations. For example:

class GitData(StaticData):
    def __init__(self, data_dir, name, options, build_dir=None):
        build_dir = build_dir if build_dir is not None else os.path.join(data_dir, "git")

and then further above:

def parse_static_data_item(data_dir, name, options, build_dir=None):
    type = options["type"]
    ...
    elif type == "git":
        data = GitData(data_dir, name, options, build_dir=build_dir)

and also

class StaticDataStore:
    def __init__(self, data_dir: str, static_data_dict: dict, build_dir=None):
        self.static_data = {}
        for name, options in static_data_dict.items():
            data = parse_static_data_item(data_dir, name, options, build_dir=build_dir)
            self.static_data[name] = data

so that we can put in nodes.py or equivalent:

self.output_root = options['output_root']
self.build_dir = options.get('build_dir', os.path.join(self.output_root, 'build'))
static_data = options.get('static_data', {})
self.static_data = wellies.StaticDataStore(self.data_dir, static_data, build_dir=self.build_dir)

And ultimately is the build_dir the same for all git repos?

My understanding is that build_dir is only relevant for GitData(StaticData) class. Perhaps it's better to provide it with the static_data options?
The price is potential duplication of paths defined in config.yaml and data.yaml if non-standard, i.e. other than $TMPDIR/build.

I leave it to the developers to choose what they prefer as best aligned with the wellies spirit. I provide both options in two different commits.

[FussyDuck]

All committers have signed the CLA.

[corentincarton]

Thanks @vandome, I made some changes, can you check if that works?