Make _find_input_files public

[znicholls]

Description

Makes _find_input_files public. This is a super useful function and would be great if it was part of the 'public API' or at least well-documented and explicitly tested so that it can be used with confidence in other parts of the package (my particular interest is for #1572).

Link to documentation:

---

Before you get started

• ☝ Create an issue to discuss what you are going to do

Checklist

It is the responsibility of the author to make sure the pull request is ready to review. The icons indicate whether the item will be subject to the 🛠 Technical or 🧪 Scientific review.

• 🧪 The new functionality is relevant and scientifically sound
• 🛠 This pull request has a descriptive title and labels
• 🛠 Code is written according to the code quality guidelines
• 🧪 and 🛠 Documentation is available
• 🛠 Unit tests have been added
• 🛠 Changes are backward compatible
• 🛠 Any changed dependencies have been added or removed correctly
• 🛠 The list of authors is up to date
• 🛠 All checks below this pull request were successful

---

To help with the number pull requests:

• 🙏 We kindly ask you to review two other open pull requests in this repository

[znicholls]

@sloosvel and @valeriupredoi it looks like you are the right ones to ask about this from my very basic git blame investigation. I just wanted to ask for your thoughts on a) the docstrings (suggestions of how to describe the variable argument would be super helpful) and b) how to test this, should I just basically copy tests/integration/test_data_finder.py::test_get_input_filelist, adapt for this slightly lower level function and then add other tests for my specific use cases?

[sloosvel]

I'm very sorry but I don't think I will be able to take a look at this right now, as I will be focusing on the PR that are open for the release!

[znicholls]

Totally fine, this should definitely wait until after the release

[bouweandela]

@znicholls I'm also trying to add a nicer version of this to the public API, but it's still work in progress: #1609.

[znicholls]

Super nice. Should I close this then? Fine by me but would be good to have a rough expected timeline for #1609

…

On Tue, 7 Jun 2022 at 3:26 pm, Bouwe Andela ***@***.***> wrote: @znicholls <https://github.com/znicholls> I'm also trying to add a nicer version of this to the public API, but it's still work in progress: #1609 <#1609>. — Reply to this email directly, view it on GitHub <#1618 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AFUH5G2FXXJHGKFHPFZ2R3DVN5EWXANCNFSM5YASEKRQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[valeriupredoi]

@znicholls - cheers for this! Sorry for the radio silence, I just got back from holidays. I like the idea of making this public and importable/usable in the API - but I think we should have this integrated in a broader functionality e.g. a dry-run that tells the user how/where files are found, and if the files that are found are sufficient for a run etc, also optionally to have CMOR checks done on the data found, and logging issues found from there. By all means, this is a good start for that but I'd like to hear what @bouweandela thinks wrt #1609 before I come back here with more than thoughts&prayers 😁

[znicholls]

@valeriupredoi all good, let's wait and see where #1609 lands before coming back to this. The dry run idea sounds awesome!

[znicholls]

(Feel free to close if you don't like having draft PRs floating around)

[zklaus]

I for one love having draft PRs floating around. More if they have an (aspirational) milestone assigned.

[valeriupredoi]

absolutely, me too! Draft is a word I like too (eg draft beer 🍺 )

[bouweandela]

would be good to have a rough expected timeline for #1609

I will try really hard to get that into the next release. It is kind of working now, but there are still about 100 broken tests that need careful inspection and if necessary an update. And then it needs to be reviewed of course..

[valeriupredoi]

can help! Test fixing and/or reviewing, lemme know 🍺

[bouweandela]

#1609 grew into a very big change, so I'm breaking it up into smaller bits. Here is a pull request that adds the data finder functions to the public API: #1835

[bouweandela]

This is now available as the function esmvalcore.local.find_files.