Enhanced functionality of cross-instrument file transfer API endpoints

[tieneupin]

Previously, we set up endpoints to facilitate the downloading of TIFF files from a specific instrument. This approach worked as a stopgap measure, but was not easily scalable to other instruments and data types.

This PR adds new endpoints to the instrument and backend servers, along with new keys to the MachineConfig Pydantic model, that allow for file downloads to be configured on a per-instrument basis. The old TIFF file download endpoints have been left in for now for backwards compatibility, but should ideally be removed from the repo in a future PR to minimise bloat.

• Changed the upstream_data_directories field into a dictionary, with the data directory mapped to the instrument name. Updated file download endpoints to work with this change
• Introduced the upstream_data_search_strings field, which takes a list of glob search strings for each instrument
• Updated the find_upstream_visits FastAPI endpoint to return a nested dictionary, where the visit names and their full file paths are further grouped by instrument name
• Added FastAPI endpoints to the instrument and backend servers to use the more generic search strings to find and download files

[codecov]

Codecov Report

❌ Patch coverage is 41.78082% with 85 lines in your changes missing coverage. Please review.
✅ Project coverage is 35.26%. Comparing base (528a1fe) to head (8e67e84).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #680      +/-   ##
==========================================
+ Coverage   35.05%   35.26%   +0.21%     
==========================================
  Files          91       91              
  Lines       11161    11225      +64     
  Branches     1486     1487       +1     
==========================================
+ Hits         3912     3959      +47     
- Misses       7122     7138      +16     
- Partials      127      128       +1     

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.