Optimised 'bootstrap' API endpoints

[tieneupin]

Applied various improvements to the bootstrap API endpoints:

• Switched to using StreamingResponse when downloading files
• Switched from using requests.get() to reusing a requests.Session() object instead to stop creating new connections unless necessary
• Fixed how base URLs are resolved on index pages (a trailing "/" is needed in the URL path)
• Simplified URL parsing and rewriting as a result of resolving base URLs correctly
• Added debug logs to record what URLs are being accessed.
• Added logic to construct functioning URL dynamically to endpoints that have to return a working URL for the client to use.

[codecov]

Codecov Report

Attention: Patch coverage is 21.42857% with 77 lines in your changes missing coverage. Please review.

Project coverage is 28.44%. Comparing base (f4df956) to head (58c1ffe).
Report is 5 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #549      +/-   ##
==========================================
- Coverage   28.96%   28.44%   -0.53%     
==========================================
  Files          80       80              
  Lines       10495    11109     +614     
  Branches     1394     1609     +215     
==========================================
+ Hits         3040     3160     +120     
- Misses       7354     7836     +482     
- Partials      101      113      +12     

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.