Data transfer times are long -- speed up network and use Globus?

[rly]

Describe the use case that is addressed by this feature.

This cell of the full example notebook:

cache.get_session_table().head()

took about 4 minutes to run.

This cell, which downloads the 2.4 GB NWB file:

session_id = 756029989 # for example
session = cache.get_session_data(session_id)

took about 15 minutes to download on a 500 mbps wired connection. For comparison, I downloaded a file with the same size from Google Drive in just over 1 minute, which suggests to me that my connection is not the bottleneck.

I know the Allen Institute might be facing high server loads with the latest data release, but is there any way to speed up data downloads on the Allen Institute's side?

Describe the solution you'd like

Faster networks for data transfers.

For sharing large files, I also suggest using Globus, which would support fast, reliable data transfers between the Allen Institute and the many research institutions that use Globus. Globus can also seamlessly resume downloads after an interruption -- very useful for big data files that take a long time to download.

Describe alternatives you've considered

Additional context

Do you want to work on this issue?

Sorry if this is not the right place to raise this issue.

[bendichter]

I'm facing the same problem and I would also like to see a globus option for download