add large file retrieval example to README

[jjkoehorst]

Since we do not have direct access to the iRods system I was wondering what is the most optimal way of retrieving data.

I found out that: shutil.copyfileobj(fsrc=gzip.open(obj.open('r+')),fdst=gzip.open(fileName, 'wb')) works but is very inefficient.

In other tickets I noticed: obj.manager._download(obj=obj, local_path=fileName) but this results in a SYS_FILE_DESC_OUT_OF_RANGE. Any clue on what is the most efficient way of retrieving the files from iRods and store it locally?

[trel]

Hi @jjkoehorst,

This will get a data_object and write it to disk...

  obj = session.data_objects.get('/tempZone/home/rods/foo')
  open('outfile.txt','w').write(obj.open().read().decode())

You'll probably want to loop through the contents with read(bytes) for a large file.

Does this answer your question?

[jjkoehorst]

Yes I wasn't sure if this would be the most efficient approach on downloading files

[arezaii]

note that for non-text file types, like the tif file I was downloading, the above method produces an
encoding error and required the following modification to write as a binary file:
open('outfile.txt','wb').write(obj.open().read())
Also, I would support adding the simple example above to the README ;)