Gitlab-PreProd

Skip to content

ZipUrl actually download data in memory

Bug description and behaviour

ZipUrl actually downloads data in memory

Steps to reproduce

from  eossr.utils import ZipUrl
url = 'https://zenodo.org/record/5599198/files/Micro-CT%20data.zip'
ZipUrl(url)

is downloading the 6GB of data in this zip file.

Expected behaviour?

from  eossr.utils import ZipUrl
url = 'https://zenodo.org/record/5599198/files/Micro-CT%20data.zip'
ZipUrl(url)

should be instantaneous and should not download the data

Relevant logs and/or screenshots

Possible fixes

The lib https://github.com/gtsystem/python-remotezip/ is a potential fix but does not work out of the box with Zenodo zip files

RemoteIOError: 429 Client Error: Too Many Requests for url: https://zenodo.org/record/3232985/files/CamDavidsonPilon/lifelines-v0.21.2.zip
Edited by Vuillaume