For IO and toy codes to handle Equinor's Open Domain Volve Dataset

To download large data files from https://data-equinor-com.azurewebsites.net/dataset/volve, save download_file.py into the directory in which you want to store your data. Then in a terminal window, run the following command within that directory :

python download_file.py

You can control which file is downloaded by modifying the url entry near the top of file. In addition, you may change the chunk_size input within the function download_file(url, chunk_size = 2**30) to what you deem is necessary. The default is set to download the 2.6 TB file Seismic.ST10010.zip, at a chunk size of 1 GB. It has been verified to work on Python 3.6.

Please look at ST10010_file_list.txt before you decide to download this file and verify this is the data you want. The file unzip'd totals 3.3 TB, which means that you will need at minimum 6 TB to manage this one dataset for both the download and the extraction.

download_file.py can resume a download if the download is cut for whatever reason. Just re-run the above command and it will resume from the appropriate byte position. If you have a problem with the download being interrupted too often, you may wish to use a wrapper file to run the script until the full file is downloaded.

The Volve dataset is open domain and can be accessed via this script on networks outside of Equinor.