How to download EODATA products using s3cmd
Verifying connection
At first, please verify your connection with EO DATA bucket by using list command.
s3cmd ls s3://EODATA
You should obtain this output:
DIR s3://EODATA/C3S/
DIR s3://EODATA/CAMS/
DIR s3://EODATA/CEMS/
DIR s3://EODATA/CLMS/
DIR s3://EODATA/CMEMS/
DIR s3://EODATA/Envisat-ASAR/
DIR s3://EODATA/Envisat/
DIR s3://EODATA/Jason-3/
DIR s3://EODATA/Landsat-5/
DIR s3://EODATA/Landsat-7/
DIR s3://EODATA/Landsat-8/
DIR s3://EODATA/SMOS/
DIR s3://EODATA/Sentinel-1-COG/
DIR s3://EODATA/Sentinel-1-RTC/
DIR s3://EODATA/Sentinel-1/
DIR s3://EODATA/Sentinel-2/
DIR s3://EODATA/Sentinel-3/
DIR s3://EODATA/Sentinel-5P/
DIR s3://EODATA/Sentinel-6/
DIR s3://EODATA/auxdata/
If you have encountered any problems, please contact us.
Downloading EOData product
For downloading we advise you to use s3cmd sync command (for the first time it downloads a whole product):
s3cmd sync s3://EODATA[/PREFIX]/ LOCAL_DIR/
Example for downloading a Sentinel-1 product:
s3cmd sync s3://EODATA/Sentinel-1/SAR/SLC/2016/05/14/S1A_IW_SLC__1SDV_20160514T030935_20160514T031002_011249_011058_2013.SAFE/ S1A_IW_SLC__1SDV_20160514T030935_20160514T031002_011249_011058_2013.SAFE/
Please mind to append the trailing / to both source and target directories.
Please mind that if you get the path of the product from the Creodias Finder, you will have to alter the original path received from Finder.
You have to replace /eodata with s3://EODATA.
Please see it at presented example:
a) original path:
/eodata/Sentinel-1/SAR/SLC/2018/10/01/S1B_IW_SLC__1SDV_20181001T030840_20181001T030907_012953_017EC9_BA79.SAFE
b) altered path for s3cmd:
s3://EODATA/Sentinel-1/SAR/SLC/2018/10/01/S1B_IW_SLC__1SDV_20181001T030840_20181001T030907_012953_017EC9_BA79.SAFE
Alternative solution
Entire product is being structured in a whole directory, not a single file. Therefore, you have to use –recursive parameter in order to avoid any “skipping” of files.
Example for downloading a Sentinel-1 product:
s3cmd get --recursive s3://EODATA/Sentinel-1/SAR/SLC/2018/10/01/S1B_IW_SLC__1SDV_20181001T030840_20181001T030907_012953_017EC9_BA79.SAFE/ S1B_IW_SLC__1SDV_20181001T030840_20181001T030907_012953_017EC9_BA79.SAFE
The above command will download S1B_IW_SLC__1SDV_20181001T030840_20181001T030907_012953_017EC9_BA79.SAFE product to your local folder “S1B_IW_SLC__1SDV_20181001T030840_20181001T030907_012953_017EC9_BA79.SAFE” which shall be previously created.
Troubleshooting
If nothing happens when executing the commands, or you could not download a product properly, please issue -d switch to see a debug output for further diagnosis:
s3cmd -d get --recursive s3://EODATA/Sentinel-1/SAR/SLC/2016/05/14/S1A_IW_SLC__1SDV_20160514T030935_20160514T031002_011249_011058_2013.SAFE/
If attempts to delete downloaded products raise “Permission denied” kind of errors, edit permissions on the product folder with command:
chmod -R +w S1A_IW_SLC__1SDV_20160514T030935_20160514T031002_011249_011058_2013.SAFE
