FAQ

You can filter datasets by “Variable type”, by “Area”, “Resolution” etc.. A free text search is also available.

Click on “View more” to browse detail information on the dataset selected.

2.  Do I need to register before downloading the data?

Yes, you need to be registered to download data. To request a registration, click on “Create an account”.

3.  How to download datastore data?

You first need to register (see FAQ 2).

Then, from the catalogue, you click on the “Download” button below the dataset you are interested in. You are then invited to fill in your login and password.

Then, the dataset download page is displayed. A dataset is a set of one or more variable, that have 3 or 4 dimensions (longitude, latitude, [depth], time).

This page allows you to fill the extraction criteria:

• 1 and 2 you select your geographical area (longitude and latitude)
• 3 your temporal coverage
• 4 you can click on “Download” to directly download a netCDF file

• 4 you can click on “Script” if you want to automate your download, using a python script (see FAQ 4 and FAQ 5)

4.  Can I automate download of data?

YES , command lines can be particularly useful:
•    if you want to download data regularly and automatically (without having to connect manually to the web portal each time)
•    if you want to download a large amount of data because each downloading command is volume limited

To achieve this you can build command lines using a python script. It can be launched under different environments:  Windows (possibly through Cygwin), Linux and MacOS. See also FAQ 5.

5.  How to write and run the script to download data?

The CLS datastore web portal allows you generating a template command line using the python script for downloading data you are interested in. This command line integrates the extraction parameters. In order to first create your command line, you have to navigate as if you wanted to download the data through the web portal (see FAQ 3).

Then, click on “Script” (after selecting your criteria in the web page).

The popup displayed explain how to proceed: you need to have python installed, you need to download the motu-client package and extract the archive. Then, copy and paste the command line written in the popup, and adapt to your local environment the path of your motu-client package, and finally replace username and password by yours (usually proxy parameter are useless). After the run of the command line, you should receive a file data.nc corresponding to your request.

Of course, in order to download the data every day for example, you just need to modify the parameter “date” in your command.

Do like that for each dataset you want to download.

6.  Is a download limited in terms of volume?

Volume limitations for a single request have been set up on download mechanisms in order to guaranty good performances. The maximum amount of data you can download is 1 Gigabyte for a single request. If your single request is beyond this size, an error message will be immediately displayed: “The size of the data to download – … Megabytes- exceeds the maximum allowed – 1024 Megabytes”. You can also encounter a “proxy error 502”. This error message appears when a request lasts too long.

In order to overcome the limited volume associated to a single request, you need to use multiple requests (i.e.: a loop within a script command line) hence splitting the data volume into smaller pieces at a time. For example in your script command line, you can change the temporal extraction windows.

7.  How to download a large amount of data?

One download request results into one file, so downloading a large amount of data (i.e. more than 1Gb) with a single request is not allowed. Moreover, it is not recommended to manipulate files larger than 1Gb, for usability purpose.

Downloading a very large amount of data, especially for downloading large time coverage is possible through the download service: The simplest way is to perform a loop on dates. You need to split your download requests into several download requests.

For example, below you can find a loop coded in a simple Unix/Linux shell script.

# Definition of the dates

DATES=(2016-06-01 2016-06-02 2016-06-03 2016-06-04 2016-06-05 2016-06-06)

# Loop on the dates

for (( i = 0 ; i < ${#DATES[@]} ; i++ ))

do

#Download request (should be adapted)

python /opt/python/motu-client.py –quiet -u [username] -p [userpassword] -m http://motu.cls.fr/cls-gateway-servlet/Motu -s catsat -d dataset-catsat-nrt-global-infrared-sst-hr-daily -x 90 -X 144 -y -20 -Y 25 -t ${DATES[$i]} -T ${DATES[$i]} -o ./ -f test.nc

done

The array DATES shall be adapted to each situation (time coverage of the dataset and your own need). Of course, depending on the way you want to code the loop (python, perl, java…), you will have to adapt the code. The line in blue is a command line to download one time step of data.