Meteosat Second Generation Data Retriever

May 2018

The Meteosat Second Generation Data Retriever (MSG Data Retriever) is a tool for converting raw Meteosat Second Generation (MSG) files into a known raster-GIS or raster image file format.

General usage

Fill in all fields from left to right and from top to bottom according to your query, and press "Execute" to retrieve the matching data and perform the specified conversion.
Depending on the output megabyte size, execution can take a long time.
While executing, the function of the "Exit" button is changed to "Abort", with which the operation can be aborted.

Explanation of the user-interface fields

Satellite
Choose the satellite for retrieving data. Currently MSG (located at 0 degrees), RSS (located at 9.5 degrees) and IODC (located at 41.5 degrees) are supported.
Note that you must pre-configure the Data Sources in the File menu accordingly. It is not possible to retrieve data for a satellite for which there is no Data Source configured.

Date/time range (UTC)
Choose the dates of interest.
Choosing the "From" date and time, resets the "To" date and time to the same value. This is done so that time series resulting in large files are not selected by accident.
Explicitly set the "To" date and time to another value if you need time series.
"Repeat interval": choose the number of minutes between successive images.

Series
Check "Multiple Channels in one File" for making stacks of channels. The channels come as multiple bands in the output file (not all output formats support multiple bands). This can also be used to make RGB composites for picture formats (like JPEG), but then, exactly three channels must be selected. When unchecked (default), each channel is put into a separate file.
Check "Multiple Times in one File" for making time series in one file. The images come as multiple bands in the output file (not all output formats support multiple bands). This can be combined with the previous option. However, usually a GIS can't handle the combined file very well, so only make the combination if you know what you are doing. When this option is unchecked, for each timestamp (thus after every repeat interval) a separate file is made.

Conversion
Original DN values: choose this option to get the original 10 bits values (DN = Digital Number). The result is 16 bits images, of which only 10 bits are used.
8 bits values: choose this option to get 8 bits values (the most significant 8 bits are kept, i.e. the DN is divided by 4). The result is 8 bits images. This is a.o. useful for output formats that only support 8 bits values in a band, e.g. JPEG, GIF.
Radiometric (mW/m2/sr(cm-1)-1): choose this option to get radiometrically corrected images in this unit (this results in 32 bits floating point numbers)
Radiometric (W/m2/sr/um): choose this option to get radiometrically corrected images in this unit (this results in 32 bits floating point numbers)
Temperature (Kelvin): choose this option to get the temperature (this results in 32 bits floating point numbers)

Channels
Check the channels that you need. The channels can be re-ordered by moving them with the mouse (drag/drop) or by clicking the "Sort" button. Choosing a specific order can be useful when the option "Multiple Channels in one File" is checked, as the channels are put in the file in this order. To make a RGB JPEG for example (instead of a BGR), re-order the channels. Note that for "Multiple Channels in one File" the HRV band can't be combined with the others as its resolution is higher, while all bands in a file must have the same resolution.

Use bounding box
Check this option to get only the selected area in the map into the output file. When unchecked, the entire image is put into the output file, regardless of the indication in the Lat/Lon boxes.
Use the "select" button to drag and select an area on the map.
Use the "pan" button to pan the map (when zoomed in).
Use the "zoom-in" button to click or drag and zoom in on the map.
Use the "zoom-out" button to click or drag and zoom out on the map.
When the output projection is MSG (the original), the bounding box is exactly the box that is selected, i.e. the Lat/Lon bounds are only there as an indication.
When the output projection is LatLon, the bounding box matches the Lat/Lon values, i.e. the bounding box is no longer an accurate representation of what will be the output.
When the output projection is UTM, the output is the most logical bounding box. However, neither the selected box nor the Lat/Lon values are an accurate representation of the output.

Output
Format: choose one of the available output formats. The list of formats is shorter when projection is LatLon or UTM, as for some formats re-projection is not supported.
Folder: choose the destination folder for the output file(s).
File prefix: File names are generated automatically. Optionally type a prefix here in order to distinguish between files.
Projection: choose the output projection: MSG is the original projection, whereby the original geometry and pixel size of the images is preserved. In LatLon and UTM a nearest neighbour resampling algorithm is used.

Other fields
Files: the number of output files is indicated here. Check this number in order to determine if the date / time range, the selected channels, and the series options are filled in correctly.
MB: the estimated size of the raw output (i.e. rows x cols x images x bytes-per-pixel). This number is inaccurate for compressed formats (like JPEG). It is also inaccurate for small-size images (100x100 pixels or less) for formats that have high overhead per image (like ERDAS Imagine).
Ignore errors: when an error is encountered, e.g. files are missing, this options makes sure the errors are ignored. This is useful for batch jobs that will take several hours (otherwise, for every error that is encountered, a popup will appear and the job will be paused). If there is no output files at all, please uncheck this option in order to see if there is an error message explaining this. This option can be checked / unchecked while executing. It is therefore recommended to have this option unchecked initially, and check it after a few minutes for jobs that must run unattended.
Show console: When this option is checked, an ms-dos box is popped up, showing the progress of the commands gdal_translate.exe and gdalwarp.exe that do the real work. This option can be checked / unchecked while executing.

Menu
Open: opens a project file, i.e. a file containing settings that were previously saved.
Save: saves a project file. All settings (date / time range, channels selected, bounding box, series, conversion and output) are saved in this file.
Data Sources: indicate one or more folders with raw MSG satellite data. The data sources determine the date / time range that can be selected.
Select background map: choose another .shp file to be used as the background map for the bounding box window. Only .shp files in the MSG projection at longitude 0 degrees are currently supported. This option is useful when a background map with higher resolution is needed, or one showing the coverage of the HRV channel, or one without the Lat/Lon grid. In order to be able to use a .shp file as a background map, there must be a corresponding .shx file. A map where all polygons are closed, is displayed as blue (sea) and yellow (land). Otherwise it is shown as green (the segments).

Known issues and limitations
Limitations of use of the MSG Data Retriever
Limitations of use of Meteosat data

ITC receives the Meteosat data under an Educational/Research license from KNMI (Royal Dutch Meteorological Institute) which is the national license agency for EUMETSAT. This data can only be used by ITC staff and students for education or research. This excludes the use of the data for any commercial purposes. When data is used, a copyright reference has to be made to EUMETSAT.
Questions related to the data distrubution in ITC can be send to Ben Maathuis (b.h.p.maathuis@utwente.nl)
 
Credits

Bas Retsios, Jan Hendrikse, Ambro Gieske, Boudewijn van Leeuwen, Ben Maathuis
 
Support

MSG Data Retriever comes without any formal way of support. However, for questions directly related to the software you can contact Bas Retsios (v.retsios@utwente.nl), and for any other information contact Ben Maathuis (b.h.p.maathuis@utwente.nl)