Meteosat Second Generation Data Retriever
May 2018
The Meteosat Second Generation Data Retriever (MSG Data Retriever)
is a tool for converting raw Meteosat Second Generation (MSG) files
into a known raster-GIS or raster image file format.
General usage
Fill in all fields from left to right and from top to bottom according
to your query, and press "Execute" to retrieve the matching data and
perform the specified conversion.
Depending on the output megabyte size, execution can take a long time.
While executing, the function of the "Exit" button is changed to "Abort", with which the operation can be aborted.
Explanation of the user-interface fields
Satellite
Choose the satellite for retrieving data. Currently MSG (located at 0 degrees), RSS (located at 9.5 degrees) and IODC (located at 41.5 degrees) are supported.
Note that you must pre-configure the Data Sources in the File menu accordingly. It is not possible to retrieve data for a satellite for which there is no Data Source configured.
Date/time range (UTC)
Choose the dates of interest.
Choosing the "From" date and time, resets the "To" date and time to the
same value. This is done so that time series resulting in large files
are not selected by accident.
Explicitly set the "To" date and time to another value if you need time series.
"Repeat interval": choose the number of minutes between successive images.
Series
Check "Multiple Channels in one File" for making stacks of channels.
The channels come as multiple bands in the output file (not all output
formats support multiple bands). This can also be used to make RGB
composites for picture formats (like JPEG), but then, exactly three
channels must be selected. When unchecked (default), each channel is
put into a separate file.
Check "Multiple Times in one File" for making time series in one file.
The images come as multiple bands in the output file (not all output
formats support multiple bands). This can be combined with the previous
option. However, usually a GIS can't handle the combined file very
well, so only make the combination if you know what you are doing. When
this option is unchecked, for each timestamp (thus after every repeat
interval) a separate file is made.
Conversion
Original DN values: choose this option to get the original 10 bits
values (DN = Digital Number). The result is 16 bits images, of which
only 10 bits are used.
8 bits values: choose this option to get 8 bits values (the most
significant 8 bits are kept, i.e. the DN is divided by 4). The result
is 8 bits images. This is a.o. useful for output formats that only
support 8 bits values in a band, e.g. JPEG, GIF.
Radiometric (mW/m2/sr(cm-1)-1): choose this option to get
radiometrically corrected images in this unit (this results in 32 bits
floating point numbers)
Radiometric (W/m2/sr/um): choose this option to get radiometrically
corrected images in this unit (this results in 32 bits floating point
numbers)
Temperature (Kelvin): choose this option to get the temperature (this results in 32 bits floating point numbers)
Channels
Check the channels that you need. The channels can be re-ordered by
moving them with the mouse (drag/drop) or by clicking the "Sort"
button. Choosing a specific order can be useful when the option
"Multiple Channels in one File" is checked, as the channels are put in
the file in this order. To make a RGB JPEG for example (instead of a
BGR), re-order the channels. Note that for "Multiple Channels in one
File" the HRV band can't be combined with the others as its resolution
is higher, while all bands in a file must have the same resolution.
Use bounding box
Check this option to get only the selected area in the map into the
output file. When unchecked, the entire image is put into the output
file, regardless of the indication in the Lat/Lon boxes.
Use the "select" button to drag and select an area on the map.
Use the "pan" button to pan the map (when zoomed in).
Use the "zoom-in" button to click or drag and zoom in on the map.
Use the "zoom-out" button to click or drag and zoom out on the map.
When the output projection is MSG (the original), the bounding box is
exactly the box that is selected, i.e. the Lat/Lon bounds are only
there as an indication.
When the output projection is LatLon, the bounding box matches the
Lat/Lon values, i.e. the bounding box is no longer an accurate
representation of what will be the output.
When the output projection is UTM, the output is the most logical
bounding box. However, neither the selected box nor the Lat/Lon values
are an accurate representation of the output.
Output
Format: choose one of the available output formats. The list of formats
is shorter when projection is LatLon or UTM, as for some formats
re-projection is not supported.
Folder: choose the destination folder for the output file(s).
File prefix: File names are generated automatically. Optionally type a prefix here in order to distinguish between files.
Projection: choose the output projection: MSG is the original
projection, whereby the original geometry and pixel size of the images
is preserved. In LatLon and UTM a nearest neighbour resampling
algorithm is used.
Other fields
Files: the number of output files is indicated here. Check this number
in order to determine if the date / time range, the selected channels,
and the series options are filled in correctly.
MB: the estimated size of the raw output (i.e. rows x cols x images x
bytes-per-pixel). This number is inaccurate for compressed formats
(like JPEG). It is also inaccurate for small-size images (100x100
pixels or less) for formats that have high overhead per image (like
ERDAS Imagine).
Ignore errors: when an error is encountered, e.g. files are missing,
this options makes sure the errors are ignored. This is useful for
batch jobs that will take several hours (otherwise, for every error
that is encountered, a popup will appear and the job will be paused).
If there is no output files at all, please uncheck this option in order
to see if there is an error message explaining this. This option can be
checked / unchecked while executing. It is therefore recommended to
have this option unchecked initially, and check it after a few minutes
for jobs that must run unattended.
Show console: When this option is checked, an ms-dos box is popped up,
showing the progress of the commands gdal_translate.exe and
gdalwarp.exe that do the real work. This option can be checked /
unchecked while executing.
Menu
Open: opens a project file, i.e. a file containing settings that were previously saved.
Save: saves a project file. All settings (date / time range, channels
selected, bounding box, series, conversion and output) are saved in
this file.
Data Sources: indicate one or more folders with raw MSG satellite data.
The data sources determine the date / time range that can be selected.
Select background map: choose another .shp file to be used as the
background map for the bounding box window. Only .shp files in the MSG
projection at longitude 0 degrees are currently supported. This option is useful when a
background map with higher resolution is needed, or one showing the
coverage of the HRV channel, or one without the Lat/Lon grid. In order
to be able to use a .shp file as a background map, there must be a
corresponding .shx file. A map where all polygons are closed, is
displayed as blue (sea) and yellow (land). Otherwise it is shown as
green (the segments).
Known issues and limitations
- When "Multiple Timestamps in one File" is checked, and a date /
time range over two or more data sources is involved, the output file
is split at the end of each data source (separate data sources can't be
combined in the same file).
- As the GDAL output drivers are still under development, not all
output formats may work properly for all combinations of options. I.e.
some formats may not support multiple bands in a file, or others may
not support 32 bits float output, or resampling. You may receive an
error message, or the application may crash. The solution is to either
change the selected options to something that the output format can
handle, or choose another output format.
- If the application crashes due to a problem in one of the output
drivers, you may need to kill it with the task manager. In this case,
also kill a process called "gdal_translate.exe" or "gdalwarp.exe" in
the task manager's list of processes.
- The projection of the original MSG images is called "Geostationary
Satellite View". At this moment, this projection is only known in the
ILWIS GIS.
- As a workaround for some problems with the GDAL library, all data is
assigned the WGS84 ellipsoid. However, in order to get the highest
possible accurracy, a custom ellipsoid should be assigned to the
original raw data, with parameters a = 6378169 and 1/f = 295.488065897.
If you need this, do not re-project images with the MSG Data Retriever,
but load them in a GIS that knows the Geostationary Satellite View
projection and assign this custom ellipsoid before re-projecting.
- For re-projection, nearest neighbour resampling is used. The pixel size is determined by
GDAL, as follows: A resolution is computed with the intent that the
length of the distance from the top left corner of the output imagery
to the bottom right corner would represent the same number of pixels as
in the source image. Note that if the image is somewhat rotated, the
diagonal taken isnt of the whole output bounding rectangle, but instead
of the locations where the top/left and bottom/right corners transform.
The output pixel size is always square. This is intended to
approximately preserve the resolution of the input data in the output
file.
Limitations of use of the MSG Data Retriever
- MSG Data Retriever is protected under Dutch Copyright Law and
International Treaty. You may use the software to process SERVIRI L1.5
data. You may copy the software to any system within ITC. You may not
copy the software to any source outside ITC.
- MSG Data Retriever consists of a graphical user interface,
GDAL_translate, GDALwarp and a wavelet decompression library.
GDAL_translate and GDALwarp are open source software
(http://www.gdal.org/). This means that ITC has no
direct control over the output generated by these programs.
- The wavelet decompression library is copyrighted software. ITC is
licensed to use this software for the decompression of SEVIRI data
only. This means that it is not allowed to distribute the software
outside ITC. The user of the MSG Data Retriever shall not make any attempt
to use the wavelet decompression library outside or apart from the MSG
Data Retriever.
- You acknowledge that the MSG Data Retriever may not satisfy all
your requirements or be free from defects. No warranty conditions apply
nor is ITC liable to you for actual damages for any cause whatsoever.
Limitations of use of Meteosat data
ITC receives the Meteosat data under an Educational/Research license
from KNMI (Royal Dutch Meteorological Institute) which is the national
license agency for EUMETSAT. This data can only be used by ITC staff
and students for education or research. This excludes the use of the
data for any commercial purposes. When data is used, a copyright
reference has to be made to EUMETSAT.
Questions related to the data distrubution in ITC can be send to Ben Maathuis (b.h.p.maathuis@utwente.nl)
Credits
Bas Retsios, Jan Hendrikse, Ambro Gieske, Boudewijn van Leeuwen, Ben Maathuis
Support
MSG Data Retriever comes without any formal way of support. However,
for questions directly related to the software you can contact Bas
Retsios (v.retsios@utwente.nl), and for any other information contact
Ben Maathuis (b.h.p.maathuis@utwente.nl)