SEAScope.lib package¶

Submodules¶

SEAScope.lib.plots module¶

This module provides methods for plotting the data extracted by SEAScope.

SEAScope.lib.plots.plot_2d_data(source_name, field_name, field_data, output_path)[source]¶

Plot data extracted by SEAScope using a polygon.

Parameters:

source_name (str) – Name of the source (granule) data has been extracted from
field_name (str) – Name of the extracted field
field_data (numpy.array) – Extracted data (for a single field/channel)
output_path (str) – Path of the file wherein the plot will be saved

SEAScope.lib.plots.plot_transect(dists, transects, output_path)[source]¶

Plot data extracted by SEAScope using a polyline (transect).

Parameters:

dists (:obj:list of :obj:float) – Cumulative distance from origin to each point of the transect, in kilometers
transects (:obj:dict) – Values extracted along the polyline for each (granule, field) pair
output_path (str) – Path of the file wherein the plot will be saved

SEAScope.lib.utils module¶

This module provides methods for creating and reformatting data structures.

exception SEAScope.lib.utils.DynamicPackingFloatIncompatbilityError[source]¶

Bases: Exception

Raised when SEAScope.lib.utils.set_field() is called with both dynamic_packing and as_float set to True`: dynamic packing is only implemented for unsigned bytes

exception SEAScope.lib.utils.FieldShapeError[source]¶

Bases: ValueError

Raised when the data associated with a field is neither 1D nor 2D.

exception SEAScope.lib.utils.IncompatibleFieldError[source]¶

Bases: ValueError

Raised when a field cannot be added to a granule because it is not compatible with already attached fields (shape mismatch).

exception SEAScope.lib.utils.InvalidShapeError[source]¶

Bases: ValueError

Raised when the shape passed to SEAScope.lib.utils.get_lonlat() does not have exactly 2 elements (shape for a 2D matrix).

SEAScope.lib.utils.create_collection(label)[source]¶

Helper method to create the dictionary that represents a collection.

Most fields are initialized with default values that you can/should customize before passing the generated dictionary to other methods.

Parameters:

label (str) – Label describing the collection. This is the text that SEAScope will display in the catalogue for this collection.

Returns:

A tuple which contains two elements:

an integer which is the local identifier, i.e. the identifier within a source, for the collection

a dictionary which contains all the information SEAScope needs to identify and format data associated with the collection variables.

Return type:

tuple(int, dict)

Example

>>> import SEAScope.lib.utils
>>> result = SEAScope.lib.utils.create_collection('A custom collection')
>>> coll_id, coll_dict = result
>>> print(coll_id)
10
>>> print(coll_dict)
{'id': {'granuleLevel': False,
        'sourceId': 1,
        'collectionId': 10,
        'granuleId': 0,
        'variableId': 0},
 'mustBeCurrent': False,
 'xSeamless': False,
 'ySeamless': False,
 'NEWSAligned': False,
 'label': 'A custom collection',
 'tags': {},
 'variables': [],
 'variablesList': {},
 'defaultVariable': 0}

SEAScope.lib.utils.create_granule(collection_id, gcps, start, stop)[source]¶

Helper method to create the dictionary that represents a granule.

Most fields are initialized with default values that you can/should customize before passing the generated dictionary to other methods.

Time coverage is [start, stop[

Parameters:

collection_id (int) – Local (i.e. specifc to the current data source) identifier for the collection of the granule
gcps (list of dict) – List of Ground Control Points, represented as dictionaries with four keys: lon, lat, i and j, that give the spatial coverage of the granule and provide intermediary points to map data on a sphere
start (datetime.datetime) – Begining of the time coverage of the granule, in UTC (included)
stop (datetime.datetime) – End of the time coverage of the granule, in UTC (excluded)

Returns:

A tuple which contains two elements:

an integer which is the local identifier, i.e. the identifier within a source, for the granule

a dictionary which contains all the information SEAScope needs to identify and format data associated with the granule.

Return type:

tuple(int, dict)

SEAScope.lib.utils.create_rendering_config(col_id=0, var_id=0)[source]¶

Helper method to create the dictionary that represents a rendering configuration (for a variable).

Most fields are initialized with default values that you can/should customize before passing the generated dictionary to other methods.

Parameters:

col_id (int) – Local (i.e. specifc to the current data source) identifier for the collection
var_id (int) – Identifier for the variable within the collection

Returns:

A dictionary which contains the rendering parameters to use for the variable which corresponds to the (col_id, var_id) couple.

Return type:

dict

Example

>>> import SEAScope.lib.utils
>>> rcfg = SEAScope.lib.utils.create_rendering_config(533, 5)
>>> print(rcfg)
{'rendered': True,
 'logscale': False,
 'min': 0,
 'max': 1,
 'opacity': 1.0,
 'zindex': 500,
 'color': [50, 12, 87],
 'colormap': 'grayscale',
 'renderMethod': 'RASTER',
 'particlesCount': 1000,
 'particleTTL': 10,
 'streamlineLength': 20,
 'streamlineSpeed': 0.0,
 'filterMode': 'NEAREST',
 'billboardsSize': 32,
 'billboardsDensity': 0,
 'lineThickness': 1,
 'target': {'granuleLevel': False,
            'sourceId': 1,
            'collectionId': 533,
            'granuleId': 0,
            'variableId': 5}}

SEAScope.lib.utils.create_variable(collection, label, fields, units='', dims=2)[source]¶

Helper method to create the dictionary that represents a variable.

Most fields are initialized with default values that you can/should customize before passing the generated dictionary to other methods.

Parameters:

collection (dict) – Dictionary which represents a collection. This dictionary is either handcrafted or generated by the create_collection() method.
label (str) – Label describing the variable. This is the text that will be displayed in the SEAScope catalogue.
units (str) – Units of the geophysical data contained in the variable. Defaults to an empty string.
dims (int) – Number of dimensions of the data matrix associated with the variable. If this parameter is set to 1 the default rendering method for this variable will be TRAJECTORIES, otherwise it will be RASTER.

Returns:

A dictionary which contains all the information SEAScope needs to identify the variable and render the associated data.

Return type:

dict

Example

>>> import SEAScope.lib.utils
>>> result = SEAScope.lib.utils.create_collection('A custom collection')
>>> coll_id, coll_dict = result
>>> var_dict = SEAScope.lib.utils.create_variable(coll_dict, 'my variable',
>>>                                               ['u_field', 'v_field'],
>>>                                               'm.s-1')
>>> print(var_dict)
{'id': 0,
 'label': 'my variable',
 'units': 'm.s-1',
 'fields': ['u_field', 'v_field'],
 'defaultRenderingMethod': 'RASTER',
 'tags': {},
 'collection': {'granuleLevel': False,
                'sourceId': 1,
                'collectionId': 10,
                'granuleId': 0,
                'variableId': 0},
 'rendering': {'rendered': True,
               'logscale': False,
               'min': 0,
               'max': 1,
               'opacity': 1.0,
               'zindex': 500,
               'color': [50, 12, 87],
               'colormap': 'grayscale',
               'renderMethod': 'RASTER',
               'particlesCount': 1000,
               'particleTTL': 10,
               'streamlineLength': 20,
               'streamlineSpeed': 0.0,
               'filterMode': 'NEAREST',
               'target': {'granuleLevel': False,
                          'sourceId': 1,
                          'collectionId': 10,
                          'granuleId': 0,
                          'variableId': 0}}}

SEAScope.lib.utils.geoloc_from_gcps(gcplon, gcplat, gcplin, gcppix, lin, pix)[source]¶

SEAScope.lib.utils.get_lonlat(extraction, shape)[source]¶

Rebuild lon/lat grids at an arbitrary resolution using interpolation, based on the GCPs provided by SEAScope extracted data.

Parameters:

extraction (dict) – Extraction from a single granule, deserialized and reformatted. This is usually one item of dictionary returned by SEAScope.lib.get_extracted_data()
shape (tuple) – Shape of the resulting lon/lat grids

Returns:

A tuple which contains two elements:

a grid of interpolated longitudes for the spatial covergae of the extraction
a grid of interpolated latitudes for the spatial coverage of the extraction

Return type:

tuple(numpy.ndarray, numpy.ndarray)

Raises:

InvalidShapeError – Error raised when the shape passed as parameter does not match a 2D matrix

Example

>>> import SEAScope.lib
>>> import SEAScope.lib.utils
>>> extraction = SEAScope.lib.get_extracted_data('127.0.0.1', 11155)
>>> print([str(x) for x in extraction.keys()])
['/seascope/data/ecmwf/ECMWF_20151201T12Z/ECMWF_20151201T12Z_idf_00.nc']
>>> lons, lats = SEAScope.lib.utils.get_lonlat(extraction['/seascope/data/ecmwf/ECMWF_20151201T12Z/ECMWF_20151201T12Z_idf_00.nc'], (1000,1000))  # noqa:E501

SEAScope.lib.utils.import_granule_from_syntool(data_path, wkt, col_id, start, stop, field)[source]¶

Read data extracted as Numpy objects using Syntool and import them in SEAScope.

Note that Syntool dialog that pops up once the extraction is ready not only contains download links to get the extracted data but also the shape of the extracted area. You must save the shape (displayed in WKT format) in order to use this method as it is used as spatial coverage for the granule.

Parameters:

data_path (string) – Path of the data extracted in Numpy format
wkt (string) – Shape of the extracted data, expressed in WKT format
col_id (int) – Local, i.e. specific to the current data source, identifier for the collection that the granule containing the imported data will be attached to
start (datetime.datetime) – Begining of the time coverage of the granule, in UTC (included)
stop (datetime.datetime) – End of the time coverage of the granule, in UTC (excluded)
field (string) – Name of the field that the extracted data corresponds to

Example

>>> import SEAScope.lib.utils
>>> import datetime
>>> coll_id, coll_dict = SEAScope.lib.utils.create_collection('Syntool data')  # noqa:E501
>>> npy_path = '/tmp/3857_REMSS_MWOI_SST_v05.0-20190527120000-REMSS-L4_GHRSST-SSTfnd-MW_OI-GLOB-v02.0-fv05.0.npy'  # noqa:E501
>>> shape_wkt = 'POLYGON((-42.3633 24.1267,-21.0938 24.1267,-21.0938 41.9677,-42.3633 41.9677,-42.3633 24.1267))'  # noqa:E501
>>> start_dt = datetime.datetime(2019, 5, 27, 0, 0, 0)
>>> stop_dt = datetime.datetime(2019, 5, 28, 0, 0, 0)
>>> granule_id, granule_dict = SEAScope.lib.utils.import_granule_from_syntool(npy_path, shape_wkt, coll_id, start_dt, stop_dt, 'sst')  # noqa:E501
>>> print(granule_id)
1000
>>> print(granule_dict)
{'id': {'granuleLevel': True,
        'sourceId': 1,
        'collectionId': 11,
        'granuleId': 1000,
        'variableId': 0},
 'metadata': {'sourceId': 1,
              'collectionId': 11,
              'granuleId': 1000,
              'dataId': 'user_generated_granule_1000',
              'dataModel': 'ROW_CELL',
              'start': 1558915200000,
              'stop': 1559001600000,
              'uris': [{'uri': 'Python: user_generated_granule_1000',
                        'xArity': 244,
                        'yArity': 242,
                        'resolution': 1000000000,
                        'subsampling_factor': 0,
                        'shape': {'xArity': 2.0,
                                  'yArity': 2.0,
                                  'gcps': [{'lon': -21.0938,
                                            'lat': 24.1267,
                                            'i': 243,
                                            'j': 241},
                                           {'lon': -42.3633,
                                            'lat': 24.1267,
                                            'i': 243,
                                            'j': 0},
                                           {'lon': -21.0938,
                                            'lat': 41.9677,
                                            'i': 0,
                                            'j': 241},
                                           {'lon': -42.3633,
                                            'lat': 41.9677,
                                            'i': 0,
                                            'j': 0}]}}],
              'title': 'user_generated_granule_1000',
              'institution': '',
              'comment': '',
              'file_id': 'Python: user_generated_granule_1000',
              'product_version': '0.0.0',
              'lat_min': 24.1267,
              'lat_max': 41.9677,
              'lon_min': -42.3633,
              'lon_max': -21.0938,
              'creator_email': '',
              'station_id': '',
              'platform': '',
              'sensor': ''},
 'data': {'sst': {'info': {'channels': 1,
                           'xArity': 244,
                           'yArity': 242,
                           'dataType': 'uint8',
                           'offset': [0.0],
                           'scaleFactors': [1.0],
                           'fillValues': [255]},
                           'buffer': [...]}}}

SEAScope.lib.utils.init_ids(collection_offset, granule_offset)[source]¶

Initialize the local, i.e. specific to current data source, identifiers generators for collections and granules.

Parameters:

collection_offset (int) – Initial value for the collection identifiers generator. Please note that the initial value cannot be less than 3 because identifiers “1” and “2” are reserved for the “User polygons” and “User polylines” collections that SEAScope creates automatically (these two collections share the same source as the ones you create using the Python bindings)
granule_offset (int) – Initial value for the granule identifiers generator. Please note that the initial value cannot be less than 1000 so that SEAScope has enough reserved identifiers for the polygons and polylines drawn by the user.

SEAScope.lib.utils.raw2data(raw_result)[source]¶

Reformat raw data received from SEAScope, after it has been deserialized by FlatBuffers, into Numpy masked arrays.

Parameters:: raw_result (dict) – Deserialized version of the data received from SEAScope
Returns:: Data received from SEAScope transformed into Numpy masked arrays
Return type:: dict

SEAScope.lib.utils.set_field(granule, field_name, field_data, dynamic_packing=True, as_float=False)[source]¶

Attach data to a granule

Parameters:

granule (dict) – Dictionary that represents a granule, such as the output of SEAScope.lib.utils.create_granule()
field_name (str) – Name that will identify the data within the granule. Please note that granules that belong to the same collection must all have the same fields
field_data (numpy.ndarray or numpy.ma.MaskedArray) – Data that must be attached to the granule
dynamic_packing (bool, optional) – If False the values contained in field_data will be included “as is”, otherwise (default behavior) the min and max values of field_data are used to project the data in the ubyte domain: min value is mapped to 0, max value to 254 and intermediary values are interpolated. 255 is used to represent masked data.
as_float (bool, optional) – If True data are converted to float32 values, otherwise they are converted to unsigned bytes (default behavior).

Raises:

DynamicPackingFloatIncompatbilityError – Raised when dynamic_packing and as_float are both True because dynamic packing is only implemented for unsigned bytes
FieldShapeError – Raised when field_data is neither 1D nor 2D
IncompatibleFieldError – Raised when a field cannot be added to a granule because it is not compatible with already attached fields (shape mismatch)

Module contents¶

This module provides some helper methods required by the seascope-processor command to process data extracted by SEAScope.

SEAScope.lib.get_dists(mask)[source]¶

Compute the cumulative distance between a list of locations provided by SEAScope when the user makes a transect.

Parameters:: mask (dict) – Data structure (similar to granules) created by SEAscope when the user extracts data along a polyline (transect)
Returns:: The cumulative distances covered at each point while following the transect, in kilometers
Return type:: list of float

SEAScope.lib.get_extracted_data(host='localhost', port=11155)[source]¶

Fetch extracted data from SEAScope and format the result to make it easier to manipulate.

Parameters:

host (str) – IP address of the network interface that the SEAScope application listens to.
port (int) – Port number that the SEAScope application listens to.

Returns:

The data extracted by SEAScope, reformatted to be easier to analyse and manipulate

Return type:

dict

SEAScope.lib.load_pyo(input_path)[source]¶

Load a file from the filesystem and deserialize its content as a Python object.

Parameters:: input_path (str) – Path of the file which contains a serialized Python object
Returns:: The deserialized Python object
Return type:: object

SEAScope.lib.save_pyo(obj, output_path)[source]¶

Serialize a Python object and save the result on the filesystem.

Parameters:

obj (object) – A serializable Python object
output_path (str) – Path of the file that will contain the serialized object

SEAScope.lib package¶

Submodules¶

SEAScope.lib.plots module¶

SEAScope.lib.utils module¶

Module contents¶

Table of Contents

This Page