<prefix>:<type>:<additional_info>
This article only applies to movement data
This article is only relevant when working with movement data, not when working with time series data. You can read more about the differences between the two in this article. |
One of the ways of defining the structure of your .csv
files is by uploading a data properties file.
This file is a .csv
file where each row specifies what kind of data a column in your data files contains.
Row 1 specifies the properties of column 1 in your data .csv
files,
Row 2 the properties of column 2,
…
The data properties file is a .csv
file where each row specifies what kind of data a column in your data files contains.
Each of the rows specifies the following properties about a column:
Name (string): the display name used to provide a human-readable name for the column. This name will be shown in the UI.
Type (string with special syntax): the type of the data property. This is used to indicate which column contains the id, location, timestamp, … and to specify whether a property is a numeric property, a category (enum), or text. See the next section for the syntax.
Description (string): a human-understandable description of what the property represents. This is for example used in tooltips in the UI.
Include for analysis (boolean: true or false): only properties that are included for analysis can be used for filtering and styling. If you set this to false for the field that corresponds to the id of your records, your users will not be able to style by the id, filter by the id, or see an overview of the visible asset ids. This can be useful when privacy is a concern.
Accuracy value (number): the analytics engine rounds numeric properties down to the nearest multiple of the accuracy value. For example if the property represents the length of a ship in metres, using a value of 0.1 for the accuracy value will tell the analytics engine to work with values with 1 digit after the decimal point. For example, a length value 2.31 will then be rounded down to 2.3. The larger the accuracy value, the more compression the platform can perform and the better the performance. This is especially important for larger data sets. This setting is only used for numeric properties. It is also ignored for the mandatory properties (id, location and timestamp).
No data value (number, string or empty): some data sets use a special value to indicate that the actual value for a property is unknown or not set.
Minimum valid value (number of empty): when specified for a numeric property, all values below this value will be considered as unknown or not set. This setting is only used for numeric properties. It is also ignored for the mandatory properties (id, location and timestamp).
Maximum valid value (number of empty): when specified for a numeric property, all values above this value will be considered as unknown or not set. This setting is only used for numeric properties. It is also ignored for the mandatory properties (id, location and timestamp).
The following is an example of a data properties file.
For readability, it is displayed as a table and not as a .csv
file (but you can download the .csv
version here):
MMSI |
id:long |
unique identifier |
TRUE |
0 |
|||
time |
time:yyyy-MM-dd’T’HH:mm:ss |
time string |
FALSE |
0 |
|||
latitude |
y:float |
y-coordinate |
FALSE |
0 |
|||
longitude |
x:float |
x-coordinate |
FALSE |
0 |
|||
Speed over Ground |
float |
Speed over Ground |
TRUE |
1 |
102.3 |
||
Course over Ground |
float |
Course over Ground |
TRUE |
1 |
360.0 |
||
Heading |
heading:float |
Heading |
TRUE |
1 |
511 |
0 |
360 |
Vessel name |
string |
Name of the vessel |
FALSE |
0 |
|||
IMO |
string |
IMO |
FALSE |
0 |
|||
Callsign |
string |
Callsign |
FALSE |
0 |
|||
Vessel type |
enum |
VesselType |
TRUE |
0 |
|||
Status |
enum |
Status |
TRUE |
0 |
|||
Length |
length:float |
length |
TRUE |
0.1 |
0 |
||
Width |
width:float |
width |
TRUE |
0.1 |
0 |
||
Draft |
float |
Draft |
TRUE |
0.1 |
0 |
||
Cargo |
float |
Cargo |
TRUE |
1 |
In the above example, the line
Heading |
heading:float |
Heading |
TRUE |
1 |
511 |
0 |
360 |
indicates that the corresponding column
Is named Heading
Contains numbers that are stored as floats and that represent a heading
The description to use for that column is also Heading
The column should be included in the analytics
The accuracy value for this numeric property is 1, all heading values are rounded down to the nearest integer
When the column contains the value 511, it should be interpreted as "no data"
Only values in the interval [0,360] are valid values
The type property definition consists out of 3 parts:
<prefix>:<type>:<additional_info>
The prefix:
id:
use this prefix for the column representing the identifier of the record.
There can be only one column defined as the id.
x:
use this prefix for the column representing the WGS84 longitude coordinate in degrees form (e.g., 179.0523).
In case your data uses a different coordinate reference system, you can specify this as additional info. There can be only one column defined as the x coordinate.
y:
use this prefix for the column representing the WGS84 latitude coordinate in degrees form (e.g., 50.0234).
In case your data uses a different coordinate reference system, you can specify this as additional info. There can be only one column defined as the y coordinate.
z:
use this prefix for an elevation expressed as meters above sea level.
It is used for 3D coordinates, for example when tracking air tracks.
There can be only one column defined as the z coordinate.
width:
, length:
: use the prefixes for the columns representing the width and length of the asset.
The visual analytics page uses these properties to scale the icons that represent the current position of the asset.
The platform assumes these properties are expressed in meters.
toFront:
, toBack:
, toLeft:
, toRight:
use the prefixes if the relative position of the sensor recording the location of the asset is known.
Sometimes the location-recording sensor is located off-center. For large assets like container ships, this additional offset can introduce an error of several meters on the location of the asset. When the data exposes this offset, the platform can compensate for it.
The following image illustrates how the platform interprets the different options:
The platform assumes these properties are expressed in meters.
heading:
use this prefix for a heading expressed in degrees clockwise from the north.
None of the other properties take a prefix
The type is one of the following: time
, string
, float
, double
, short
, int
, long
or enum
.
time
: indicates that this column contains the time stamp of the records.
There can only one column defined as the timestamp.
string
: indicates that this column contains text.
float
: indicates that this column contains a single precision (32 bit) floating point number.
double
: indicates that this column contains a double precision (64 bit) floating point number.
short:
: indicates that this column contains a half precision (16 bit) integer number in the range [-32768, 32767].
int
: indicates that this column contains a single precision (32 bit) integer number in the
range [-2147483648, 2147483647].
long
: indicates that this column contains a single precision (64 bit) integer number.
enum
: indicates that this column contains a values that are categories or enumerations.For example
different brands of a car, or types of a vessel.
Some properties allow specifying additional info:
When the x:
and y:
coordinates are expressed in a different coordinate reference system, you specify the EPSG code as additional info.
For example x:float:EPSG:4269
and y:float:EPSG:4269
The platform tries to auto-detect the time format for the time
property.
However, if the format is known, it can be specified by defining the pattern as additional info.
For example time:yyyy-MM-dd’T’HH:mm:ss
.
The syntax for the patterns is documented here.
You can use the following flowchart to assist you in selecting the correct type: