This article only applies to time series data

This article is only relevant when working with time series data, not when working with movement data.

You can read more about the differences between the two in this article.

What is the data properties file ?

One of the ways of defining the structure of your .csv files is by uploading a CSV data properties file.

This file is a .csv file where each row specifies what kind of data a column in your data files contains.

  • Row 1 specifies the properties of column 1 in your data .csv files,

  • Row 2 the properties of column 2,

  • …​

Contents of the data properties file

The data properties file is a .csv file where each row specifies what kind of data a column in your data files contains.

Each of the rows specifies the following properties about a column:

  • Name (string): the display name used to provide a human-readable name for the column. This name will be shown in the UI.

  • Type (string with special syntax): the type of the data property. This is used to indicate which column contains the id, timestamp, …​ and to specify whether a property is a numeric property, a category (enum), or text. See the next section for the syntax.

  • Description (string): a human-understandable description of what the property represents. This is for example used in tooltips in the UI.

  • Include for analysis (boolean: true or false): only properties that are included for analysis can be used for filtering and styling. If you set this to false for the field that corresponds to the id of your records, your users will not be able to style by the id, filter by the id, or see an overview of the visible asset ids. This can be useful when privacy is a concern.

  • Aggregation interval (number): the analytics engine maps numeric properties to bins or intervals. For example if the property represents the length of a ship in metres, using a value of 2 for the aggregation interval will tell the analytics engine to report statistics on the vessel length in intervals of 2 metres: [0,2[, [2,4[ , [4,6[, …​ . You won’t be able to distinguish between vessels with a length of 3 and 4 metres because their lengths are mapped onto the same interval [2,4[. This setting is only used for numeric properties. It is also ignored for the mandatory properties (id, location and timestamp).

  • No data value (number, string or empty): some data sets use a special value to indicate that the actual value for a property is unknown or not set.

Example

The following is an example of a data properties file.

For readability, it is displayed as a table and not as a .csv file (but you can download the .csv version here):

ID

id:string

Road segment id

FALSE

0

Speed

double

The speed on the segment

TRUE

0.1

Travel time

int

Travel time in seconds

TRUE

1

Timestamp

time

The timestamp at which the recording was taken

FALSE

0

Owner

enum

The owner of the segment

FALSE

0

Borough

string

The borough containing the segment

FALSE

0

Link name

string

The name of the link

FALSE

0

In the above example, the line

Speed

double

The speed on the segment

TRUE

0.1

indicates that the corresponding column

  • Is named Speed

  • Contains numbers that are stored as doubles

  • The description to use for that column is "The speed on the segment"

  • The column should be included in the analytics

  • The aggregation interval for this numeric property is 0.1

  • There is no specific value that represents that there is no data

Syntax of the type property

The type property definition consists out of 3 parts:

<prefix>:<type>:<additional_info>
  • The prefix:

    • id: use this prefix for the column representing the identifier of the record. There can be only one column defined as the id.

    • None of the other properties take a prefix

  • The type is one of the following: time, string, float, double, short, int, long or enum.

    • time: indicates that this column contains the time stamp of the records. There can only one column defined as the timestamp.

    • string: indicates that this column contains text.

    • float: indicates that this column contains a single precision (32 bit) floating point number.

    • double: indicates that this column contains a double precision (64 bit) floating point number.

    • short:: indicates that this column contains a half precision (16 bit) integer number in the range [-32768, 32767].

    • int: indicates that this column contains a single precision (32 bit) integer number in the range [-2147483648, 2147483647].

    • long: indicates that this column contains a single precision (64 bit) integer number.

    • enum: indicates that this column contains a values that are categories or enumerations. For example different brands of a car, or types of a vessel.

  • Some properties allow specifying additional info:

    • The platform tries to auto-detect the time format for the time property. However, if the format is known, it can be specified by defining the pattern as additional info.

      For example time:yyyy-MM-dd’T’HH:mm:ss.

      The syntax for the patterns is documented here.

You can use the following flowchart to assist you in selecting the correct type:

Figure 1. Flowchart to assist you in selecting the correct type