Home Intro Tutorials Articles Rest API Contact
Sign in Try it

Introduction

This article provides common examples of advanced filtering, including ASSET, ASSET_ROUTE, and RECORD filtering or a combination of these. Also refer to the Filtering introduction and Query language syntax articles.

The examples always first specify what the asset is, what the data and metadata properties are, and then provide the advanced query filter in the Query language syntax. An explanation helps you to better understand the filter.

Show all commercial fleet traffic

Data properties: [tripId (id:string), timestamp (time), longitude (x:double), latitude (y:double), speed (float)]

Metadata properties: [tripId (id:string), vehicleType (enum: "passenger", "commercial"), providerType (enum: "private", "fleet")].

Areas: []

ASSET(
   AND(
      EQ(START_PROP("vehicleType"), "commercial"),
      EQ(START_PROP("providerType"), "fleet"))
   )
)

Since vehicleType and providerType are metadata properties, we can filter using an ASSET filter. This is more efficient than filtering in the RECORD stage and rules out all non-commercial and all non-fleet traffic immediately at the trip (asset) level. The AND function is used to combine the two equality (EQ) functions. Note that we have to use the START_PROP function and not the PROP function which is undefined in the ASSET stage.

Show all locations where vehicles are driving more than 50kmh

Data properties: [tripId (id:string), timestamp (time), longitude (x:double), latitude (y:double), speed (float)]

Metadata properties: [tripId (id:string), vehicleType (enum: "passenger", "commercial"), providerType (enum: "private", "fleet")].

Areas: []

RECORD(
    GT(PROP("speed"), 50)
)

Since speed is a data property it can vary for every record (waypoint, GPS point) of a trip, hence we have to filter on speed in the RECORD stage. Here we can use the PROP function and not the START_PROP as we are referring to the record’s property, and not the property of the start (first) record.

Show all locations where fleet vehicles are driving more than 90kmh

Data properties: [tripId (id:string), timestamp (time), longitude (x:double), latitude (y:double), speed (float)]

Metadata properties: [tripId (id:string), vehicleType (enum: "passenger", "commercial"), providerType (enum: "private", "fleet")].

Areas: []

ASSET(
    EQ(START_PROP("providerType"), "fleet"))
) |
RECORD(
    GT(PROP("speed"), 90)
)

This is a combination of filtering on assets (trips) and records (waypoints). The ASSET filter and RECORD filter are combined using the | symbol.

You have to read this as: First, select all trips from fleets, and then from the resulting trips, select only the waypoints (records) where the speed property is over 90kmh.

Select all locations with vehicles with a high congestion level in New York City

Data properties: [vehicleId (id:string), timestamp (time), longitude (x:double), latitude (y:double), speed (float), congestionLevel (enum: "low", "medium", "high")]

Metadata properties: [vehicleId (id:string), vehicleType (enum: "car", "truck", "bus"), providerType (enum: "government", "private")].

Areas: [New York City, Los Angeles, Chicago, Houston, Phoenix]

ASSET(
    EQ(START_PROP("providerType"), "government"))
) |
RECORD(
    AND(
        EQ(PROP("congestionLevel"), "high"),
        INSIDE(AREA(name="New York City"))
    )
)

This query selects records of vehicles provided by the government with a high congestion level inside New York City.

The query is a combination of an ASSET and a RECORD filter, where providerType can be used in an ASSET part because it is a metadata property and hence remains constant for all records of the same vehicle. The congestionLevel is a data property, and hence could change over time and is used in the RECORD filter part. The constraint to look only at New York City is used also in the RECORD filter and assumes a shape on the map exists with the name New York City.

Show all PM2.5 and PM10 sensor records with a pollution level above 50.

Data properties: [sensorId (id:string), timestamp (time), pollutantLevel (float)]

Metadata properties: [sensorId (id:string), sensorType (enum: "PM2.5", "PM10", "CO", "NO2"), installationDate (string)].

Areas: []

ASSET(
    EQ(START_PROP("sensorType"), "PM2.5", "PM10")
) |
RECORD(
    GT(PROP("pollutantLevel"), 50)
)

This query identifies records of high levels of small partical pollutants.

This is a combination of filtering on assets (sensors) and records (air quality measurements). The ASSET filter and RECORD filter are combined using the | symbol.

Since sensorType is a metadata (i.e., constant) sensor property, we can filter on PM2.5 and PM10 in an ASSET filter. The RECORD filter then selects only the records with a pollutant level greater than (GT) 50. Since the same sensor can have multiple recors with different pollutant levels, this query needs to be done in the RECORD part.

Show all locations where fleet vehicles are driving more than 90kmh during the morning commute between 6am and 10am

Data properties: [tripId (id:string), timestamp (time), longitude (x:double), latitude (y:double), speed (float)]

Metadata properties: [tripId (id:string), vehicleType (enum: "passenger", "commercial"), providerType (enum: "private", "fleet")].

Areas: []

ASSET(
    EQ(START_PROP("providerType"), "fleet"))
) |
RECORD(
    AND(
        GT(PROP("speed"), 90),
        BETWEEN(HOUR(TIME(), use_configured_timezone=true), 6, 10),
        BETWEEN(WEEKDAY(TIME(), use_configured_timezone=true), 1, 5)
    )
)

This is a combination of filtering on assets (trips) and records (waypoints). The ASSET filter and RECORD filter are combined using the | symbol.

You have to read this as: First, select all trips from fleets, and then from the resulting trips, select only the waypoints (records) where the speed property is over 90kmh and where the timestamp is between 6 and 10 in the timezone configured by the user and the weekday between Monday and Friday (working days) in the time-zone configured by the user.

Note that the filtering on weekdays from Monday to Friday is added to comply with the fact that work-related commute traffic only happens during those days.

Show all Cargo traffic at Port of Antwerp

Data properties: [mmsi (id:string), timestamp (time), longitude (x:double), latitude (y:double), sog (float), cog (float), vesselType (enum: 0, 20, 21,…​, 30,…​), status(enum: 0, 1, 2, 3, 4, 5, 6,…​)]

Metadata properties: [].

Areas: [Port of Antwerp, Port of Rotterdam, Port of Hamburg]

ASSET_ROUTE(
    INSIDE(AREA(name="Port of Antwerp"))
) |
RECORD(
    EQ(PROP("vesselType"), 70, 71, 72, 73, 74, 75, 76, 77, 78, 79)
)

This is a combination of filtering on asset routes (part of a voyage) and records (AIS records). The ASSET_ROUTE filter and RECORD filter are combined using the | symbol.

You have to read this as: First, select all parts of the vessel’s journeys that are inside the area with the name Port of Antwerp and from the resulting set of AIS records, select only those that have a vesselType property value set in the 70 range, which for AIS corresponds to the different Cargo ship types. The ASSET_ROUTE filter of course assumes that an area with the name Port of Antwerp exists.

Note that the parts of the selected vessel journeys outside of Port of Antwerp are not selected and hence are trimmed off the result. Note also that filtering on vesselType is done in the RECORD filter stage, as this property, which is typical in AIS data, is a data property and not a metadata property.

Show all Tanker traffic through Port of Rotterdam

Data properties: [mmsi (id:string), timestamp (time), longitude (x:double), latitude (y:double), sog (float), cog (float), vesselType (enum: 0, 20, 21,…​, 30,…​), status(enum: 0, 1, 2, 3, 4, 5, 6,…​)]

Metadata properties: [].

Areas: [Port of Antwerp, Port of Rotterdam, Port of Hamburg]

ASSET_ROUTE(
    ANY(),
    INSIDE(AREA(name="Port of Rotterdam")),
    ANY()
) |
RECORD(
    EQ(PROP("vesselType"), 80, 81, 82, 83, 84, 85, 86, 87, 88, 89)
)

This is a combination of filtering on asset routes (part of a voyage) and records (AIS records). The ASSET_ROUTE filter and RECORD filter are combined using the | symbol.

You have to read this as: First, select all records of all vessels passing through Port of Rotterdam, and from the resulting set of AIS records, select only those that have a vesselType property value set in the 80 range, which for AIS corresponds to the different Tanker ship types.

The ASSET_ROUTE filter of course assumes that an area with the name Port of Rotterdam exists. Note that the parts of the selected vessel journeys outside of Port of Rotterdam are also selected thanks to the ANY() legs that come before the INSIDE statement. So basically this filter selects all records of all tankers that at some point in time pass through the Port of Rotterdam area. Note also that filtering on vesselType is done in the RECORD filter stage, as this property, which is typical in AIS data, is a data property and not a metadata property.

Show all Tanker and Cargo traffic from Port of Antwerp to Port of Rotterdam

Data properties: [mmsi (id:string), timestamp (time), longitude (x:double), latitude (y:double), sog (float), cog (float), vesselType (enum: 0, 20, 21,…​, 30,…​), status(enum: 0, 1, 2, 3, 4, 5, 6,…​)]

Metadata properties: [].

Areas: [Port of Antwerp, Port of Rotterdam, Port of Hamburg]

ASSET_ROUTE(
    INSIDE(AREA(name="Port of Antwerp")),
    ANY(),
    INSIDE(AREA(name="Port of Rotterdam"))
) |
RECORD(
    OR(
        EQ(PROP("vesselType"), 70, 71, 72, 73, 74, 75, 76, 77, 78, 79),
        EQ(PROP("vesselType"), 80, 81, 82, 83, 84, 85, 86, 87, 88, 89)
    )
)

This is a combination of filtering on asset routes (part of a voyage) and records (AIS records). The ASSET_ROUTE filter and RECORD filter are combined using the | symbol.

You have to read this as: First, from all vessels having records first in the Port of Antwerp area and then, later, in the Port of Rotterdam area, only select the records that are inside the two port areas and the records where the vessel is in-between the two areas, select only those that have a vesselType property value set in the 70 or 80 ranges, which for AIS corresponds to the different Cargo and Tanker ship types.

The ASSET_ROUTE filter of course assumes that an areas with the name Port of Antwerp and Port of Rotterdam exist. Note that the parts of the selected vessel journeys before entering Port of Antwerp and after leaving Port of Rotterdam are not selected, as there are no ANY() legs before and after the first and last INSIDE statement.

Note that the in-between ANY() leg is necessary, because the two areas do not touch and typically there are AIS records in-between the areas. Note also that filtering on vesselType is done in the RECORD filter stage, as this property, which is typical in AIS data, is a data property and not a metadata property.

Show all Tanker and Cargo traffic from Port of Antwerp to Port of Rotterdam and vice versa

Data properties: [mmsi (id:string), timestamp (time), longitude (x:double), latitude (y:double), sog (float), cog (float), vesselType (enum: 0, 20, 21,…​, 30,…​), status(enum: 0, 1, 2, 3, 4, 5, 6,…​)]

Metadata properties: [].

Areas: [Port of Antwerp, Port of Rotterdam, Port of Hamburg]

ASSET_ROUTE(
    SWITCH_LEG(
        LINEAR_LEG(
            INSIDE(AREA(name="Port of Antwerp")),
            ANY(),
            INSIDE(AREA(name="Port of Rotterdam"))
        ),
        LINEAR_LEG(
            INSIDE(AREA(name="Port of Antwerp")),
            ANY(),
            INSIDE(AREA(name="Port of Rotterdam"))
        )
    )
) |
RECORD(
    OR(
        EQ(PROP("vesselType"), 70, 71, 72, 73, 74, 75, 76, 77, 78, 79),
        EQ(PROP("vesselType"), 80, 81, 82, 83, 84, 85, 86, 87, 88, 89)
    )
)

This is a combination of filtering on asset routes (part of a voyage) and records (AIS records). The ASSET_ROUTE filter and RECORD filter are combined using the | symbol.

The ASSET_ROUTE filter uses a SWITCH_LEG to select voyages in either of two directions. Each of which is represented by the LINEAR_LEG.

You have to read this as: First, from all vessels having records first in the Port of Antwerp area and then, later, in the Port of Rotterdam area or the other way around, only select the records that are inside the two port areas and the records where the vessel is in-between the two areas, select only those that have a vesselType property value set in the 70 or 80 ranges, which for AIS corresponds to the different Cargo and Tanker ship types.

The ASSET_ROUTE filter of course assumes that an areas with the name Port of Antwerp and Port of Rotterdam exist. Note that the parts of the selected vessel journeys before entering Port of Antwerp and after leaving Port of Rotterdam (and vice versa) are not selected, as there are no ANY() legs before and after the first and last INSIDE statement.

Note that the in-between ANY() statements are necessary, because the two areas do not touch and typically there are AIS records in-between the areas. Note also that filtering on vesselType is done in the RECORD filter stage, as this property, which is typical in AIS data, is a data property and not a metadata property.

Show all heavy traffic coming from the A1 driving through Lüdenscheid

Data properties: [vehicleId (id:string), longitude (x:double), latitude (y:double), speed (float), heading (float), timestamp (time)]

Metadata properties: [vehicleId (id:string), vehicleWeightClass (enum: "light", "medium", "heavy), providerType (enum: "consumer", "fleet"), tripDistance (float)].

Areas: [A1, A45, A6, Lüdenscheid, Köln, Dortmund, Aachen, Frankfurt]

ASSET(
    EQ(START_PROP("vehicleWeightClass"), "heavy"))
) |
ASSET_ROUTE(
    ANY(),
    INSIDE(AREA(name="A1")),
    ANY(),
    INSIDE(AREA(name="Lüdenscheid")),
    ANY(),
    AND(
        NOT(INSIDE(AREA(name="Lüdenscheid"))),
        END()
    )
)

This is a combination of filtering on assets (vehicles) and on the route the assets take. The ASSET filter and ASSET_ROUTE filter are combined using the | symbol.

You have to read this as: First, select all vehicles that have a weight class heavy, and then from the resulting vehicles, select only the waypoints (records) of the vehicle driving through the user-defined area A1, then continues until it drives through the area with name Lüdenscheid, and then continues until the end point is outside Lüdenscheid. Vehicles that follow routes that do not match this sequence, are not selected.

The first ANY() in the ASSET_ROUTE ensures we also select the GPS records for the selected vehicles before entering the A1 area. The END() statement evaluates to true for the last record of an asset, in this case a vehicle. Note also the use of the NOT statement, which inverts the inside statement.

Show all heavy traffic coming from the A1 and ending in Lüdenscheid

Data properties: [vehicleId (id:string), longitude (x:double), latitude (y:double), speed (float), heading (float), timestamp (time)]

Metadata properties: [vehicleId (id:string), vehicleWeightClass (enum: "light", "medium", "heavy), providerType (enum: "consumer", "fleet"), tripDistance (float)].

Areas: [A1, A45, A6, Lüdenscheid, Köln, Dortmund, Aachen, Frankfurt]

ASSET(
    EQ(START_PROP("vehicleWeightClass"), "heavy"))
) |
ASSET_ROUTE(
    ANY(),
    INSIDE(AREA(name="A1")),
    ANY(),
    AND(
        INSIDE(AREA(name="Lüdenscheid")),
        END()
    )
)

This is a combination of filtering on assets (vehicles) and on the route the assets take. The ASSET filter and ASSET_ROUTE filter are combined using the | symbol.

You have to read this as: First, select all vehicles that have a weight class heavy, and then from the resulting vehicles, select only the waypoints (records) of the vehicle driving through the user-defined area A1, then continues until it drives inside the area with name Lüdenscheid where it also ends.

The first ANY() in the ASSET_ROUTE ensures we also select the GPS records for the selected vehicles before entering the A1 area. The END() statement evaluates to true for the last record of an asset, in this case a vehicle. Note that vehicles that follow this route, but do not end in the area with name Lüdenscheid are not selected.

Show all heavy traffic coming from the A1 and ending in Lüdenscheid

Data properties: [tripId (id:string), longitude (x:double), latitude (y:double), speed (float), heading (float), timestamp (time)]

Metadata properties: [tripId (id:string), vehicleWeightClass (enum: "light", "medium", "heavy"), providerType (enum: "consumer", "fleet"), tripDistance (float)].

Areas: [A1, A45, A6, Lüdenscheid, Köln, Dortmund, Aachen, Frankfurt]

ASSET(
    EQ(START_PROP("vehicleWeightClass"), "heavy"))
) |
ASSET_ROUTE(
    ANY(),
    INSIDE(AREA(name="A1")),
    ANY(),
    AND(
        INSIDE(AREA(name="Lüdenscheid")),
        END()
    )
)

This is a combination of filtering on assets (vehicles) and on the route the assets take. The ASSET filter and ASSET_ROUTE filter are combined using the | symbol.

You have to read this as: First, select all vehicles that have a weight class heavy, and then from the resulting vehicles, select only the waypoints (records) of the vehicle driving through the user-defined area A1, then continues until it drives inside the area with name Lüdenscheid where it also ends.

The first ANY() in the ASSET_ROUTE ensures we also select the GPS records for the selected vehicles before entering the A1 area. The END() statement evaluates to true for the last record of an asset, in this case a vehicle. Note that vehicles that follow this route, but do not end in the area with name Lüdenscheid are not selected.

Show all trips leaving Antwerp and driving to Brussel that take more than 2 hours

Data properties: [tripId (id:string), longitude (x:double), latitude (y:double), speed (float), heading (float), timestamp (time)]

Metadata properties: [].

Areas: [Gent, Antwerpen, Brussel, Hasselt, Leuven, Namur, Liège, Brugge]

ASSET(
    GT(DURATION(), 7200)
) |
ASSET_ROUTE(
    AND(
        INSIDE(AREA(name="Antwerpen")),
        START()
    ),
    ANY(),
    AND(
        INSIDE(AREA(name="Brussel")),
        END()
    )
)

This is a combination of filtering on assets and their routes. The ASSET filter and ASSET_ROUTE filter are combined using the | symbol.

You have to read this as: First, select all trips that take more than 2 hours. Then from those trips only take the ones that start in Antwerpen and end in Brussels.

The DURATION() call, when used in an ASSET filter that is positioned before an ASSET_ROUTE filter, returns the difference in seconds between the last record of the asset and the first. Since DURATION() returns this time in seconds, we have to provide the 2 hours constraint also in seconds, where 2h = 2*60*60s = 7200 seconds.

The ASSET_ROUTE uses three legs, the first one selects the first record if inside the Antwerpen area. The second part (ANY()) represents the part of the route in-between Antwerpen and Brussel. The third part requires the end point of the trip to be inside the Brussel area.

Show all medium and heavy traffic driving through Antwerpen and then driving through Brussel that take less than 1 hour to get from Antwerpen to Brussel

Data properties: [tripId (id:string), timestamp (time), latitude (y:double), longitude (x:double)]

Metadata properties: [tripId (id:string), vehicleWeightClass (enum: "light", "medium", "heavy")].

Areas: [Gent, Antwerpen, Brussel, Hasselt, Leuven, Namur, Liège, Brugge]

ASSET(
    EQ(START_PROP("vehicleWeightClass"), "medium", "heavy")
) |
ASSET_ROUTE(
    INSIDE(AREA(name="Antwerpen")),
    ANY(),
    INSIDE(AREA(name="Brussel"))
) |
ASSET(
    LT(DURATION(), 3600)
)

This is a combination of filtering on assets and their routes. The ASSET filter, ASSET_ROUTE` filter and second ASSET filter are combined using the | symbol.

You have to read this as: First, select all trips for medium and heavy weight class, and from those trips all trips that drive through Antwerpen and then through Brussel, but only the records of the trips inside Antwerpen, in-between Antwerpen and Brussel, and inside Brussel. Then select from those routes, only the ones that take less than one hour (3600 seconds) to make it from Antwerpen to Brussel.

The DURATION() call, when used in an ASSET filter that is positioned after an ASSET_ROUTE filter, returns the difference in seconds between the last record of the asset’s route and the first. Since DURATION() returns this time in seconds, we have to provide the 1 hour constraint also in seconds, where 1h = 1*60*60s = 3600 seconds.

The ASSET_ROUTE uses three legs, the first one selects the records inside the Antwerpen area. The second part (ANY()) represents the part of the route in-between Antwerpen and Brussel. The third part selects the records to be inside the Brussel area. Trips that do not follow this sequence or that follow the sequence but take more than 1 hour are not selected.

Show all Tanker traffic departing Shangai and then sailing along one of two routes, the first being the Suez Canal and the second along Cape Agulhas, before ending in Antwerpen.

Data properties: [mmsi (id:long), timestamp (time), longitude (x:float), latitude (y:float), status(enum: At anchor, Moored, Underway using engine,…​), vesselType (enum: Unknown, Cargo, Tanker, Fishing,…​)]

Metadata properties: [].

Areas: [Beijing, Shangai, Rotterdam, Antwerpen, Suez Canal, Cape Agulhas, Houston]

ASSET_ROUTE(
    AND(
        START(),
        INSIDE(AREA(name="Shangai"))
    ),
    ANY(),
    SWITCH_LEG(
        INSIDE(AREA(name="Suez Canal")),
        INSIDE(AREA(name="Cape Agulhas"))
    ),
    ANY(),
    AND(
        END(),
        INSIDE(AREA(name="Antwerpen"))
    )
) |
RECORD(
    EQ(PROP("vesselType"), "Tanker",
)

This is a combination of filtering on asset routes (part of a voyage) and records (AIS records). The ASSET_ROUTE filter and RECORD filter are combined using the | symbol.

The ASSET_ROUTE filter multiple INSIDE filters and the START() and END() statements to ensure the route starts and ends in Shangai and Antwerpen respectively. The two options in-between are represented by the SWITCH_LEG statement.

You have to read this as: First, from all vessels having their start record in the Shangai and then sailing through one of two options, being through the Suez Canal or along Cape Agulhas, and later endering in Antwerpen, select only those that have a vesselType property value set to "Tanker".

The ASSET_ROUTE filter of course assumes that areas with the name Shangai and Antwerpen exist on the map, as well as the areas with the names Suez Canal and Cape Agulhas. The use of the START() and END() statements are really necessary to select only the vessels that start/end in the designated areas and not also the vessels that sail through.

Note that the in-between ANY() statements are necessary as well, because the different areas do not touch and typically there are AIS records in-between the areas. Note also that filtering on vesselType is done in the RECORD filter stage, as this property, which is typical in AIS data, is a data property and not a metadata property. Only if we would be certain that it remains constant for a vessel, we could also use it in an ASSET stage, but then by using the START_PROP statement.