DataPrime Expression Language (DPXL) reference
The DataPrime Expression Language, or DPXL, is an expression language based on DataPrime expression syntax.You can use it to define rich expression-based filters, such as when setting up streaming.
DPXL expressions are a subset of DataPrime expressions, such as those used in the filter operator.
DPXL expressions are versioned to maintain predictability and stability. With versioning DPXL can be enhanced over time without changing the semantics of existing expressions.
Each DPXL expression starts with a version identifier <vX>, currently <v1>. This is followed by the actual boolean expression, including literals, logical and comparison constructs, keypath access, and functions.
<v1> <boolean-expression>
The <v> prefix is automatically included when using the UI and does not need to be specified. However, a DPXL expression used in an API must start with the <v1> prefix.
# A filter that returns true if my_text field has the value 'example'
<v1> $d.my_text == 'example'
# A filter that returns true if the event's timestamp is before the beginning of the year 2024
<v1> $m.timestamp < @'2024-01-01T00:00:00'
# A filter that returns true if the application name starts with 'dev-'
<v1> $l.applicationName.startsWith('dev-')
# A filter that returns true if the field region_id is us-east-1 or us-east-2
<v1> region_id:string.in('us-east-1', 'us-west-2')
Data types
DPXL supports different data types.
| Data type | Example |
|---|---|
| string | 'us-east-1’’dev-’ |
| number | 23-12.32 |
| bool | truefalse |
| timestamp | @(’2023-01-01T00:00Z’)@’now’ |
| regular expression | /H.*o$//^prod-.*/ |
| severity | VERBOSEDEBUGINFOWARNINGERRORCRITICAL |
In addition, there’s a null literal, which can be used with all other types.
Operators
DPXL supports multiple operators.
| Operator | Meaning | Example | Example description |
|---|---|---|---|
&& |
Logical AND | country == ‘us’ && region == ‘us-south’ |
Will return true if both the country is us and the region is us-south. |
|| |
Logical OR | age > 40 || country == ‘us’ |
Will return true if either the age is above 40, or the country is us. |
! |
Logical NOT | !region.contains(’us-’) |
Will return true if the region does not contain us- |
Ordering
You can control the evaluation order inside an expression using parenthesis. For example:
region.startsWith('us-') &&
(country == 'us' && (age > 40 || age < 10)) || (country == 'il' && age > 25)
Comparison operators
DPXL supports multiple comparison operators.
| Operator | Meaning | Examples |
|---|---|---|
> |
Greater than | duration > 40.5$m.timestamp > @(’2023-01-01T00:00:00’) |
>= |
Greater than or equals | duration >= 40.5 |
< |
Less than | age < 20 |
<= |
Less than or equals | age <= 20lastName <= ‘Smith’ |
== |
Equals | region == 'us-south' |
!= |
Not equal to | first_name != 'joe' |
Keypath
Keypaths are divided into three different parts, each with a separate prefix:
$m- Metadata
$l- Labels, such as
applicationNameorsubsystemName $d- User data (default prefix)
$m – Metadata keypaths
| Keypath | Data type | Description |
|---|---|---|
$m.timestamp |
timestamp | Contains the timestamp of the event |
$m.severity |
severity | Contains the severity of the event |
$l – Label keypaths
| Keypath | Data type |
|---|---|
$l.applicationname |
string |
$l.subsystemname |
string |
$d – User data keypaths
Any user keypath can be accessed using $d.<keypath> including nested keypaths.
$d is the default prefix. Any keypath that does not contain a prefix will be considered a user data field.
Functions
Functions provide additional capabilities within DPXL expressions.
| Function | Description | Example |
|---|---|---|
<s>.startsWith(<substr>):bool |
Checks if a string <s> starts with the specified substring <substr> |
region.startsWith('us-') |
<s>.endsWith(<substr>):bool |
Checks if a string <s> ends with the specified substring <substr> |
firstName.endsWith(’Jo’) |
<s>.contains(<substr>):bool |
Checks if a string <s> contains the specified substring <substr> |
stream.contains(’err’) |
<s>.matches(<regex>):bool |
Checks if a string <s> matches the specified pattern provided by the <regex> |
hostname.matches(/prod-.*/) |
<value>.in(<value1>,<value2>,...) |
Checks if value is one of the provided values | value1-valueN region.in(’us-east’,’us-south’) |
Inferring data types
DPXL attempts to infer the expected datatype of keypaths. For example, when processing age > 50, it will infer that age is expected to be a number. In cases where DPXL cannot infer the data type for a keypath, it
will require the necessary information about the type. For example:
'123':number
region1:string == region2
my_key:number > my_other_key
Examples
The following are DPXL examples that you can use as a basis for your own DPXL expressions.
# Allow access only to logs where the application name is "production"
<v1> $l.applicationname == 'production'
# Allow access only to logs in which app name starts with dev, or the field "region_id" in the data is us-east
<v1> $l.applicationname.startsWith('dev-') && region_id == 'us-east'
# Allow access only to logs in which the field "country" is not one of the listed below.
<v1> !$d.country:string.in('us','il','gr')
# Allow access only to logs where the pod name matches the regex provided
<v1> kubernetes.pod_name.matches(/^kafka-[0-9]+/)
# Allow access only to logs that don't have a DEBUG severity
<v1> $m.severity != DEBUG
# Allow access only to logs in which some query duration is very large
<v1> query_duration_seconds > 100
# Allow access only to logs up to the beginning of the year 2024
<v1> $m.timestamp < @'2024-01-01T00:00:00'
# Disallow access to all logs entirely
<v1> false
Inference limitations
The in function cannot automatically infer the expected type of a keypath. To use the in function you need to indicate the type. For example:
<v1> !$d.country:string.in('us','il','gr')