Processors
Processors in Imply Lumi perform the work in pipelines to transform events. A processor can perform actions such as transforming and mapping attributes or deleting attributes. Depending on its type, the input for the processor may be the event message, incoming event metadata, or a static value. The processor output is a user attribute on the event.
To learn about creating pipelines, see Transform events with pipelines.
This topic describes the types of processors available in Imply Lumi.
Processor types
The following table summarizes the processor types and their inputs and outputs:
| Processor | Input | Configuration | Result |
|---|---|---|---|
| Attribute mapper | Incoming event metadata | Boolean option to override the attribute when it already exists | New user attribute or updated user attribute |
| Attribute remover | Not applicable | Comma-separated list of incoming event metadata | Absence of the specified attributes |
| Regex parser | Incoming event metadata or event message | Regular expression with one or more capturing groups | One or more user attributes, where the number of output attributes matches the number of capturing groups |
| Value mapper | Static value | Boolean option to override the attribute when it already exists | New user attribute or updated user attribute |
Attribute mapper
The attribute mapper maps the value of one attribute to the value of another attribute.
If the incoming metadata contains a field with the same name as the output attribute, you can override its value or leave it unchanged. Otherwise, the processor adds the attribute to the event.
To override the existing metadata, select the toggle to Override value when output attribute exists. If you don't select the toggle, the existing attribute is unchanged and no value is mapped. The override applies even if the input value is an empty string or one or more whitespace characters.
An exception to the override function is when the input attribute value is null. The processor doesn't override when the input value is null.
Example
- Input
- Event metadata:
status: 401
- Configuration
- Input attribute:
status - Output attribute:
http_status
- Output
- User attribute:
http_status: 401
Attribute remover
The attribute remover removes one or more attributes.
Use this processor to drop unneeded fields to reduce storage size and improve query performance. You can also use the attribute remover to drop personally identifiable information, whether to remove it completely or to remove the source metadata after redaction.
Example
- Input
- Event metadata:
userid: wilma
- Configuration
- Attributes to remove:
userid
- Output
- User attribute: none
Regex parser
The regular expression (regex) processor parses an input value using a regular expression.
The regex parser can parse a string from incoming event metadata or the event message. By default, the regex parser parses the event message. Leave the input field empty to parse the event message, otherwise specify an incoming metadata field.
The number of capturing groups in the regular expression determines the number of output attributes.
If a capturing group matches more than one result, the processor only uses the first result.
For example, for the regex pattern (abc), the test string abc abc would only return the first abc.
If the incoming metadata contains a field with the same name as any of the output attributes, the processor overrides the existing metadata when it matches the regex pattern. This behavior applies even if the match is an empty string or whitespace character.
Example
- Input
- Event message:
Deployment successful. System 1 status: [ok] System 2 status: [alert]
- Configuration
- Input: (blank for event message)
- Regular expression:
status: [(\w*)] - Output attributes:
status
- Output
- User attribute:
status: ok
Value mapper
The value mapper maps a static value to an attribute.
If the incoming metadata contains a field with the same name as the output attribute, you can override its value or leave it unchanged. Otherwise, the processor adds the attribute to the event.
To override the existing metadata, select the toggle to Override value when output attribute exists. If you don't select the toggle, the existing attribute is unchanged and no value is mapped. The override applies even if the input value is an empty string or one or more whitespace characters.
An exception to the override function is when the input attribute value is null. The processor doesn't override when the input value is null.
Example
- Input
- Event metadata:
host: 23.192.228.84
- Configuration
- Static value:
example.com
- Output
- User attribute:
host: example.com
Limitations
Imply Lumi doesn't currently support extractions on time fields.
Learn more
See the following topics for more information:
- Transform events with pipelines for an overview of pipelines and processors.
- How to transform events with pipelines for a tutorial on pipelines.