Dot Nester

Transform

Synopsis

Converts nested structures into flattened ones using dot notation for field names.

Schema

dot_nester:
  - field: <ident>
  - format: <string>
  - target_field: <ident>
  - delimiter: <string>
  - depth: <numeric>
  - description: <text>
  - if: <script>
  - ignore_failure: <boolean>
  - ignore_missing: <boolean>
  - on_failure: <processor[]>
  - on_success: <processor[]>
  - tag: <string>

Configuration

Field	Required	Default	Description
field	N	-	Field containing the nested object to flatten. If empty or omitted, flattens the entire document
format	N	-	Output format: ecs, cim, asim, cef, leef, csl. When not specified, maintains original case
target_field	N	field	Field to store the flattened result. If not specified, overwrites the source field
delimiter	N	.	Character(s) used to separate nested field names in the flattened result
depth	N	-1	Maximum nesting depth to flatten. Use -1 for unlimited depth
description	N	-	Explanatory note
if	N	-	Condition to run
ignore_failure	N	false	If true, errors are ignored
ignore_missing	N	false	If true, missing fields are skipped
on_failure	N	-	See Handling Failures
on_success	N	-	See Handling Success
tag	N	-	Identifier

Details

The processor is used for log normalization, schema conversion, and deep object simplification.

note

You can use target_field when you need to preserve the original structure, set an appropriate depth for the nesting level, and select a delimiter based on your naming convention.

Available formats to comply with specific schema conventions:

Default (no format) - Preserves original field name casing, e.g. User.FirstName.LastName, and uses the specified delimiter. Defaults to dot notation.

ASIM - Preserves original field name casing, e.g. User.FirstName.LastName. See Appendix for details.

ECS - Converts all field names to lowercase, e.g. user.firstname.lastname, and uses the specified delimiter. Defaults to dot notation. See Appendix for details.

CIM - Converts all field names to lowercase, e.g. user.firstname.lastname. See Appendix for details.

warning

Only mapped objects are flattened. Non-map values will cause an error unless ignore_failure is set to true.

Examples

Documents

All nested structures in the document...	{ "user": { "name": "John", "address": { "city": "New York", "country": "USA" } }, "metadata": { "version": "1.0" } } dot_nester: - target_field: flattened
are flattened:	{ "flattened": { "user.name": "John", "user.address.city": "New York", "user.address.country": "USA", "metadata.version": "1.0" } }

Fields

Nested objects...	{ "nested": { "field1": "value1", "field2": { "subfield": "value2" } } } dot_nester: - field: nested - target_field: flattened
are flattened with dot notation:	{ "nested": { "field1": "value1", "field2": { "subfield": "value2" } }, "flattened": { "field1": "value1", "field2.subfield": "value2" } }

Delimiter

Using underscore as delimiter...	{ "nested": { "field1": "value1", "field2": { "subfield": "value2" } } } dot_nester: - field: nested - delimiter: "_" - target_field: flattened
separates paths:	{ "flattened": { "field1": "value1", "field2_subfield": "value2" } }

Depth

Limiting the depth...	{ "root": { "level1": { "level2": { "level3": "deep value" }, "sibling": "value" } } } dot_nester: - field: root - depth: 2 - target_field: flat
flattens only at the specified level:	{ "flat": { "level1.sibling": "value", "level1.level2": { "level3": "deep value" } } }

Conditions

Specific criteria limits flattening...	{ "source": "external", "nested": { "field": "value" } } dot_nester: - field: nested - if: "source == 'external'"
only with fields that meet them:	{ "source": "external", "nested": { "field": "value" } }

Formats

Using the ECS format...	{ "User": { "Name": "John", "Details": { "Age": 30 } } } dot_nester: - field: User - format: ecs
converts field names to lowercase:	{ "User": { "name": "John", "details.age": 30 } }