1. Home
  2. Docs
  3. Managing data quality
  4. Event Recovery for Snowplow Insights
  5. Configuration
  6. Configuration Examples

Configuration Examples

Minimal example

The most basic configuration that will run for the particular failure type: adapter-failure, in a particular version, and mark all others as failed because of missing configuration mappings would look like this:

{ "schema": "iglu:com.snowplowanalytics.snowplow/recoveries/jsonschema/4-0-0", "data": { "iglu:com.snowplowanalytics.snowplow.badrows/adapter_failures/jsonschema/1-0-0": [ { "name": "pass-through-flow", "conditions": [], "steps": [] } ] } }
Code language: JSON / JSON with Comments (json)

In above scenario we would be resubmitting all the received adapter_failures without any modifications.

Multiple flows example

The next example shows a way of setting up multiple configuration flows for specific failure types. Configurations will be matched top to bottom and the first from the top will be chosen.

{ "schema": "iglu:com.snowplowanalytics.snowplow/recoveries/jsonschema/4-0-0", "data": { "iglu:com.snowplowanalytics.snowplow.badrows/enrichment_failures/jsonschema/1-0-0": [ { "name": "main-flow", "conditions": [ { "op": "Test", "path": "$.payload.raw.vendor", "value": { "regex": "com.snowplow\\.*" } } ], "steps": [ { "op": "Replace", "path": "$.raw.refererUri", "match": "(?U)^.*$", "value": "https://console.snplow.com/" } ] }, { "name": "pass-through", "conditions": [], "steps": [] } ] } }
Code language: JSON / JSON with Comments (json)

In above scenario we would only process the 1-0-0 version of enrichment_failures. We would be modifying those for which vendor in raw payload starts with com.snowplow. For those rows that match that field value, we replace the full refererUri contents with https://console.snplow.com/. All other enrichment_failures will be resubmitted as they were originally (pass-through scenario).

Putting it all together

Below is an advanced example making use of most of the features and performing multiple operations on a very specific subset of failed events.

{ "schema": "iglu:com.snowplowanalytics.snowplow/recoveries/jsonschema/4-0-0", "data": { "iglu:com.snowplowanalytics.snowplow.badrows/enrichment_failures/jsonschema/1-0-0": [ { "name": "main-flow", "conditions": [ { "op": "Test", "path": "$.processor.artifact", "value": { "value": "beam-enrich" } }, { "op": "Test", "path": "$.payload.raw.vendor", "value": { "regex": "com.snowplow\\.*" } }, { "op": "Test", "path": "$.processor.artifact", "value": { "size": { "eq": 11 } } }, { "op": "Test", "path": "$.processor.artifact", "value": { "size": { "gt": 3 } } }, { "op": "Test", "path": "$.processor.artifact", "value": { "size": { "lt": 30 } } } ], "steps": [ { "op": "Replace", "path": "$.raw.refererUri", "match": "(?U)^.*$", "value": "https://console.snplow.com/" }, { "op": "Remove", "path": "$.raw.parameters.aid", "match": "(?U)^.*$" }, { "op": "Replace", "path": "$.raw.headers", "match": "X-Forwarded-Proto:.*", "value": "X-Forwarded-Proto: http" }, { "op": "Replace", "path": "$.raw.parameters.cx.data.[?(@.schema=~iglu:org.w3/PerformanceTiming/jsonschema/1-0-0)].data.loadEventEnd", "match": "(?U)^.*$", "value": "1" }, { "op": "Cast", "path": "$.raw.parameters.cx.data.[?(@.schema=~iglu:org.w3/PerformanceTiming/jsonschema/1-0-0)].data.domComplete", "from": "Numeric", "to": "Boolean" }, { "op": "Replace", "path": "$.raw.parameters.cx.data.[1].data.domComplete", "match": "false", "value": "true" }, { "op": "Cast", "path": "$.raw.parameters.cx.data.[?(@.data.navigationStart=~([0-9]+))].data.domComplete", "from": "Boolean", "to": "Numeric" } ] }, { "name": "impossible-flow", "conditions": [ { "op": "Test", "path": "$.processor.artifact", "value": { "value": "lorem-ipsum" } } ], "steps": [] } ] } }
Code language: JSON / JSON with Comments (json)

If you’d like to learn more about Snowplow Insights you can book a demo with our team, or if you’d prefer, you can try Snowplow technology for yourself quickly and easily.