What is a Failed Event?
A Failed Event is simply what we label any event that was not able to be processed through your pipeline.
While an event is being processed by the pipeline it is checked to ensure it meets the specific formatting or configuration expectations; these include checks like: does it match the schema it is associated with, were Enrichments successfully applied and was the payload sent by the tracker acceptable.
All failed events are routed to storage (AWS S3 or GCP cloud storage).
There are multiple points at which an event can fail in the pipeline; collection, validation and enrichment.
The following are the different types of failures that would create a Failed Event:
Collector Payload Format Violation
If something goes wrong when the enrich job tries to deserialize the payload serialized by the collector, a failed event of this type is emitted. For instance this could happen in the case of:
- Malformed HTTP requests
- Invalid query string encoding in URL
- Path not respecting /vendor/version
Collector payload format violation schema can be found here.
We support many 3rd party webhooks that can be connected to the Snowplow pipeline. Whenever the event sent by a webhook doesn’t respect the expected structure and list of fields for this webhook, an adapter failure is emitted.
This could happen for instance if a webhook is updated and stops sending a field that it was sending before.
Adapter failure schema can be found here.
Events sent by a Snowplow tracker are expected to follow our tracker protocol. Whenever this protocol is not correctly respected, a tracker protocol violation bad row is emitted, usually revealing an error on the tracker side.
Tracker protocol violation schema can be found here.
This type of failed event is emitted whenever the size of the enriched event is too big for the message queue. In this case it will be truncated and wrapped in a size violation failed event instead.
Size violation schema can be found here.
The Snowplow pipeline accepts self-describing events and entities. To be processed successfully
- there needs to be a corresponding schema for the specific event or entity
- the event or entity must conform to the structure described in the schema
Whenever an event comes across and is not valid for one of these 2 cases, a schema violation failed event is emitted.
Schema violation schema can be found here.
This type of bad row is emitted whenever something goes wrong in the enrichment process. This could happen for example if the credentials used for the SQL enrichment are wrong, or because of a bad link to the MaxMind database for the IP lookups enrichment.
Enrichment failure schema can be found here.
Other Failure Types
The following are other error types that could be routed into failed events, however they are less common than the ones above.
loader_iglu_error 2-0-0 – when an error happens in a loader caused by an Iglu subsystem e.g. some schema is not available, invalid schema version (breaking change in ADDITION), old Iglu Server (for RDB/Postgres Loaders because we need a special endpoint for all schemas) etc. Payload is fully parsed and validated enriched event JSON
loader_parsing_error 2-0-0 – created in any Loader (via Analytics SDK), if parsing of a canonical TSV event format has failed, e.g. if line has not enough columns (not 131) or event_id is not UUID. Payload is plain string.
loader_recovery_error 1-0-0 – recovery software couldn’t re-insert the row into DB due to a runtime failure or invalid data in a source. This is not a payload for generic “recovery job”, but for loader-specific, such as BigQuery Repeater (the only producer so far). Payload is plain string.
loader_runtime_error 1-0-1 – just very generic runtime error that we didn’t catch, e.g. some NPE or DynamoDB outage for example. Usually happens because there’s a bug in software or configuration. Payload is fully parsed and validated enriched event JSON
relay_failure 1-0-0 – if a relay fails and provides an error it will be captured here.