Pipeline Components and Applications

  1. Home
  2. Docs
  3. Pipeline Components and Applications
  4. Loaders and storage targets
  5. Snowplow Snowflake Loader

Snowplow Snowflake Loader

The Snowflake Loader consists of two independent applications:

  • Snowplow Snowflake Transformer – Spark job responsible for transformning enriched events into Snowflake-compatible format
  • Snowplow Snowflake Loader – CLI application responsible for loading Snowplow-compatible enriched events into Snowflake DB

Both applications communicate through DynamoDB table, called “processing manifest” and used to maintain pipeline state. Both applications use same self-describing JSON configuration file of schema com.snowplowanalytics.snowplow.storage/snowflake_config/jsonschema/1-0-3, which contain both transformer- and loader-specific properties.

The Snowflake loader is publicly available since version 0.3.1.

Articles