Open Source Components and Applications

  1. Home
  2. Docs
  3. Open Source Components and Applications
  4. Snowplow Snowflake Loader

Snowplow Snowflake Loader

The Snowflake Loader consists of two independent applications:

  • Snowplow Snowflake Transformer – Spark job responsible for transformning enriched events into Snowflake-compatible format
  • Snowplow Snowflake Loader – CLI application responsible for loading Snowplow-compatible enriched events into Snowflake DB

Both applications communicate through DynamoDB table, called “processing manifest” and used to maintain pipeline state. Both applications use same self-describing JSON configuration file of schema com.snowplowanalytics.snowplow.storage/snowflake_config/jsonschema/1-0-2, which contain both transformer- and loader-specific properties.

The Snowflake loader is publicly available since version 0.3.1.

Articles