The Snowflake Loader consists of two independent applications:
- Snowplow Snowflake Transformer – Spark job responsible for transformning enriched events into Snowflake-compatible format
- Snowplow Snowflake Loader – CLI application responsible for loading Snowplow-compatible enriched events into Snowflake DB
Both applications communicate through DynamoDB table, called “processing manifest” and used to maintain pipeline state. Both applications use same self-describing JSON configuration file of schema com.snowplowanalytics.snowplow.storage/snowflake_config/jsonschema/1-0-3, which contain both transformer- and loader-specific properties.
The Snowflake loader is publicly available since version 0.3.1.