Open Source Components and Applications

  1. Home
  2. Docs
  3. Open Source Components and Applications
  4. Snowplow Snowflake Loader
  5. Upgrade guide

Upgrade guide

Upgrading from 0.1.0

  • To update Snowflake infrastructure another setup must be launched – it’ll create a new file format snowplow_enriched_json
  • Both setup and load subcommands now accept required --snowflake-region option

Upgrading from 0.2.0

  • Biggest change is that both transformer and loader now accept common configuration file (--configuration) and Iglu Resolver config (--resolver) instead of specific CLI options
  • Static credentials are considered deprecated now, use AWS Role (or Snowflake stage)
  • If you don’t want to store credentials in config and already have existing stage – you’ll need to add credentials to stage object.
  • Snowflake transformer now depends on Spark 2.2.0 and therefore requires amiVersion in cluster.json to be set to 5.9.0

Upgrading from 0.3.0

  • Due to several columns in being widened to support pseudonymization and MaxMind changes, the table schema on Snowflake will need to be migrated – in order to automatically update the relevant column definitions, run java -jar snowplow-snowflake-loader-0.4.0.jar migrate --loader-version 0.4.0

Upgrading from 0.4.0

  • Due to bad row support, URL of S3 bucket which bad rows will be written need to specified in the config with badOutputUrl config field. Also, Snowflake config version need to be bumped to 1-0-2 in the self describing json.