To update Snowflake infrastructure another setup must be launched – it’ll create a new file format snowplow_enriched_json
Both setup and load subcommands now accept required --snowflake-region option
Upgrading from 0.2.0
Biggest change is that both transformer and loader now accept common configuration file (--configuration) and Iglu Resolver config (--resolver) instead of specific CLI options
Static credentials are considered deprecated now, use AWS Role (or Snowflake stage)
If you don’t want to store credentials in config and already have existing stage – you’ll need to add credentials to stage object.
Snowflake transformer now depends on Spark 2.2.0 and therefore requires amiVersion in cluster.json to be set to 5.9.0
Upgrading from 0.3.0
Due to several columns in atomic.events being widened to support
pseudonymization and MaxMind changes, the table schema on Snowflake will
need to be migrated – in order to automatically update the relevant
column definitions, run java -jar snowplow-snowflake-loader-0.4.0.jar migrate --loader-version 0.4.0
Upgrading from 0.4.0
Due to bad row support, URL of S3 bucket which bad rows will be written need to specified in the config with badOutputUrl config field. Also, Snowflake config version need to be bumped to 1-0-2 in the self describing json.