Pipeline Components and Applications

  1. Home
  2. Docs
  3. Pipeline Components and Applications
  4. Loaders and storage targets
  5. RDB Loader
  6. Upgrade Guides
  7. 2.0.0 Upgrade Guide

2.0.0 Upgrade Guide

RDB Loader 2.0.0 brings ability to send shredding complete messages from Shredder to SNS topic and splits configs of RDB Loader and RDB Shredder. From now on, Loader and Shredder will not use same config.

Official announcement

Assets

RDB Shredder is published on S3:

  • s3://snowplow-hosted-assets-eu-central-1/4-storage/rdb-shredder/snowplow-rdb-shredder-2.0.0.jar

RDB Loader and RDB Stream Shredder distributed as Docker images, published on DockerHub:

  • snowplow/snowplow-rdb-loader:2.0.0
  • snowplow/snowplow-rdb-stream-shredder:2.0.0

Sending shredding complete messages from Shredder to SNS

Shredding complete message can be sent to SNS topic with following queue configuration:

"queue": { "type": "sns", "topicArn": "arn:aws:sns:eu-central-1:123456789:test-sns-topic", "region": "eu-central-1" }
Code language: JavaScript (javascript)

New separate configs

RDB Loader and RDB Shredder were using the same config HOCON until version 2.0.0. Starting from 2.0.0, they will use separate configs. Reference docs for new configs can be found on the following pages:

RDB Loader configuration

RDB Shredder configuration