Set up the stream collector

Available on Terraform Registry

A Terraform module is available which deploys the stream collector on a AWS EC2 without the need for this manual setup.

Run the collector

The stream collector is on docker hub with several different flavours. Pull the image that matches the sink you are using:

docker pull snowplow/scala-stream-collector-kinesis:2.3.1
docker pull snowplow/scala-stream-collector-pubsub:2.3.1
docker pull snowplow/scala-stream-collector-kafka:2.3.1
docker pull snowplow/scala-stream-collector-nsq:2.3.1
docker pull snowplow/scala-stream-collector-stdout:2.3.1

The application is configured by passing a hocon file on the command line:

docker run --rm \
  -v $PWD/config.hocon:/snowplow/config.hocon \
  -p 8080:8080 \
  snowplow/scala-stream-collector-${flavour}:2.3.1 --config /snowplow/config.hocon
Code language: JavaScript (javascript)

Alternatively, you can download and run a jar file from the github release.

java -jar scala-stream-collector-kinesis-2.3.1.jar --config /path/to/config.hocon

Health check

Pinging the collector on the /health path should return a 200 OK response:

curl http://localhost:8080/health
Code language: JavaScript (javascript)