Configuration options

A template configuration file is available from Github: config.hocon.sample.

Stream configuration

You will need to update the config file with your good and bad Pub/Sub topic names:

• collector.streams.good: The name of the good input stream of the tool which you choose as a sink. This is stream where the events which have successfully been collected will be stored in.
• collector.streams.bad: The name of the bad input stream of the tool which you choose as a sink. This is stream where the events that are too big will be stored in.
• collector.streams.useIpAddressAsPartitionKey: Whether to use the incoming event’s ip as the partition key for the good stream/topic

HTTP settings

Also verify the settings of the HTTP service:

• collector.interface
• collector.port

Buffer settings

You will also need to set appropriate limits for:

• collector.streams.buffer.byteLimit
• collector.streams.buffer.recordLimit
• collector.streams.buffer.timeLimit

Sinks

The collector.streams.sink.enabled setting determines which of the supported sinks to write raw events to:

• "kinesis" for writing Thrift-serialized records and error rows to a Kinesis stream
• "googlepubsub" for writing Thrift-serialized records and error rows to a Google Cloud Pubsub topic
• "stdout" for writing Base64-encoded Thrift-serialized records and error rows to stdout and stderr respectively
• "kafka" for writing Thrift-serialized records and error rows to a Kafka topic
• "nsq" for writing Thrift-serialized records and error rows to NSQ topic

If you switch to "stdout", we recommend setting ‘akka.loglevel = OFF’ and ‘akka.loggers = []’ to prevent Akka debug information from polluting your event stream on stdout.

You should fill the rest of the collector.streams.sink section according to your selection as a sink.

To use stdout as a sink comment everything in the collector.streams.sink but collector.streams.sink.enabled which should be set to stdout.

Setting the domain name

Set the cookie name using the collector.cookie.name setting.

Setting the domain name in collector.cookie.domain can be useful if you want to make the cookie accessible to other applications on your domain. In our example above, for example, we’ve setup the collector on collector.snplow.com. If we do not set a domain name, the cookie will default to this domain. However, if we set it to .snplow.com, that cookie will be accessible to other applications running on *.snplow.com.

Please, refer to RFC 6265 for the domain matching rules.

Setting the cookie duration

The cookie expiration duration is set in collector.cookie.expiration. If no value is provided, cookies set the default to expire after one year (i.e. 365 days). If you don’t want to set a third party cookie at all it could be disabled by setting collector.cookie.enabled to false. Alternatively, it could be achieved if collector.cookie.expiration is set to 0 (from version 0.4.0).

Additional configuration options

There are a number of other options which should be set at your discretion based on your needs:

• collector.p3p allows you to configure a P3P header
• collector.crossDomain allows you to configure a cross domain policy
• collector.doNotTrackCookie lets you specify a first-party cookie the Scala Stream Collector will be checking against. If it is present in the incoming requests, the collector will ignore them.
• collector.redirectMacro allows you to template your redirect URL
• collector.rootResponse allows you to customize what the collector sends back when the / route is reached

enableDefaultRedirect = false
paths {
  "/com.acme/redirect-me" = "/r/tp2"
}