Pipeline Components and Applications

  1. Home
  2. Docs
  3. Pipeline Components and Applications
  4. Dataflow Runner
  5. Guide for DevOps

Guide for DevOps

Dataflow runner is a zero-dependency binary and can be found at the following locations:

Once downloaded, unzip it (Linux for example):

$ unzip dataflow_runner_0.5.0_linux_amd64.zip

Run it like so:

$ ./dataflow-runner NAME: dataflow-runner - Run templatable playbooks of Hadoop/Spark/et al jobs on Amazon EMR USAGE: dataflow-runner [global options] command [command options] [arguments...] VERSION: 0.5.0 AUTHOR: Joshua Beemster <support@snowplowanalytics.com> COMMANDS: up Launches a new EMR cluster run Adds jobflow steps to a running EMR cluster down Terminates a running EMR cluster run-transient Launches, runs and then terminates an EMR cluster help, h Shows a list of commands or help for one command GLOBAL OPTIONS: --log-level value logging level, possible values are debug,info,warning,error,fatal,panic (default: "info") --help, -h show help --version, -v print the version COPYRIGHT: (c) 2016-2020 Snowplow Analytics Ltd
Code language: PHP (php)