Snowplow is designed to make it easy for you to change your tracking design in a safe and backwards-compatible way as your organisational data needs evolve.
JSON schemas are used to describe the structure of your data. Each schema carries a version number expressed as three numeric digits. As your schema evolves, all previous versions of that schema remain on your registry to ensure backwards-compatibility.
Why is versioning important?
As well as good practice, versioning has an important role in telling Snowplow Loaders how to handle the changes when loading into your data warehouse(s). For example, for certain changes there will be a need to create new columns, update columns or even create whole new tables. For this reason, it’s important you understand when your change is breaking and version correctly.
What are the rules for versioning my schemas?
- New schemas always start at version
1-0-0 - If you make a backward compatible change to a schema, you increment the patching version i.e.
1-0-0>1-0-1. A common example of this would be adding an optional field to the schema. - If you make a change that breaks a schema e.g. add a new compulsory field, or change a field type, this creates a new major version i.e.
1-0-0->2-0-0 - If you widen a field so that it can have additional types e.g. make a field that used to be of type
integerand change it to accept eitherstringorinteger, increment the minor version i.e.1-0-0->1-1-0
For more information read about SchemaVer.
Do I have to increase the version every time I make a change?
Snowplow has the concept of ‘patching’ a schema, that is making a change and keeping the version the same.
When drafting and testing a schema you might not need to keep increasing your version number. This makes the cycle of change and test more efficient as you do not need to increment the version in your tracking code. However, when you promote edits to a Production environment you are required to increase the version.