Consent Tracking Custom Module

This custom module is built as an extension of the dbt-snowplow-unified package.

It transforms raw consent_preferences and cmp_visible event data into derived tables for easier querying. These events are generated by the Enhanced Consent plugin of the JavaScript tracker.

Important

For the incremental logic to work within the module you must use at least RDB Loader v4.0.0, as the custom module relies on the additional load_tstamp field for dbt native incrementalisation.

Whenever a new consent version is added to be tracked, the model expects an allow_all event in order to attribute the events to the full list of latest consent scopes. It is advisable to send a test event of that kind straight after deployment so that the model can process the data accurately.

Overview

This custom module consists of a series of dbt models which produce the following aggregated models from the raw consent tracking events:

• snowplow_unified_consent_log: Snowplow incremental table showing the audit trail of consent and Consent Management Platform (cmp) events.
• snowplow_unified_consent_users: Incremental table of user consent tracking stats.
• snowplow_unified_consent_totals: Summary of the latest consent status, per consent version.
• snowplow_unified_cconsent_scope_status: Aggregate of current number of users consented to each consent scope.
• snowplow_unified_cmp_stats: Used for modeling cmp_visible events and related metrics.
• snowplow_unified_consent_versions: Incremental table used to keep track of each consent version and its validity.

Operation

It is assumed that the Snowplow unified package is already installed and configured as per the Quick Start instructions.

Enable the module

You can enable the custom module through the snowplow__enable_consent variable most conveniently set in your dbt_project.yml file:

dbt_project.yml

vars:
  snowplow_unified:
    snowplow__enable_consent: true
    

Run the module

If you have previously run the model without this optional module enabled, you can simply enable the module and run dbt run --selector snowplow_unified as many times as needed for this module to catch up with your other data. If you only wish to process this from a specific date, be sure to change your snowplow__start_date, or refer to the Custom module section for a detailed guide on how to achieve this the most efficient way.

If you haven't run the web package before, then you can run it using dbt run --selector snowplow_unified either through your CLI, within dbt Cloud, or for Enterprise customers you can use the BDP console. In this situation, all models will start in-sync as no events have been processed.