There are several extension points for this application:
- Bad Row types
By definition steps allow performing modifications on existing bad row data points’ payloads. We defined a configuration DSL for Steps that is turned into actions with
Inspectables are data structures on which Steps can be performed. Therefore in order to add a new
config.StepDSL structure for it has to be defined
- an implementation of action defined as per DSL has to be defined for
The latter can be described in form of
transform function that builds upon a recursive generic JSON-transforming structure.
Conditions are a lot like steps but as they are triggered earlier before
Inspectables are created. They do operate upon JSON data and need to implement behavior for basic JSON types. Therefore in order to add a new
config.ConditionDSL structure for it has to be defined
- functions for performing
Conditionapplication for basic/composite JSON types have to be defined
Bad Row types
Once new recoverable Bad Row types are added they need to be turned into
Recoverable by supplying appropriate
Recoverable instances. If a new
Payload format is added it has to be turned into
Inspectable as well.
It is important to note that we only represent recoverable bad rows as
Recoverable instances. Not all
BadRows are recoverable and we strongly believe in not representing invalid states as possible.