As of 24/07 2020 the new version 2.10.0 is out

Conformance Improvements

  • #1437 New Conformance Rule FillNulls: value for the new column is the source column value where all the nulls are replaced by the literal specified.
  • #1446 New Conformance Rule Coalesce: value for the new column is the first non-null value from the list of columns.

Helper scripts Improvements

  • #1400 Added ability to specify (override) the jar to be used in helper scripts via environment variable SPARK_JOBS_JAR_OVERRIDE

Implementation Changes

  • #1015 Refactoring of SparkJobs.jar and deduplication of shared code between Standardization and Conformance.