As of 18/09 2020 the new version 2.13.0 is out
Standardization Improvements
- #1482 New metadata
strict_parsing
boolean option on Decimal type. If true It will create a cast error in errCol when the number of decimal digits in the input value is higher than the defined scale of the type.
Conformance Improvements
- #1480 Adds a check for Conformance Rules that they are not trying to modify a column from the original schema. If this is desired user can use option
conformance.allowOriginalColumnsMutability
and set it to true. In that case, it will only print a warning.
Helper Scripts Improvements
- #1481 Spark jobs are run disconnected on cluster mode for better resource usage.
- #1501 Updated Helper scripts to recognize
--empty-values-as-nulls
and--null-value
options. - #1513 Helper scripts for Windows.