As of 29/05 2020 the new version 2.6.0 is out

Standardization Improvements

  • #1319 CSV input parameters (delimiter, quote and escape) can be specified as unicode such as: U+00A1, u00a1 or 0161. Additionally, for specifying ""(empty string), meaning for example no escape, no quote, the keyword none can be used now.
  • #1353 Added support for loading ASCII fixed-width files using cobol as raw data format.

Standardization & Conformance Improvements

  • #1220 Sending errors to a Kafka topic can now be used enabling built-in plugins.

Menas Improvements

  • #1250 “Schema load from URL” - URL validation added - supports localhost, domain names, IP v4 and v6.
  • #1234 AVSC typed schema can be exported as same other types.

Menas Fixes

Streaming Improvements

  • #1333 Added an ability to specify information data source for streaming conformance jobs. It can be either an event time column, processing time, or a constant value.

Helper scripts Improvements

  • #670 Overpartitioning (too many tiny files) of output data is resolved by configuring Dynamic Resource Allocation (DRA) and adaptive execution.
  • #1098 Possible to sett DRA as a default for Enceladus jobs. DRA scales resource allocation to the actual size of the job (input size). Compared to fixed allocation, this makes a) small jobs more resource efficient b) large jobs faster. It reduces the effort for resource configuration on per case basis.

How to enable DRA see the How to run documentation page, chapter Helper scripts.