As of 29/05 2020 the new version 2.6.0 is out
Standardization Improvements
- #1319 CSV input parameters (delimiter, quote and escape) can be specified as unicode such as:
U+00A1
,u00a1
or0161
. Additionally, for specifying""
(empty string), meaning for example no escape, no quote, the keywordnone
can be used now. - #1353 Added support for loading ASCII fixed-width files using
cobol
as raw data format.
Standardization & Conformance Improvements
Menas Improvements
- #1250 “Schema load from URL” - URL validation added - supports localhost, domain names, IP v4 and v6.
- #1234 AVSC typed schema can be exported as same other types.
Menas Fixes
- #1347 Each time a schema gets updated by uploading/remote-loading a new schema-file, the attachment is filed under the correct (same) version.
- #1347 MongoDb remapping script added to 1347-attachment-version-off-by-one-mapping-script.js
Streaming Improvements
- #1333 Added an ability to specify information data source for streaming conformance jobs. It can be either an event time column, processing time, or a constant value.
Helper scripts Improvements
- #670 Overpartitioning (too many tiny files) of output data is resolved by configuring Dynamic Resource Allocation (DRA) and adaptive execution.
- #1098 Possible to sett DRA as a default for Enceladus jobs. DRA scales resource allocation to the actual size of the job (input size). Compared to fixed allocation, this makes a) small jobs more resource efficient b) large jobs faster. It reduces the effort for resource configuration on per case basis.
How to enable DRA see the How to run documentation page, chapter Helper scripts.