As of 11/12 2023 the new version 2.27.2 is out
Standardization and Conformance Fixes
- #2195 When using
s3
path, the protocol in the path description is used, instead of hard-codeds3://
prefix, thus supporting other s3 protocols too.
Menas Improvements
- #2199 Replaced DB-intensive
documentCount
forestimatedDocumentCount
. That should improve the load on MongoDB
Helper Scripts Improvements
- #2195 Added
--jceks-path
option to run scripts that will add--conf spark.hadoop.fs.s3a.bucket.SOME_NAME=jceks:/path/to/file.jceks
. Use it as--jceks-path spark.hadoop.fs.s3a.bucket.SOME_NAME=jceks:/path/to/file.jceks
- #2195 Added
s3a_wrapper.sh
script to auto-detect the jceks file and auto-clean-up temporary versions from S3A