We are happy to announce a fresh and exciting release of Flowman. The focus of this release is again on improving JDBC support. In particular the following important features have been implemented
- New ‘ jdbcCommand’ target for executing arbitrary SQL statements for JDBC sinks
- Support direct SQL statements in JDBC relations for creating tables
- Upgrade Delta Lake to 2.0.0/2.1.0 (for Spark 3.2 and 3.3 respectively)
- Better error messages
Of course, this release also contains a bunch of smaller bug fixes and other improvements.
Detailed Changes
- github-232: [BUG] Column descriptions should be propagates in UNIONs
- github-233: [BUG] Missing Hadoop dependencies for S3, Delta, etc
- github-235: Implement new
resthook with fine control - github-229: A build target should not fail if Impala “COMPUTE STATS” fails
- github-236: ‘copy’ target should not apply output schema
- github-237: jdbcQuery relation should use fields “sql” and “file” instead of “query”
- github-239: Allow optional SQL statement for creating jdbcTable
- github-238: Implement new ‘jdbcCommand’ target
- github-240: [BUG] Data quality checks in documentation should not fail on NULL values
- github-241: Throw an error on duplicate entity definitions
- github-220: Upgrade Delta-Lake to 2.0 / 2.1
- github-242: Switch to Spark 3.3 as default
- github-243: Use alternative Spark MS SQL Connector for Spark 3.3
- github-244: Generate project HTML documentation with optional external CSS file
Download
As usual, you can download the latest version from the Download section or directly from GitHub.
