Flowman 0.27.0 released


We are happy to announce a fresh and exciting release of Flowman. The focus of this release is again on improving JDBC support. In particular the following important features have been implemented

  • New ‘ jdbcCommand’ target for executing arbitrary SQL statements for JDBC sinks
  • Support direct SQL statements in JDBC relations for creating tables
  • Upgrade Delta Lake to 2.0.0/2.1.0 (for Spark 3.2 and 3.3 respectively)
  • Better error messages

Of course, this release also contains a bunch of smaller bug fixes and other improvements.

Detailed Changes

  • github-232: [BUG] Column descriptions should be propagates in UNIONs
  • github-233: [BUG] Missing Hadoop dependencies for S3, Delta, etc
  • github-235: Implement new rest hook with fine control
  • github-229: A build target should not fail if Impala “COMPUTE STATS” fails
  • github-236: ‘copy’ target should not apply output schema
  • github-237: jdbcQuery relation should use fields “sql” and “file” instead of “query”
  • github-239: Allow optional SQL statement for creating jdbcTable
  • github-238: Implement new ‘jdbcCommand’ target
  • github-240: [BUG] Data quality checks in documentation should not fail on NULL values
  • github-241: Throw an error on duplicate entity definitions
  • github-220: Upgrade Delta-Lake to 2.0 / 2.1
  • github-242: Switch to Spark 3.3 as default
  • github-243: Use alternative Spark MS SQL Connector for Spark 3.3
  • github-244: Generate project HTML documentation with optional external CSS file


As usual, you can download the latest version from the Download section or directly from GitHub.

Flowman 1.1.0 released

We are happy to announce the release of Flowman 1.1.0. This release contains many small improvements and bugfixes. Flowman now finally supports

Flowman at Smartclip

smartclip is a successful and growing company specialized for online video advertisement. More importantly, smartclip was one of the first companies implementing