Flowman 1.1.0 released

We are happy to announce the release of Flowman 1.1.0. This release contains many small improvements and bugfixes. Flowman now finally supports Spark 3.4.1. Major Features As mentioned above, Spark 3.4.1 is now officially supported by Flowman. For users running Flowman in Azure Synapase or AWS EMR, you can now

Read More »

Flowman — A Declarative ETL Framework for Apache Spark

Don’t reinvent the wheel by writing more boilerplate code. Focus on critical business logic and delegate the tricky details to a clever tool. Introduction Apache Spark is a powerful framework for building flexible and scalable data processing applications. Spark offers a relatively simple yet powerful API that provides the capabilities

Read More »

Flowman at Smartclip

smartclip is a successful and growing company specialized for online video advertisement. More importantly, smartclip was one of the first companies implementing Flowman for their business critical core data pipelines. These are responsible for building the data foundation for both internal an external reporting. The amount of data processed every

Read More »

Flowman 1.0.0 released

Flowman version 1.0.0 has finally arrived. For several years, multiple companies are using Flowman in production as a robust and reliable solution for efficiently building data transformation pipelines. Therefore it only made sense to leave the zero-versions behind and increase the major version number to “1” to underline the robustness

Read More »

Flowman 1.0 has landed

We are excited and proud to announce the official release of Flowman 1.0. Flowman is a tool for performing complex data transformations in a structured manner. It follows an everything-as-code approach with a purely declarative specification language for describing the transformations and data models. This helps enterprises to accelerate their

Read More »

Flowman 0.30.0 released

A little bit later than anticipated, a new version of Flowman has been released. It didn’t make it as a Christmas present, but it became a welcome New Years present instead. This release has many technical changes under the hood as a preparation for being able to build fat jars.

Read More »

Flowman 0.29.0 released

We are happy to announce a fresh and exciting release of Flowman. This release is mainly a maintenance release with a couple of minor improvements Of course, this release also contains a bunch of smaller bug fixes and other improvements. Detailed Changes Download As usual, you can download the latest

Read More »

Flowman 0.28.0 released

We are happy to announce a fresh and exciting release of Flowman. This release is mainly a maintenance release with a couple of minor improvements New command line options -X and -XX for increasing log level Improve compatibility with MySQL and MariaDB New Maven archetype for quickly setting up a

Read More »

Flowman 0.27.0 released

We are happy to announce a fresh and exciting release of Flowman. The focus of this release is again on improving JDBC support. In particular the following important features have been implemented New ‘ jdbcCommand’ target for executing arbitrary SQL statements for JDBC sinks Support direct SQL statements in JDBC

Read More »

Flowman 0.26.0 released

The new version 0.26.0 of Flowman has been released. This version contains a lot of work with a strong focus on improving working with JDBC targets like MariaDB/MySQL, Postgres, MS SQL Server, Azure SQL and Oracle. For example, column collations and comments are now correctly propagated into relational databases, changing

Read More »