We are happy to announce the release of Flowman 1.1.0. This release contains many small improvements and bugfixes. Flowman now finally supports Spark 3.4.1.
Major Features
As mentioned above, Spark 3.4.1 is now officially supported by Flowman. For users running Flowman in Azure Synapase or AWS EMR, you can now use the cloud native secrets managers to store credentials.
Detailed Changes
- github-413: Support Azure Key Vault for retrieving secrets
- github-415: Improve documentation for Velocity templating
- github-361: Remove broken support for Databricks
- github-402: Support Spark 3.4
- github-417: Fix URL to flowman.io in all Maven modules
- github-416: Support specifying multiple targets separated by commas on CLI
- github-414: Support AWS Secrets Manager for retrieving secrets
- github-419: Add a command line option to kernel server to listen on specific address
- github-326: Truncate target graphs at sinks
- github-410: Support relative paths in project imports
- github-421: Provide new ‘session’ environment variable
- github-422: Upgrade Spark to 3.4.1
- github-423: Migrating a MariaDB/MySQL table from a text type to a numeric type fails
- github-425: Support building and running Flowman with Java 17
- github-388: Replace Akka http with Jersey/Jetty in Flowman History Server
- github-385: Update Flowman Tutorial
- github-412: Create tutorial for using Flowman Maven Plugin
- github-371: Automate EMR integration test via AWS CLI
- github-431: Update EMR build profil to 6.12
- github-428: Move project selector into top bar in Flowman History Server
- github-433: Add Trino JDBC driver as plugin
- github-434: Use sshj instead of ganymed for sftp
- github-435: Flowman should also load a project.yaml (with an ‘a’ in the extension)
- github-436: Document minimum Maven version of Flowman-maven-plugin
- github-438: Empty or non-existing module directories should not lead to an error
- github-452: [BUG] SQL assertions do not support empty strings as expected values
- github-450: Update Spark to 3.3.3
- github-454: Fix scope of netty-codec
- github-446: Fix deadlock when running targets in parallel
