The new version 0.24.0 of Flowman has been released. This release contains a YAML schema generator to provide an appropriate YAML/JSON schema for the code editor of your choice. By using this schema, you will benefit from better syntax validation and auto-complete features, for example in VS Code or IntelliJ. Moreover model documentation capabilities and data quality checks have been improved by considering valuable input from real world projects.
Detailed Changes
- github-168: Support optional filters in data quality checks
- github-169: Support sub-queries in filter conditions
- github-171: Parallelize loading of project files
- github-172: Update CDP7 profile to the latest patch level
- github-153: Use non-privileged user in Docker image
- github-174: Provide application for generating YAML schema
Breaking Changes
We take backward compatibility very seriously. But sometimes a breaking change is needed to clean up code and to enable new features. This release contains some breaking changes, which are annoying but simple to fix. In order to avoid YAML schema inconsistencies, some entities needed to be renamed, as described in the following table:
| category | old kind | new kind |
| mapping | const | values |
| mapping | empty | null |
| mapping | read | relation |
| mapping | readRelation | relation |
| mapping | readStream | stream |
| relation | const | values |
| relation | empty | null |
| relation | jdbc | jdbcTable, jdbcQuery |
| relation | table | hiveTable |
| relation | view | hiveView |
| schema | embedded | inline |
