Incremental Update 13

Today, we’re happy to announce the release of feldera v0.33! This updates brings support for a new and popular data format: Iceberg!

Iceberg

In v0.33 you can now ingest data from Apache Iceberg, a popular open table data format, into Feldera. You can find more information on how to achieve this in the docs, but generally speaking we support two different catalogues (Amazon Glue or REST based) and can ingest iceberg data from various sources (local files, S3, or GCS).

More connector updates

Aside from a new Iceberg connector, the rest of our connector eco-system also saw some significant upgrades:

• The CSV parser now supports custom delimiters and can be configured to omit header lines
• The JSON format can now be configured to accept multi-line JSON
• We fixed a deadlock in the HTTP input connector that would appear when using concurrent HTTP connections to a single pipeline

Connector Orchestration

We also polished and documented the API to orchestrate input connectors. This API gives you fine-grained control over which connectors are active at any point in time during the lifetime of a pipeline so you can control exactly when data from different sources are ingested.

Webconsole Updates

Last but not least, the web-console also saw a series of updates. We are now almost complete with the UI redesign that we started two releases ago. We're very happy with how it turned out, please try it and let us know what you think!