Leverage Object Stores in Feldera Pipelines (S3, Azure Blob, GCS)

Last month, we announced S3 support for our pipelines. Today, we're making it real: you can now try it out directly in the online sandbox. No SQL changes needed—just tweak a bit of JSON in your pipeline settings, and you're set. Here’s an example configuration to store pipeline state in an AWS S3 bucket:

"storage": {
  "backend": {
    "name": "object",
    "config": {
      "url": "s3://bucket",
      "access_key_id": “<<KEY>>”,
      "secret_access_key": “<<SECRET>>”
    }
  }
}

Along with S3, Feldera also supports Google GCS, Azure Blob Storage, and other object stores as well. This unlocks a massive shift: pipelines are no longer constrained by RAM or local disk. With cloud object storage, your pipeline state can scale beyond 100 TB—something many of our enterprise users have already put to the test.

This feature is available immediately as part of Feldera Enterprise. If you're dealing with large-scale, real-time workloads and need "infinite" storage, contact us—we’d love to get you to try it.