Operational and analytical architecture unified - As Kafka
Streambased serves Kafka clients via K.S.I. (Kafka Service for Iceberg). Kafka topics served by this component represent the same core Streambased seamless logical view of Kafka and Iceberg but this time served via Kafka protocol. Kafka's partition and offset concepts are mapped from columns in the Iceberg data allowing Kafka clients to interact with them as if they were Kafka topics.
The Tech
The K.S.I. architecture consists of:
A proxy
Streambased serves Kafka requests via a proxy component that sits between the underlying Kafka/Iceberg infrastructure and clients. This proxy (based on the open-source Kroxylicious project) intercepts requests that can be served from the Streambased logical view and satisfies them via a separate code path. All requests that are not relevant to the logical view are simply passed through.
An Iceberg engine
In order to fetch Iceberg table records (that will later be transformed and served as Kafka), K.S.I. must employ an Iceberg engine to query the underlying infrastructure. Streambased supports external engines (such as Trino or Spark) for this purpose, or it can use a small embedded service.
A row processor
This component reformats the column-oriented Iceberg data into the key/value-based messages that Kafka clients expect. Governance steps like Schema Registry integration are applied here too.
Let's find the right solution for your data
We're here to help you unlock the full potential of your streaming data. Tell us about your challenges or ideas — and let's explore how Streambased can support your business.