We are speaking at Kafka Summit!

Connecting Aiven Kafka to
Streambased

This guide shows how to connect your Streambased deployment to Aiven Kafka deployments.

Pradeep Sekar
Devrel
12/02/2024

Table of Contents

Share This Tutorial
Facebook
Twitter
LinkedIn

Introduction

This guide shows how to connect your Streambased deployment to Aiven Kafka deployments. In addition to following this guide we encourage you to review the documentation at: https://streambased-io.github.io/streambased/index.html and the Streambased webpage: https://streambased.io/. Feel free to reachout at info@streambased.io with any questions.

 

Note: We suggest running Streambased in a docker environment. Ensure that your environment has docker and docker-compose installed and is able to run containers with the amd64 platform. If you do not wish to use docker, bare metal steps can be found here: https://streambased-io.github.io/streambased/tutorial.html 

Step 1: Download Streambased

The easiest way to get started with Streambased is to use one of our pre-packaged demos. A demo template is available at: https://github.com/streambased-io/streambased-demos to download, locate the “<> Code” button and click “Download Zip”.

Unpack the zip and navigate to the “aiven” directory. This will be the base directory for our installation.

Run the following command to setup the environment: 

				
					./bin/setup.sh
				
			

Step 2: Create a truststore

From the Aiven services UI, click on your environment and click “Quick Connect” 

Select “Java” from the “Connect With” option and select “SASL” as the authentication method. 

Copy the truststore into the “serverConfig” directory in the Aiven demo and name it:

				
					client.truststore.jks
				
			

Step 4: Grab your Aiven config

From the “Overview” screen (Apache Kafka tab), select authentication method: “SASL”, and grab the following information:

				
					Service URI
User
Password

				
			

From the “Overview” screen (Schema Registry tab) grab the following information:

				
					Host
Port
User
Password

				
			

Step 5: Configure Streambased

A template for Streambased config has already been created in the demo. To use your own configurations we’ll first configure Streambased acceleration, change serverConfig/indexer.properties

				
					indexed.topics -> comma separated list of topics you wish to accelerate with Streambased

schema.registry.schema.registry.url -> from Schema Registry config enter: https://{Host}:{Port}

schema.registry.basic.auth.user.info -> from Schema Registry config enter: {User}:{Password}

consumer.bootstrap.servers -> from Apache Kafka config enter: {Service URI}

consumer.sasl.jaas.config -> from Apache Kafka config enter: org.apache.kafka.common.security.plain.PlainLoginModule required username='{User}' password='{Password}';

				
			

Next let’s configure the Streambased Server, change serverConfig/catalog/kafka.properties

				
					kafka.nodes -> from Apache Kafka config enter: {Service URI}

kafka.confluent-schema-registry-url -> from Schema Registry config enter: https://{Host}:{Port}

kafka.confluent-schema-registry-basic-auth-user-info -> from Schema Registry config enter: {User}:{Password}

				
			

Finally, change serverConfig/aiven_client.properties

				
					bootstrap.servers ->   from Apache Kafka config enter: {Service URI}

sasl.jaas.config -> from Apache Kafka config enter: org.apache.kafka.common.security.plain.PlainLoginModule required username='{User}' password='{Password}';

				
			

Step 6: Start Streambased

All that’s left to do is to start the Streambased services:

				
					docker-compose up -d
				
			

Now you can connect to all of your Kafka data with JDBC. For instructions on how to do this check out our guides for Superset, DBT, Tableau and Jupyter

Share This Tutorial
Facebook
Twitter
LinkedIn