Connecting Aiven Kafka To Streambased

Introduction

‍

This guide shows how to connect your Streambased deployment to Aiven Kafka deployments. In addition to following this guide we encourage you to review the documentation at: https://streambased-io.github.io/streambased/index.html and the Streambased webpage: https://streambased.io/. Feel free to reachout at info@streambased.io with any questions.

Note: We suggest running Streambased in a docker environment. Ensure that your environment has docker and docker-compose installed and is able to run containers with the amd64 platform. If you do not wish to use docker, bare metal steps can be found here: https://streambased-io.github.io/streambased/tutorial.html

‍

Step 1: Download Streambased

‍

The easiest way to get started with Streambased is to use one of our pre-packaged demos. A demo template is available at: https://github.com/streambased-io/streambased-demos to download, locate the “<> Code” button and click “Download Zip”.

‍

‍

Unpack the zip and navigate to the “aiven” directory. This will be the base directory for our installation.

Run the following command to setup the environment:

‍

./bin/setup.sh

‍

Step 2: Create a truststore

‍

From the Aiven services UI, click on your environment and click “Quick Connect”

‍

‍

Select “Java” from the “Connect With” option and select “SASL” as the authentication method.

‍

‍

Copy the truststore into the “serverConfig” directory in the Aiven demo and name it:

‍

client.truststore.jks

‍

Step 4: Grab your Aiven config

‍

From the “Overview” screen (Apache Kafka tab), select authentication method: “SASL”, and grab the following information:

‍

Service URI
User
Password

‍

From the “Overview” screen (Schema Registry tab) grab the following information:

Host
Port
User
Password

‍

Step 5: Configure Streambased

‍

A template for Streambased config has already been created in the demo. To use your own configurations we’ll first configure Streambased acceleration, change serverConfig/indexer.properties

‍

indexed.topics -> comma separated list of topics you wish to accelerate with Streambased

schema.registry.schema.registry.url -> from Schema Registry config enter: https://{Host}:{Port}

schema.registry.basic.auth.user.info -> from Schema Registry config enter: {User}:{Password}

consumer.bootstrap.servers -> from Apache Kafka config enter: {Service URI}

consumer.sasl.jaas.config -> from Apache Kafka config enter: org.apache.kafka.common.security.plain.PlainLoginModule required username='{User}' password='{Password}';

‍

Next let’s configure the Streambased Server, change serverConfig/catalog/kafka.properties

kafka.nodes -> from Apache Kafka config enter: {Service URI}

kafka.confluent-schema-registry-url -> from Schema Registry config enter: https://{Host}:{Port}

kafka.confluent-schema-registry-basic-auth-user-info -> from Schema Registry config enter: {User}:{Password}

‍

Step 6: Start Streambased

‍

All that’s left to do is to start the Streambased services:

‍

docker-compose up -d

Now you can connect to all of your Kafka data with JDBC. For instructions on how to do this check out our guides for Superset, DBT, Tableau and Jupyter

‍

‍