Connecting To Streambased From DBT

Pre-requisites

‍

You must have a running Streambased server before following this guide. For details on how to run Streambased see the documentation here: https://streambased-io.github.io/streambased/index.html or run one of the demos here: https://github.com/streambased-io/streambased-demos

‍

Step 1: Install dependencies

‍

To work with DBT we must first install the Trino libraries:

‍

pip install dbt-trino

‍

Step 2: Create a project

‍

Let’s init a DBT project:

‍

dbt init

‍

Give the project a meaningful name and select the trino database:

‍

‍

‍

Step 3: Configure the database connection

‍

In the newly created project directory create a new file named “profiles.yml”. This should contain the following:

‍

<project name>:
target: dev
outputs:
  dev:
    type: trino
    method: none
    user: e30=
    database: kafka
    host: localhost
    port: 8080
    schema: streambased
    threads: 1

‍

Note: The above assumed a local Streambased server, modify the host and port according to your deployment.

‍

‍

Step 4: Create a model

‍

In the “models” directory create a new file with the extension “.sql” (in this example src_customers.sql). This should contain the SQL you wish to run:

‍

SELECT * FROM kafka.streambased.customers WHERE name = 'TOM SCOTT'

‍

‍

Step 5: Run the query

‍

We can test run the query directly with DBT:

‍

dbt show --select models/src_customers.sql

‍

‍

‍