I want to connect streamsheets to a Kafka stream in Confluent Cloud. I Have problems to generate a Kafka_Connector in Streamsheets.
Do anyone of you have a full tutorial for a Kafka integration? From setting up a Connector to a Producer which sends messages to Kafka?
Hi phil,
We are working on a lot of new documentation and Kafka will be part of these. 
But this may take a little. So until then let’s do this:
I have a local setup I can give you.
Basically my Kafka installation is this one:
docker-compose.yml (945 Bytes)
use docker-compose up to start Kafka.
I set the docker network to “cedalo-platform” so it matches the Streamsheets Installation.
Now import this to Streamsheets:
Kafka test.json (14.9 KB)
It is a very simple App, which produces and consumes Kafka data.
It comes with the predefined streams, which will connect to the kafka setup from above.
You can edit the connector to connect to yours.
I hope that is of help to you
Cheers Tizian
Hello Tizian,
cool, I get your example to work. Thanks a lot for that.
In your example you use a local Kafka broker without authentication.
In my use case, I don’t have a local broker. I have a broker in the Confluent Cloud with TLS/SASL authentication. I make the same settings as you, but I get a connection timeout. The only change I made was to replace the Kafka URL from kafka:29092 to brokerurl:29092, select TLS/SASL authentication and set my username and password. Also brokerurl:9092 does not work.
When I go into the streamsheet container and execute “ping brokerurl” I get a reply.
My Username and Password are working with a Python App so they are correct too.
You can check the logs to understand what the exact issues may be:
docker cp streamsheets:/var/log/service-streams/stdout.log stream_stdout.log
It will copy the logs in your directory file.
DON´T Share this with the forum. This stores personal information.
If you think you found something and don´t know yourself what it means, you can paste the specific log in here I will try to help.
Thank you very much,
I found out I had a problem with our network configuration. The port for Kafka was switched off by the IT department. Now I get a connection.
1 Like