Adding file logging to Kafka Connect – #Kafka #Connect #Streaming #Data #Devops

More of a memory aid for me as I’ll forget…..

Kafka Connect’s default logging goes to the console, I prefer tailing and grepping files instead. I’m using Confluent Kafka 5.1.2 but the file locations will be pretty much similar if you are using the Apache version.

Open up the file in $KAFKA_HOME/etc/kafka directory.

Add the following lines.

log4j.appender.logFile.layout.ConversionPattern=[%d] %p %m (%c)%n


log4j.rootLogger=INFO, stdout


log4j.rootLogger=INFO, logFile

And then restart Kafka. The new log file will be in the /tmp folder.

Also worth noting…..

The file seems to have to changed from version to version. The Connect jar path in 5.1.2 for example is share/java where it’s now /usr/share/java in 5.3.1, I was trying to figure out why no Connect plugins were working at all. So it’s worth setting up either a symlink or copying all the jars to that location, or simpler would be to change the properties file.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: