More of a memory aid for me as I’ll forget…..

Kafka Connect’s default logging goes to the console, I prefer tailing and grepping files instead. I’m using Confluent Kafka 5.1.2 but the file locations will be pretty much similar if you are using the Apache version.

Open up the connect-log4.properties file in $KAFKA_HOME/etc/kafka directory.

Add the following lines.

log4j.appender.logFile=org.apache.log4j.DailyRollingFileAppender
log4j.appender.logFile.DatePattern='.'yyyy-MM-dd-HH
log4j.appender.logFile.File=/tmp/connect-worker.log
log4j.appender.logFile.layout=org.apache.log4j.PatternLayout
log4j.appender.logFile.layout.ConversionPattern=[%d] %p %m (%c)%n

Change:

log4j.rootLogger=INFO, stdout

To:

log4j.rootLogger=INFO, logFile

And then restart Kafka. The new log file will be in the /tmp folder.

Also worth noting…..

The connect-standalone.properties file seems to have to changed from version to version. The Connect jar path in 5.1.2 for example is share/java where it’s now /usr/share/java in 5.3.1, I was trying to figure out why no Connect plugins were working at all. So it’s worth setting up either a symlink or copying all the jars to that location, or simpler would be to change the properties file.