01: Apache Kafka example with Java – getting started tutorial

Apache Kafka with Java getting started tutorial demonstrates how quickly you can get started with Kafka using Docker.

Apache Kafka Example – Getting started Java tutorial

Step 1: Make sure Docker engine is installed on your computer. For example on a Mac OS $ brew cask install docker or on Windows.

Step 2: Start the Docker engine on your operating system.

Kafka services on Docker

Step 3: Create the below docker-compose.yml file to run your Kafka, zookeeper & Apache Kafka Cluster Visualization (AKHQ) services. The images for these services are sourced from Docker hub.

version: "2"

services:
  zookeeper:
    image: docker.io/bitnami/zookeeper:3.8
    ports:
      - "2181:2181"
    environment:
      - ALLOW_ANONYMOUS_LOGIN=yes

  kafka:
    image: docker.io/bitnami/kafka:3.4
    ports:
      - "9093:9093"
    environment:
      - KAFKA_CFG_ZOOKEEPER_CONNECT=zookeeper:2181
      - ALLOW_PLAINTEXT_LISTENER=yes
      - KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=KAFKA_CLIENT:PLAINTEXT,EXTERNAL:PLAINTEXT
      - KAFKA_CFG_LISTENERS=KAFKA_CLIENT://:9092,EXTERNAL://:9093
      - KAFKA_CFG_ADVERTISED_LISTENERS=KAFKA_CLIENT://kafka:9092, EXTERNAL://localhost:9093
      - KAFKA_INTER_BROKER_LISTENER_NAME=KAFKA_CLIENT
    depends_on:
      - zookeeper

  akhq:
    image: tchiotludo/akhq
    ports:
      - "9099:8080"
    environment:
      AKHQ_CONFIGURATION: |
        akhq:
          connections:
            docker-kafka-server:
              properties:
                bootstrap.servers: "kafka:9092"
    depends_on:
      - kafka

version: "2"

services:

zookeeper:

image: docker.io/bitnami/zookeeper:3.8

ports:

- "2181:2181"

environment:

- ALLOW_ANONYMOUS_LOGIN=yes

kafka:

image: docker.io/bitnami/kafka:3.4

ports:

- "9093:9093"

environment:

- KAFKA_CFG_ZOOKEEPER_CONNECT=zookeeper:2181

- ALLOW_PLAINTEXT_LISTENER=yes

- KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=KAFKA_CLIENT:PLAINTEXT,EXTERNAL:PLAINTEXT

- KAFKA_CFG_LISTENERS=KAFKA_CLIENT://:9092,EXTERNAL://:9093

- KAFKA_CFG_ADVERTISED_LISTENERS=KAFKA_CLIENT://kafka:9092, EXTERNAL://localhost:9093

- KAFKA_INTER_BROKER_LISTENER_NAME=KAFKA_CLIENT

depends_on:

- zookeeper

akhq:

image: tchiotludo/akhq

ports:

- "9099:8080"

environment:

AKHQ_CONFIGURATION: |

akhq:

connections:

docker-kafka-server:

properties:

bootstrap.servers: "kafka:9092"

depends_on:

- kafka

Step 4: Run the Docker container with the above services on the engine via a command-line terminal window where this file docker-compose.yml. Say, if the file is at /Users/yourfolder/projects/simple-kafka/docker-compose.yml

$ cd /Users/yourfolder/projects/simple-kafka/
$ docker-compose up -d

$ cd /Users/yourfolder/projects/simple-kafka/

$ docker-compose up -d

It will take a few minutes to stand up the Docker containers with the requested services for Kafka – Kafka on port 9093, Zookeeper on port 2181 & Apache Kafka Cluster Visualization (AKHQ) on port 9099.

Step 5: You can open a browser to check if AKHQ is running on localhost:9099. This is the Kafka UI.

AKHQ – Apache Kafka UI

Java Project

Step 6: Create a new maven based Java project simple-kafka. For example, in Intellij IDEA File -> New Project -> Maven.

The pom.xml file with Kafka & logback dependencies:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>org.example</groupId>
    <artifactId>simple-kafka</artifactId>
    <version>1.0-SNAPSHOT</version>

    <properties>
        <maven.compiler.source>8</maven.compiler.source>
        <maven.compiler.target>8</maven.compiler.target>
    </properties>

    <dependencies>
        <dependency>
            <groupId>org.apache.kafka</groupId>
            <artifactId>kafka-clients</artifactId>
            <version>3.0.0</version>
        </dependency>

        <dependency>
            <groupId>ch.qos.logback</groupId>
            <artifactId>logback-classic</artifactId>
            <version>1.2.3</version>
        </dependency>
    </dependencies>

</project>

<?xml version="1.0" encoding="UTF-8"?>

<project xmlns="http://maven.apache.org/POM/4.0.0"

xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">

<groupId>org.example</groupId>

<artifactId>simple-kafka</artifactId>

<version>1.0-SNAPSHOT</version>

<maven.compiler.source>8</maven.compiler.source>

<maven.compiler.target>8</maven.compiler.target>

</properties>

<groupId>org.apache.kafka</groupId>

<artifactId>kafka-clients</artifactId>

</dependency>

<groupId>ch.qos.logback</groupId>

<artifactId>logback-classic</artifactId>

</dependency>

</dependencies>

</project>

Step 7: Set up logging xml file logback.xml.

<?xml version="1.0" encoding="UTF-8"?>
<configuration>

    <appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
        <layout class="ch.qos.logback.classic.PatternLayout">
            <Pattern>
                %d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n
            </Pattern>
        </layout>
    </appender>

    <logger name="com.myapp" level="debug" additivity="false">
        <appender-ref ref="CONSOLE"/>
    </logger>

    <root level="error">
        <appender-ref ref="CONSOLE"/>
    </root>

</configuration>

<?xml version="1.0" encoding="UTF-8"?>

%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n

</Pattern>

</layout>

</appender>

<appender-ref ref="CONSOLE"/>

</logger>

<appender-ref ref="CONSOLE"/>

</root>

</configuration>

Java Consumer

Step 8: Write a consumer in Java as shown below.

package com.myapp;

import org.apache.kafka.clients.consumer.ConsumerConfig;
import org.apache.kafka.clients.consumer.ConsumerRecord;
import org.apache.kafka.clients.consumer.ConsumerRecords;
import org.apache.kafka.clients.consumer.KafkaConsumer;
import org.apache.kafka.common.serialization.StringDeserializer;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import java.time.Duration;
import java.util.Arrays;
import java.util.Properties;

public class ConsumerApp {

    private static final Logger LOG = LoggerFactory.getLogger(ConsumerApp.class);

    public static void main(String[] args) {
        String topic = "my-first-topic";
        String group = "my-first-application";
        Properties props = new Properties();
        props.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9093");
        props.put(ConsumerConfig.GROUP_ID_CONFIG, group);
        props.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class.getName());
        props.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class.getName());

        KafkaConsumer<String, String> consumer = new KafkaConsumer<>(props);

        consumer.subscribe(Arrays.asList(topic));
        LOG.info("Subscribed to Kafka Topic " + topic);

        //Keeps running & checks for messages every 5 seconds
        while (true) {
            ConsumerRecords<String, String> records = consumer.poll(Duration.ofSeconds(5));
            for (ConsumerRecord<String, String> record : records)
                LOG.info(String.format("************ offset = %d, key = %s, value = %s\n",
                        record.offset(), record.key(), record.value()));
        }
    }
}

package com.myapp;

import org.apache.kafka.clients.consumer.ConsumerConfig;

import org.apache.kafka.clients.consumer.ConsumerRecord;

import org.apache.kafka.clients.consumer.ConsumerRecords;

import org.apache.kafka.clients.consumer.KafkaConsumer;

import org.apache.kafka.common.serialization.StringDeserializer;

import org.slf4j.Logger;

import org.slf4j.LoggerFactory;

import java.time.Duration;

import java.util.Arrays;

import java.util.Properties;

public class ConsumerApp {

private static final Logger LOG = LoggerFactory.getLogger(ConsumerApp.class);

public static void main(String[] args) {

String topic = "my-first-topic";

String group = "my-first-application";

Properties props = new Properties();

props.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9093");

props.put(ConsumerConfig.GROUP_ID_CONFIG, group);

props.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class.getName());

props.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class.getName());

KafkaConsumer<String, String> consumer = new KafkaConsumer<>(props);

consumer.subscribe(Arrays.asList(topic));

LOG.info("Subscribed to Kafka Topic " + topic);

//Keeps running & checks for messages every 5 seconds

while (true) {

ConsumerRecords<String, String> records = consumer.poll(Duration.ofSeconds(5));

for (ConsumerRecord<String, String> record : records)

LOG.info(String.format("************ offset = %d, key = %s, value = %s\n",

record.offset(), record.key(), record.value()));

}

Run the application within intellij IDEA:

.....
17:35:12.305 [main] INFO  com.myapp.ConsumerApp - Subscribed to Kafka Topic my-first-topic

.....

17:35:12.305 [main] INFO com.myapp.ConsumerApp - Subscribed to Kafka Topic my-first-topic

Produce a message from akHQ.io UI

Step 9: Go to localhost:9099 akGQ.io & create a new topic named “my-first-topic“. Click on the button “Create a topic” at the right bottom corner.

Step 10: Double click on the topic “my-first-topic” & then click on “Produce to my-first-topic“.

Apache Kafka – Produce a message from akHQ.io

Step 11: Click on the “Produce” button to send the message to the topic.


"my-message": "my test messge produced from akHQ.io"

"my-message": "my test messge produced from akHQ.io"

Java code ConsumerApp.java consumes the message

Step 12: In the console window within Intellij IDEA, the message will be consumed by ConsumerApp.java, which is already running.

17:48:16.661 [main] INFO  com.myapp.ConsumerApp - ************ offset = 0, key = my-test-key, value = "my-message": "my test messge produced from akHQ.io"

1 2	17:48:16.661 [main] INFO com.myapp.ConsumerApp - ************ offset = 0, key = my-test-key, value = "my-message": "my test messge produced from akHQ.io"

Bring down the Docker container

$ cd /Users/yourfolder/projects/simple-kafka/
$ docker-compose up -d

$ cd /Users/yourfolder/projects/simple-kafka/

$ docker-compose up -d

More Apache Kafka Examples

1) Apache Kafka example with Java Producer & Consumer Tutorial – Part 2

Apache Kafka Interview Q&As

1. Apache Kafka Interview Q&As

About
Latest Posts

Arulkumaran Kumaraswamipillai

In 1999, I pivoted from mechanical engineering to become a self-taught Java expert. Over the past 19+ years, I have built a high-demand contracting career, delivering mission-critical solutions for 13+ top-tier organisations. With a proven track record that has generated over 130+ job offers, I help enterprises solve complex data challenges at scale.

Author of the book "Java/J2EE job interview companion", which sold 35,000+ copies on amazon.com & superseded by this site with 2000+ registered users.

Amazon.com profile | LinkedIn | LinkedIn Group | YouTube

Email: java-interview@hotmail.com

Latest posts by Arulkumaran Kumaraswamipillai (see all)

How would you go about.. building an ETL pipeline to handle 3k TPS (i.e transactions per second) - April 17, 2026
00b: Python variable scopes, assigning types optionally and execution order interview Q&As - April 11, 2026
How to prepare for the Java job interviews? - January 3, 2026

Read Amazon reviews ➔ Read Goodreads reviews ➔

The contents in this Java-Success are copyrighted and from EmpoweringTech pty ltd. The EmpoweringTech pty ltd has the right to correct or enhance the current content without any prior notice. These are general advice only, and one needs to take his/her own circumstances into consideration. The EmpoweringTech pty ltd will not be held liable for any damages caused or alleged to be caused either directly or indirectly by these materials and resources. Any trademarked names or labels used in this blog remain the property of their respective trademark owners. Links to external sites do not imply endorsement of the linked-to sites. Privacy Policy

Responsive WordPress Theme powered by CyberChimps

Top

Categories

01: Apache Kafka example with Java – getting started tutorial

Kafka services on Docker

Java Project

Java Consumer

Produce a message from akHQ.io UI

Java code ConsumerApp.java consumes the message

Bring down the Docker container

More Apache Kafka Examples

Apache Kafka Interview Q&As

300+ Java Interview FAQs

300 + Big Data Interview FAQs

16+ Java Tech Key Areas

10+ Companion Techs Q&As

300+ Java Interview Q&As

Tutorials on Java & Big Data

50+ Free Java & Big Data Interview Q&As

Disclaimer