Kafka Interview Questions & Answers (for .NET Core)

-

1) What is Kafka?

Kafka is a distributed event streaming platform used to publish, store, and consume events/messages at very high scale.

In .NET:

  • We publish events using Producer
  • We consume events using Consumer
  • Kafka stores messages in topics (durable)

2) When do you choose Kafka?

Choose Kafka when you need:

✅ High throughput (millions of messages/day)
✅ Event-driven microservices
✅ Durable event storage (replayable)
✅ Multiple consumers for same event
✅ Real-time processing
✅ Loose coupling between services

Example:

  • OrderCreated event → Inventory + Payment + Email services consume it.

3) When NOT to choose Kafka?

Avoid Kafka if:

❌ You just need simple queue (like small job processing)
❌ Low volume messaging
❌ You need strict FIFO globally
❌ You want easy setup (Kafka needs infra & monitoring)

In that case RabbitMQ / Azure Service Bus is simpler.

4) Kafka vs RabbitMQ (Interview must-know)

Kafka

RabbitMQ

Event streaming

Message queue

Stores messages long time

Removes after consume

Replay possible

Replay not default

Best for analytics, events

Best for command queue

High throughput

Medium throughput

5) How do you implement Kafka in ASP.NET Core?

Typical structure:

✅ API receives request
✅ Save data in SQL Server using EF Core
✅ Publish Kafka event
✅ Consumer service processes event asynchronously

6) Which .NET library is used for Kafka?

Most common:

✅ Confluent.Kafka

Because it is:

  • Officially maintained
  • Production-ready
  • High performance

7) How do you implement Kafka Producer in .NET Core?

In .NET Core:

  • Configure Producer using ProducerConfig
  • Use IProducer<string, string>
  • Serialize payload (JSON)
  • Publish to topic using ProduceAsync()

8) How do you implement Kafka Consumer in .NET Core?

Enterprise way:

  • Create a Worker Service OR BackgroundService
  • Use ConsumerConfig
  • Subscribe to topic
  • Poll messages continuously
  • Process and commit offset

9) What is a Kafka Topic?

A topic is like a table / stream name where events are stored.

Example:

  • order-created
  • payment-success
  • invoice-generated

10) What is a Partition?

Partitions are how Kafka scales.

A topic is divided into partitions.
Each partition is an ordered log.

Benefits:
✅ Parallel consumption
✅ High throughput

11) What is Consumer Group?

A consumer group is a set of consumers sharing the load.

  • If topic has 5 partitions
  • And group has 5 consumers
    ➡️ each consumer gets 1 partition

12) What happens if consumers are more than partitions?

Extra consumers will be idle.

Example:
Partitions = 3
Consumers in group = 5
➡️ Only 3 will work, 2 will sit idle.

13) What is Offset?

Offset is the position of message inside a partition.

Kafka stores:

  • Partition number
  • Offset number

Consumers commit offset after successful processing.

14) What is Kafka Message Key?

Key decides partition routing.

If key = OrderId
➡️ all messages for same OrderId go to same partition
➡️ order is maintained per OrderId

15) How do you ensure ordering in Kafka?

Kafka guarantees ordering only:
✅ inside a partition

So to maintain order:

  • Use message key like OrderId
  • Ensure same key goes to same partition

🔥 Enterprise Kafka Questions

16) How do you handle failures in Kafka consumer?

Enterprise answer:

  1. Retry processing (in code)
  2. If still failing → send to Dead Letter Topic (DLT)
  3. Log with correlation id
  4. Do not commit offset until success

17) What is Dead Letter Topic (DLT)?

A separate Kafka topic where failed messages go.

Example:

  • Main topic: order-created
  • DLT topic: order-created-dlt

This prevents infinite re-processing loops.

18) How do you implement Retry?

In .NET:

  • Retry inside consumer using Polly OR custom retry logic
  • Retry count included in message headers

If retry exhausted:
➡️ publish to DLT

19) How do you implement Outbox Pattern with Kafka?

🔥 This is super important.

Problem:

If you do:

  1. Save order in SQL
  2. Publish Kafka event

If Kafka publish fails:
❌ DB saved but event not published
➡️ system inconsistent

Solution:

Outbox Pattern:

  1. Save order in SQL
  2. Save event in Outbox table (same transaction)
  3. Background worker reads Outbox table
  4. Publishes to Kafka
  5. Marks Outbox row as processed

✅ Guaranteed delivery
✅ Transaction-safe

20) What is the Outbox Table structure?

Typical columns:

  • Id (GUID)
  • EventType
  • Payload (JSON)
  • CreatedAt
  • ProcessedAt
  • RetryCount
  • Status

21) How does EF Core Outbox work?

Use EF Core transaction:

BEGIN TRANSACTION

Insert Order

Insert OutboxMessage

COMMIT

Kafka publishing happens later by background job.

22) What is Idempotency in Kafka?

Idempotency means:
✅ processing the same event twice should not create duplicates.

Because Kafka can deliver duplicates in some scenarios.

23) How do you implement Idempotency in .NET?

Best methods:

Option A (DB Unique constraint)

  • Store ProcessedMessageId table
  • If message id exists → skip

Option B (Upsert based on key)

  • Use unique OrderId
  • If already processed → ignore

🔐 Kafka Security Interview Questions

24) How do you secure Kafka?

Kafka security includes:

1. Encryption in transit

✅ SSL / TLS

2. Authentication

✅ SASL (SCRAM, OAuth, Kerberos)

3. Authorization

✅ ACLs (topic-level permissions)

25) What is SSL in Kafka?

SSL encrypts communication between:

  • Producer → Broker
  • Consumer → Broker

So no one can sniff data.

26) What is SASL?

SASL is authentication mechanism.

Common:

  • SASL/SCRAM-SHA-256
  • SASL/SCRAM-SHA-512

27) What are Kafka ACLs?

ACL means:
who can do what on which topic.

Example:

  • OrderService can WRITE to order-created
  • InventoryService can READ from order-created

28) How do you secure Kafka in .NET (Confluent.Kafka)?

You configure:

  • SecurityProtocol
  • SaslMechanism
  • SaslUsername
  • SaslPassword
  • SSL certificates if needed

🏢 Production + Architecture Questions

29) How do you monitor Kafka?

Enterprise monitoring:

  • Consumer lag
  • Broker health
  • Partition under-replicated
  • Message throughput
  • Error rates

Tools:

  • Prometheus + Grafana
  • Confluent Control Center
  • Azure Monitor (if cloud)

30) What is Consumer Lag?

Lag means:

Messages produced - messages consumed

If lag increases:
❌ consumers are slow
❌ processing bottleneck

31) How do you scale Kafka consumers?

Scale by:

  • Increase partitions
  • Increase consumers in same group
  • Optimize processing

32) What is exactly-once in Kafka?

Kafka supports exactly-once in certain cases (transactions).

But in microservices:
Most common practical approach is:
✅ At-least-once + Idempotency

33) What delivery guarantee does Kafka provide?

Kafka provides:

  • At-most-once (if commit before process)
  • At-least-once (commit after process) ✅ most common
  • Exactly-once (complex + transactional)

34) How do you handle schema changes in Kafka messages?

Enterprise answer:

  • Use Schema Registry (Avro/Protobuf)
    OR
  • Versioned JSON events

Example:

  • OrderCreatedV1
  • OrderCreatedV2

35) How do you design Kafka events?

Best practice:

✅ Use event names in past tense

  • OrderCreated
  • InvoiceGenerated
  • PaymentCompleted

Include:

  • EventId
  • CorrelationId
  • CreatedAt
  • Payload

36) How do you trace Kafka events across microservices?

Use:

  • CorrelationId
  • TraceId
  • Distributed tracing (OpenTelemetry)

Log it in every service.

🎯 Interview “Explain Implementation” Answer (Perfect)

37) Explain your Kafka implementation in your project

You can say:

In our .NET microservices, we use Kafka for event-driven communication.
When an order is created, we store it in SQL Server using EF Core and also write an Outbox event in the same transaction.
A background worker publishes the outbox events to Kafka using Confluent.Kafka producer.
Downstream services consume events using hosted background services with consumer groups, and we commit offsets only after successful processing.
We also implemented retry + dead letter topic and idempotency to handle duplicates.

🔥 This answer is enterprise-level.

✅ Most Asked Kafka Questions (Quick List)

  • Why Kafka is fast?
  • What is partition?
  • What is consumer group?
  • What is offset?
  • How ordering works?
  • Kafka vs RabbitMQ?
  • What is outbox?
  • What is DLT?
  • How to handle retries?
  • How to secure Kafka?
  • What is consumer lag?

 

Comments

Post a Comment

Popular posts from this blog

Debouncing & Throttling in RxJS: Optimizing API Calls and User Interactions

-
RxJS Debounce vs Throttle in Angular – Best Practices & Examples What is RxJS — quick primer RxJS (Reactive Extensions for JavaScript) provides Observables and operators to model and manipulate streams of asynchronous events — from keystrokes and HTTP requests to timers and WebSocket messages. 💡 Why this matters: Operators like debounceTime and throttleTime help you control event frequency, reducing wasted CPU and network work while improving UX. 1. Debouncing — emit after silence Debouncing delays emitting a value until a specified period of inactivity. It's best for typeahead inputs where only the final value after the user stops typing is relevant. Live search example (production-ready) // search.component.ts import { Component, OnDestroy } from '@angular/core'; import { FormControl } from '@angular/forms'; import { Subject } from 'rxjs'; import { debounc...
Read more

Promises in Angular

-
🚀 Promises in Angular – The Complete Beginner’s Guide (with Examples) 📘 Introduction In modern Angular development, handling asynchronous operations is essential. Whether you're fetching data from an API, loading files, or wrapping browser-native APIs, Promises offer a clean and readable way to deal with one-time asynchronous operations. This guide explains: What Promises are How they compare with Observables Real-world examples using .then() and async/await When to use Promises vs Observables in Angular 📚 Table of Contents 🔍 What is a Promise? 🔁 Promise Lifecycle ✨ Key Features 📌 Syntax 💻 Example 1: Fetch 💻 Example 2: setTimeout 📊 Promises vs Observables 🎯 When to Use Promises 📦 Angular Service Example 🔄 Observable vs Promise ❓ FAQ ✅ Summary 🔍 What is a Promise? A Promise is a JavaScript object that represents the eventual success (fulfilled) or failure (rejected) of an asynchronous operation. 📌 Com...
Read more

Comprehensive Guide to C# and .NET Core OOP Concepts and Language Features

-
Comprehensive Guide to C# and .NET Core OOP Concepts Class A class is a blueprint for creating objects that encapsulate both data and behavior. Access Modifiers : public, internal, abstract, sealed control visibility and inheritance. Static Classes : Cannot be instantiated, contain only static members. Partial Classes : Split across files using partial keyword for better organization. public class Car { public string Model; public void Drive() => Console.WriteLine($"{Model} is driving."); } Analogy: A car blueprint used to build many actual cars. Benefits: Reusability, Encapsulation, Abstraction, Inheritance Object An object is an instance of a class. Managed by Garbage Collector for automatic memory cleanup. Boxing/Unboxing allows value types to be treated a...
Read more