January 11, 2011

Apache Kafka open-sourced

LinkedIn open-sourced KafkaApache Kafka — a distributed event streaming platform for high-throughput real-time data pipelines. on January 11, 2011 — a distributed, persistent log for high-throughput event streaming.

What it was for

Apache KafkaKafkaApache Kafka — a distributed event streaming platform for high-throughput real-time data pipelines. appends events to partitioned topics that many consumers read at their own pace — ideal for activity tracking, log aggregation, and stream processing. It became the backbone of real-time data pipelines at LinkedIn, Uber, and Netflix before managed cloud offerings. Example: clickstream events land in a `pageviews` topic; analytics and recommendation jobs consume them independently.

Why it's here

KafkaApache Kafka — a distributed event streaming platform for high-throughput real-time data pipelines. open-sourced LinkedIn's solution for billions of daily activity events.

Why it mattered

It defined the modern event-streaming architecture that powers analytics and microservice decoupling.

What it solved

Traditional message queues could not retain and replay massive event histories at log scale.

Media

  • Apache Kafka
    ImageApache Kafka

    ™/®Apache Software Foundation, Apache License 2.0, via Wikimedia Commons

Related