gitwtfhub

wtf is poseidon?

airbnb/poseidon — explained in plain English

Analysis updated 2026-08-04 · repo last pushed 2015-08-29

5RubyAudience · developerComplexity · 2/5DormantSetup · moderate

TL;DR

A Ruby library that lets applications send and receive messages through Apache Kafka, a system for moving large amounts of data between different parts of a software platform.

Mindmap

mindmap
  root((repo))
    What it does
      Sends messages to Kafka
      Receives messages from Kafka
      Batches messages by topic
      Optional Snappy compression
    Tech stack
      Ruby
      Apache Kafka
      Snappy compression
    Use cases
      Log user actions
      Send orders to fulfillment
      Service-to-service messaging
    Audience
      Ruby developers
      Data-intensive app builders
    Maturity
      Alpha software
      Not production-ready

Code map

Detail Auto

An interactive map of this repo's files and how they connect — its source is parsed live in your browser. Click Visualize to build it.

filefunction / class

Why would anyone build with this?

REASON 1

Send user activity logs from a Ruby web app to a separate analytics service for report building.

REASON 2

Forward order details from an e-commerce platform to a fulfillment system without making the customer wait.

REASON 3

Build a data pipeline where multiple Ruby services communicate asynchronously through Kafka topics.

REASON 4

Process high-volume message streams with optional Snappy compression to reduce message size.

What's in the stack?

RubyApache KafkaSnappy

How it stacks up

airbnb/poseidoncocoapods/cocoapods-podfile_infofastlane/nightly
Stars555
LanguageRubyRubyRuby
Last pushed2015-08-292015-02-252018-12-11
MaintenanceDormantDormantDormant
Setup difficultymoderateeasymoderate
Complexity2/52/52/5
Audiencedeveloperdeveloperdeveloper

Figures from each repo's GitHub metadata at analysis time.

How do you spin it up?

Difficulty · moderate Time to first run · 30min

Requires a running Apache Kafka server (version 0.8+) and Ruby 1.9.3 or higher, plus the project is alpha software so proceed with caution.

No license information provided in the repository documentation.

Wtf does this do

Poseidon is a Ruby library that lets applications send and receive messages through Apache Kafka, a popular system for moving large amounts of data between different parts of a software platform. Think of it as a post office: your app can drop off messages and other applications can pick them up later, all flowing through Kafka as the central hub. The tool works in two main directions. As a producer, your Ruby application connects to a Kafka server and sends batches of messages tagged with topics, like routing envelopes to different departments. As a consumer, your app connects and continuously fetches new messages as they arrive, processing each one in turn. The README includes code samples showing both patterns, and the setup is straightforward enough that a developer could get a basic pipeline running quickly. A developer building data-intensive applications in Ruby would reach for this when they need different services to communicate without waiting on each other. For example, a web app might log every user action by sending a message through this tool, and a separate analytics service would pick up those events later to build reports. An e-commerce platform could use it to send order details to a fulfillment system without making the customer wait for that handoff to complete. The project explicitly labels itself as alpha software and not production-ready, meaning organizations relying on it for critical workloads should proceed with caution. It requires Ruby 1.9.3 or higher and Kafka version 0.8 or above. There is optional support for Snappy compression, which reduces the size of messages traveling through the pipeline, useful when volume is high.

Yoink these prompts

Prompt 1
I want to use the Poseidon Ruby library to send messages to Apache Kafka. Show me how to set up a producer that sends batched messages tagged with a topic to a Kafka server.
Prompt 2
Help me set up a Poseidon consumer in Ruby that connects to a Kafka server and continuously fetches new messages as they arrive, processing each one in turn.
Prompt 3
I have Ruby 1.9.3 and Kafka 0.8 running. Walk me through installing Poseidon and building a basic message pipeline with code samples for both producer and consumer patterns.
Prompt 4
Show me how to enable Snappy compression with Poseidon when sending high volumes of messages through Kafka to reduce message size.

Frequently asked questions

wtf is poseidon?

A Ruby library that lets applications send and receive messages through Apache Kafka, a system for moving large amounts of data between different parts of a software platform.

What language is poseidon written in?

Mainly Ruby. The stack also includes Ruby, Apache Kafka, Snappy.

Is poseidon actively maintained?

Dormant — no commits in 2+ years (last push 2015-08-29).

What license does poseidon use?

No license information provided in the repository documentation.

How hard is poseidon to set up?

Setup difficulty is rated moderate, with roughly 30min to a first successful run.

Who is poseidon for?

Mainly developer.

View the repo → Decode another repo

This repo across BitVibe Labs

Don't trust strangers blindly. Verify against the repo.