Run a small fault tolerant message queue for a homegrown app instead of adopting a heavier system like Kafka.
Study a working example of Raft consensus and segment based leadership to learn how distributed logs are built.
Build a learning project around distributed systems ideas like leader election and write fencing.
| nubskr/walrus | efforg/apkeep | orhun/ratty | |
|---|---|---|---|
| Stars | 1,887 | 1,950 | 2,082 |
| Language | Rust | Rust | Rust |
| Last pushed | — | 2026-05-04 | — |
| Maintenance | — | Maintained | — |
| Setup difficulty | moderate | moderate | moderate |
| Complexity | 5/5 | 2/5 | 3/5 |
| Audience | developer | researcher | developer |
Figures from each repo's GitHub metadata at analysis time.
Requires building from source with Rust and using the provided Makefile to bootstrap a local cluster.
Walrus is a distributed message streaming system built in Rust. It lets multiple computers work together as a cluster to reliably store and pass around streams of messages, in a similar spirit to systems like Kafka, but built from scratch by its author. The system splits each topic, a named stream of messages, into segments, and each segment has one designated leader node responsible for writing to it. As segments fill up, leadership rotates automatically so that the workload spreads evenly across the cluster. This means the cluster balances load on its own, without an operator manually assigning work to nodes. To stay reliable even if a node fails, Walrus uses a consensus protocol called Raft to track important cluster information, such as which node currently leads which segment. This tracking information is kept in sync across every node, so if one node goes down, the rest still know the current state and can carry on without losing data. Raft here manages only this metadata, not the actual message contents, which keeps the write path fast. Client programs that produce or consume messages can connect to any node in the cluster using a simple text based protocol over a plain network connection. The cluster automatically forwards each request to whichever node currently handles it, so client code never needs to know the cluster's internal layout. Walrus ships with a command line tool for creating topics, sending messages, reading messages, and checking cluster state, which makes it easy to try the system out. A three node cluster can be started locally with a single command to see how it behaves. The project also includes a formal mathematical specification, written in TLA+, describing exactly how the system should behave, along with checks confirming properties like never losing message order and never letting two nodes write to the same segment at once. The project is released under the MIT license and is written entirely in Rust, targeting Linux systems where it can use fast input and output features for storage.
Walrus is a Rust based distributed message streaming system that splits topics into segments, rotates segment leadership automatically, and uses Raft consensus to keep the cluster reliable if nodes fail.
Mainly Rust. The stack also includes Rust, Raft, TCP.
You can use, modify, and distribute this software freely, including for commercial purposes, as long as you keep the original copyright notice.
Setup difficulty is rated moderate, with roughly 30min to a first successful run.
Mainly developer.
This repo across BitVibe Labs
Don't trust strangers blindly. Verify against the repo.