Introducing d-engine: Distributed Coordination Without the Complexity
Distributed coordination is one of those problems that every serious infrastructure project eventually has to face. Whether you need leader election, distributed locks, configuration management, or service discovery — at some point you reach for a tool like etcd or ZooKeeper.
They work. They’re battle-tested. But they also come with a price: operational complexity, resource overhead, and an assumption that you have a dedicated team to keep them healthy.
We built d-engine to challenge that assumption.
What is d-engine?
Section titled “What is d-engine?”d-engine is a distributed coordination engine built in Rust. It implements the Raft consensus protocol to provide strong consistency guarantees across a cluster of nodes.
The goal is simple: make distributed coordination accessible — cheap to run on modest hardware, straightforward to embed in your own systems, and honest about what it does and doesn’t do.
Why build another one?
Section titled “Why build another one?”Fair question. There are already excellent projects in this space, and we learned a lot from reading their code and documentation.
What we kept running into, especially in smaller-scale use cases, was not a lack of capability — it was the cost of adoption and operation:
- Operational overhead: Running a separate coordination system often means dedicated nodes, careful monitoring, and more operational knowledge than smaller teams want to carry.
- Embedding complexity: If you want coordination to live inside your own service, the path from “I need Raft” to “I have something I can actually ship” is still quite long.
- Learning curve: Raft itself is understandable, but production implementations often add enough layers that the core protocol becomes harder to see and reason about.
d-engine is our attempt to narrow that gap. Our goals are modest:
- Cheap to run — lightweight enough for modest setups.
- Simple to integrate — a small API surface and minimal ceremony to get started.
What we’ve built so far
Section titled “What we’ve built so far”The core Raft implementation is in place:
- Leader election and log replication
- Log compaction with snapshots
- Membership changes
- A key-value store on top as the reference application
We follow established Raft best practices and reference the implementations in etcd, openraft, and TiKV wherever their solutions are demonstrably better than ours.
Follow along
Section titled “Follow along”This blog is where we’ll document the engineering decisions behind these projects — not just what we built, but why, what we tried first, and where we got it wrong.
If distributed systems, Rust, and consensus protocols are your kind of thing:
We’re building in the open. Feedback and contributions are welcome.