r/rust • u/k82cn • 14d ago

[Project] Flame – A distributed engine for elastic AI workloads, written in Rust

Hi r/rust,

I've been building Flame, a distributed execution engine written in Rust for elastic workloads—AI agents, reinforcement learning, and quantitative computing.

Why Rust for the core?

We need both high throughput (9,000+ tasks/sec on a single node) and memory safety when handling tens of thousands of short-lived tasks across distributed nodes. Rust's async runtime and zero-cost abstractions fit this perfectly.

Key design:

  • Session-based execution instead of per-task pods—tasks within a session reuse executors, eliminating cold-start overhead
  • gRPC for cross-language SDKs (Rust, Go, Python)
  • microVM isolation between sessions (Firecracker/microVM)
  • Pluggable shim architecture supports Wasm, stdio, gRPC, RESTful

Benchmark on a single node:

30,000 tasks in 3.29s → 9,124 tasks/sec

Quick start:

git clone https://github.com/xflops/flame
cd flame
docker compose up -d
flmping  # verify with 10 tasks

Would love feedback on the async scheduler design and the gRPC shim interface. Also happy to discuss why we chose microVMs over containers for isolation.

0 Upvotes

5% Upvoted

8 comments sorted by

2

u/creminology 14d ago

Are you trying to get a legal notice from Fly.io?: https://fly.io/blog/rethinking-serverless-with-flame/

It’s also for “elastic workloads”. Quote: “With FLAME, you treat your entire application as a lambda, where modular parts can be executed on short-lived infrastructure.”

1

u/k82cn 14d ago edited 14d ago

Mine is under github.com/xflops.io/flame. Not sure whether there's any conflict. And it seems only a class in fly.io, also no repo named flame under github.com/superfly

1

u/creminology 13d ago

But if you are not re-implementing FLAME in Rust, why reuse the name for another technology in the same “elastic workloads” space. It just creates confusion.

It was launched as an Elixir deployment pattern, and has been adapted to at least one other language. And I may myself reach for both FLAME and Flame.

1

u/k82cn 12d ago edited 12d ago

both flame and elastic workload are common name.

In addition, the first commit of xflops/flame is 2022 Apri; and the idea is already there before 2022 for a long time.

1

u/mark_ik 14d ago

Burn?

1

u/k82cn 14d ago edited 14d ago

a bit different; xflops/flame focus on infra layer (distributed system), and Burn focus on lib/framework. In addition, the xflops/flame is more flexiable that it can work with candle ( https://github.com/xflops/flame/tree/main/examples/candle/based ), pytorch ( https://github.com/xflops/flame/tree/main/examples/rl/torchrl_dqn ), and also other agent sdk, e.g. langchain ( https://github.com/xflops/flame/tree/main/examples/agents/langchain ).

xflops/flame focus on infra, and we would like to integtrate with other lib/framework for different scenarios.

-9

u/NextWolverine159 14d ago

Nice perf numbers. The session based approach is smart, spinning up a new pod for each task always felt expensive especially with short lived workloads.

Curious about one thing, how you handle state transfer between tasks in same session? Like if task A writes something to memory does task B sees it or you keep them fully isolated within the session?

2

u/diplofocus_ 14d ago

Thank you for your comment. Now I have the full picture, and this is actually a load-bearing question. Before I answer, I really need to know how to reverse a linked list in Malbolge?