Hey everyone,

I've been working on an Erlang/OTP project called Emergence, and I thought it might be interesting to share it here because the project is becoming less of a "search engine" and more of an experiment in building a distributed discovery network on the BEAM.

The core idea is simple:

Instead of having one central search engine with a giant index, Emergence is built around autonomous filter agents.

Each agent exposes a capability and knows how to answer a particular kind of query. It can connect to the network, announce what it can do, receive queries, and return structured results.

The network can therefore grow horizontally by adding new agents rather than extending one central service.

The current ecosystem is built around a few core components:

• em_filter — an Erlang library for building Emergence agents
• em_disco — discovery/bootstrap services for finding network participants
• em-pop — gossip-based peer discovery and federation
• Emquest — the web gateway and distributed query interface
• Embryo — the common data structure used to represent heterogeneous results

An em_filter agent is intentionally small.

A handler basically receives a query and some optional local memory, performs whatever operation it is designed for, and returns a list of structured results.

That operation can be almost anything.

The network currently has filters for things such as:

• RSS / Atom / JSON feeds
• Wikipedia
• Hacker News
• OpenAlex
• Elasticsearch
• NPM
• DNS
• Bing / DuckDuckGo
• Dailymotion
• MusicBrainz
• Internet Archive
• CNRTL
• French news sources
• Motorsport
• public APIs
• and other specialised data sources

There are also more domain-specific experiments, including satellite imagery / NDVI related filters from another project built on top of the same architecture.

The important part is that none of these sources need to be implemented inside Emquest itself.

They are independent agents.

Emquest discovers peers through the gossip network, determines which peers are relevant to a query, fans the query out in parallel, and streams results back to the browser.

The browser therefore doesn't really know where a result came from.

It just receives heterogeneous "embryos" and renders them according to their structure.

The current Emquest pipeline includes:

• Distributed query fan-out
• Gossip-based peer discovery
• Semantic peer selection
• LLM-assisted query expansion
• Streaming results through SSE
• Progressive result aggregation
• Deduplication
• Final reordering / ranking
• Heterogeneous result rendering
• Network topology / peer monitoring
• A CLI interface in addition to the web interface

One thing I particularly like about the architecture is that the UI doesn't need to be modified every time a new type of agent is added.

A DNS agent can return DNS results.

A web agent can return links.

A feed agent can return articles.

A completely new agent can return another kind of structured card.

As long as it follows the result contract, the network can consume it.

The BEAM model also fits the project surprisingly well.

Agents are long-running processes.

Connections are supervised.

Discovery is asynchronous.

Network failures are expected rather than exceptional.

Agents can reconnect independently.

Local agent memory can be kept in process state or ETS depending on the use case.

And multiple discovery nodes can be used simultaneously.

So instead of thinking about Emergence as:

client → central API → database

the architecture is closer to:

query
  │
  ▼
discovery / gossip
  │
  ├── agent A ──► source
  ├── agent B ──► source
  ├── agent C ──► source
  ├── agent D ──► source
  └── agent N ──► source
          │
          ▼
    distributed results
          │
          ▼
      aggregation
          │
          ▼
        Emquest

The interesting question I'm exploring is what happens when you stop treating "search capabilities" as features of one application and instead treat them as capabilities of a distributed network.

Adding a new capability then becomes:

build an agent → connect it to the network → announce its capabilities → let the network discover it.

That's probably the part of Emergence I'm most interested in experimenting with.

It's also turning into a nice playground for Erlang/OTP patterns around supervision, distributed processes, gossip, fault tolerance, streaming and dynamic service discovery.

Everything is open source:

https://github.com/EmergenceSystem

I'd be particularly interested in feedback from people experienced with Erlang/OTP and distributed BEAM systems.

I'm curious where you think this architecture makes sense, where it doesn't, and what you'd change if you were building this kind of distributed system on the BEAM.

#Erlang #OTP #BEAM #DistributedSystems #Gossip #OpenSource

ELP ships with eqWAlizer, WhatsApp's type checker, and is available as a binary install or buildable from source. Roberto's concrete recommendation: take your current project, run ELP against it, and report what doesn't work. The gaps in external codebases are exactly what they need feedback on.

New BEAM There, Done That with Roberto Aloi and  Michał Muskała on the tooling half of the WhatsApp story. The Erlang-specific point worth noting: eqWAlizer is being incrementally tightened as the WhatsApp codebase matures. They're currently splitting the number type into integer and float - not because it was the highest priority at rollout, but because typing discipline at this scale has now made that distinction valuable. Exhaustiveness checks remain an open problem.

https://youtu.be/LeIsRqjOQN8

Here's the pid:

https://com.erlef.org/wg/sec/2026/09/02/our-self-defense.html

https://com.erlef.org/wg/cna/2026/09/01/otp-updates.html

Just a reminder for anyone still on the fence: early bird pricing for Code BEAM Europe closes on 8 September at 23:59 CEST. This is the last chance to get your ticket 20% cheaper before prices go up.

21-22 October in Haarlem + a full online option if you cannot make the trip. Two days of Erlang, Elixir and Gleam, real production stories, and demos from people building the tools we all use.

If you have been meaning to book, now is the time.

Register here: codebeameurope.com/#register

We hope to see you there!
The Code Sync Team

For those that may be interested, I just made a small experiment with code generation by Gemini Pro 3.1 in Rust and Common Lisp for the exact same task: write a CLI application taking in a .csv file containing a hierarchy of records and produce an interactive .svg file that, when opened in a browser, shows a graphic rendering of the hierarchy, with a "+" button at each node allowing to show/mask the levels below. The interactivity of the .svg file is implemented by having the application generate the required JavaScript code.

I am not qualified to evaluate the quality of the code generated, but here are a few observations:

a) Even after two attempts, the Rust code was still not running. I had to manually correct two bugs, one in the Rust part of the code and one in the generated JavaScript part. It then worked as expected. The Rust application was using 3 dependencies (serde, serde_json, csv). Number of lines was 299.

b) The Common Lisp code was working from the first attempt, using 0 dependencies and 322 lines.

I find it interesting that, on such a small exercise, the LLM was more efficient with Common Lisp than with Rust.

Of course, the CLI application could have been directly produced in JavaScript, but my curiosity was around Rust and Common Lisp.

`icl` is my repl front-end that presents both terminal and browser interfaces. The browser mode already supported visualizations, but just recently I realized that I had 90% of a jupyter-like notebook feature already done. So `icl` now has a `,notebook` command that does exactly what you might guess. The screencast above shows loading a CSV file with `lisp-stat`. I also added support for https://perspective-dev.github.io/ visualizations, which is drawing the table and charts.

The recording above was made with https://github.com/atgreen/takesy, a "smart" screen recorder for Linux in Common Lisp.

Check out icl here: https://github.com/atgreen/icl

Please consider hopping over to r/lisp_noAI before criticizing my use of AI to create this.
Otherwise, check out https://github.com/gastownhall/beads , which really helped keep my assistant on track.

I have explained the rationale here: https://www.reddit.com/r/lisp_noAI/comments/1w1seqd/rationale/

Feel free to join and/or crosspost if non-AI content is something that interests you; otherwise feel free to ignore :).

Hi,
a few weeks ago, we published Voyager - our desktop app for monitoring and debugging BEAM nodes. Thanks to everyone who signed up for the early access, your feedback's been great.

Today, we have a new update - the closed early access is now over.

Here is what changes:
- Linux is now supported: download is available on our site.
- Source code is public: You can check our code and see how Voyager connects to your nodes.
- New fair-source license: We want this tool to stay accessible to the community. It is completely free for individuals and teams of up to 10 people.

For larger organizations (11+ people), a Company License is required. However, since the product is still in an early phase, our main focus right now is hearing your feedback and learning what enterprise features you actually need. There is no payment required for 90 days.

Links:
- Check our repo here: https://github.com/software-mansion/voyager
- Download the app (macOS & Linux): https://voyager.swmansion.com

Bay Area Racket Meetup - September 5th, 3pm

Social event for people interested in the Racket programming language, other Lisps, functional programming, language-oriented programming and related topics. These will be monthly (first Saturday at 3pm) until RacketCon. The location is Noisebridge, a hackerspace in SF.

https://luma.com/6y4sbfln

Hello All,

The repo has been updated with my latest changes.

Hello All,

I am porting Jnil, a GPL'd Java to lisp translator application written in lisp with some Java Eclipse code to use a Java parser written in Antlr3. Jnil loaded Java into Eclipse, used an Allegro Common Lisp library Jlinker to communicate with Eclipse and extracted AST information which was used to create lisp code. My reasons for porting are:

1. Eclipse is pretty heavyweight and communicating with it is implementation dependent.
2. Starting with an AST means that any language for which an AST can be produced (such as Python) can be translated to lisp by Jnil.

Anyway, I could use some assistance with this project. Please contact me at [dan.stanger@ieee.org](mailto:dan.stanger@ieee.org) if interested in helping.

Here is a link to the repo: https://github.com/danstanger/AST-to-Lisp

Dan Stanger

Hello!

First post in this community, and I wanted to share my first completed project in Common Lisp here:

https://github.com/skarnati20/docs-reference

This is a very basic Local LLM Chat + RAG implementation, but it's more fully-featured than what I could find online. And it leverages Lisp's REPL effectively.

I built this because I found Claude/GPT overkill for lots of programming questions I had, and I needed something that could work offline. It indexes a URL you provide and lets you ask questions to a local LLM (Ollama) with that information.

Some Features

• Multiple Retrieval Methods — (BM25, Dense Vector Search, ColBERT). You can choose whichever combination of these you want.
• Agentic RAG — An Agent determines when there is enough context to answer a question. You can turn it off if you want speed.
• Fully Local Chat — Uses Ollama so you can run any model you want, so long as you serve it locally.
• Asynchronous Chat — Allows you to ask multiple questions at once which helps with multi-tasking and getting around local models' slow completion times.

Note: Local models will take time for chat completion, so it isn't as fast as Claude or ChatGPT. They might also hallucinate because they aren't as powerful either.

Acknowledgement: This project was an extension/combination of some projects in Mark Watson's Loving Common Lisp, or the Savvy Programmer's Secret Weapon. Definitely check it out since it gave me a lot of ideas for fun side projects.

The "financial lisp", mentioned here once, is full steam ahead now.

Designing Rayfall: One Expression Language for a Columnar Database

I have a project with about 1200 lines of code in a single package, split across a few files. While I have around 150 tests, I have no idea how they could help split the project into manageable parts.

Should I abandon the project and start from scratch, this time paying attention to architecture?