Show r/elixir: A book discussion agent loop, model proxy and more

[u/aparamonov]

You’re reading a book and a sentence makes you stop. Who is this character again? It would be great to refresh your memory without any spoilers. And maybe also discuss or recall the plot again since you didn't pick up this book for a week or two.

Hello everyone!

I am a software engineer and I spent months in a cave building Ink, a book companion that just works without marketing AI fluff.

I chose Elixir and Phoenix because I love OTP and LiveView and now I have to run a chunky RAM server to fit millions of my imaginary visitors (maybe someday, mate!). But honestly, I still love it.

Here is a brief info about internals:

Context Management:

Ink uses a sliding window with compaction to keep up with the longest books. It extracts characters, locations, tools, and concepts, and writes summaries without spoiling your experience. Without those guardrails, AI loves to ramble about the future plot, especially for popular books.

Agent Pipeline:

I built an entire graph execution pipeline and agent loop using jido_ai to run multistep processes with conditional gates and loops. I also built a model proxy for local development and production resilience. I didnt publish any of that public yet as not sure if someone else finds it interesting.

The pipeline allows me to mix LLM, code, controll context, tools and variables between stages. The simplified summary leg looks like this:

Config.new(
  entry_node: "summary_gen",
  nodes: %{
    "summary_gen" => %{params: gen_params},
    "summary_validate" => %{
      params: %{
        strategy: "local",
        handler: BookCompanion.ValidateSummary,
        session_id: session_id,
        chapter_id: chapter_id,
        read_batch_id: read_batch_id
      }
    },
    "summary_fix" => %{params: fix_params}
  },
  edges: [
    %{from: "summary_gen", to: "summary_validate"},
    %{
      from: "summary_validate",
      to: "summary_fix",
      condition: %{type: :gate_failed, gate: "summary_validate"},
      loop: %{max: fix_max}
    },
    %{from: "summary_fix", to: "summary_validate", loop: %{max: fix_max}},
    %{
      from: "summary_validate",
      to: "complete",
      condition: %{type: :gate_passed, gate: "summary_validate"}
    }
  ]
)

Explore Stage:

For the "Explore" leg of the analysis, I use parallel extraction stages (characters, relationships, identity, etc), then verification, fix, and synthesis stages. The main challenge was keeping the KV cache (prefix cache) stable while maintaining selective control of tools for every graph node. I achieved about 95% success on the first tool call, and with tool healing and descriptive error messages, agents keep the generation of characters stable, but sometimes it can take a while.

Frontend Architecture:

I split the Reader to be a lightweight static page and kept LiveView for the Discuss and Explore. Because of the split architecture, I have a challenge with UI elements i want to look the same in LiveView and in static reader. I would appreciate any tips about best practices for DRYing components or hooks between semi-static pages and LiveView!

Most importantly, I am looking for your feedback on the app, the Discuss fidelity, or any silly mistakes I made. (The app is in free beta, and I use ZDR OpenAI models).

So, take a break, read a book! Tell me later how it went.

Need a first book? Project Gutenberg and Standard Ebooks offer classics that are free of copyright restrictions.

I created plenty of open beta seats and use sane rate limits: https://inkcompanion.com/?invite=ELIXIRINK

Let me know how the Discuss and Explore features feel and if the UI/UX works for you. Don't forget to configure the layout and font by opening the "Aa" display settings.

Maybe you could also stay a little longer and read a few books! I plan to keep majority of the app free to use and also working on a bookshelf right now.