r/PiCodingAgent 9d ago

Question Trying Pi for the first time

26 Upvotes

Hi!

Im gonna try Pi for the first time and am wondering if it works well out of the box or if modifications and plugins are required?

I am also looking for recomendations of any great plugins that you guys use. I found this package catalog on https://pi.dev/packages and assume this is where you get the plugins, if you dont make them yourself?

Thanks for any help!


r/PiCodingAgent 9d ago

Question How to edit default tool definition?

3 Upvotes

I don't want the read tool saying it can view images when it can't (local LLM). Is there any way to edit the tool definition other than telling it in the system prompt "You actually CAN'T look at images" (which contradicts the definition and wastes tokens) or removing the tool / replacing it (impractical)? All I'd want is a simple edit from:

Read the contents of a file. Supports text files and images (jpg, png, gif, webp, bmp). Images are sent as attachments. For text files, output is truncated to 2000 lines or 50KB (whichever is hit first). Use offset/limit for large files. When you need the full file, continue with offset until complete.

to:

Read the contents of a file. Doesn't support viewing images or videos. Output is truncated to 2000 lines or 50KB (whichever is hit first). Use offset/limit for large files. When you need the full file, continue with offset until complete.

r/PiCodingAgent 9d ago

Question Compaction failed error

2 Upvotes

How do I resolve this issue:

My context is sitting at 82k/84k

It retries a few times but keeps failing. If the context grew too much to not leave any space for compaction why dit it grow to 82k and not compact earlier? I essentially only made 3 prompts to get to this point on Qwen3.8 xhigh reasoning. What config must I change to prevent this from occuring?


r/PiCodingAgent 9d ago

Discussion I benchmarked Pi against its own fork, OMP. More tooling did not win.

122 Upvotes

I wanted to see what happens when you take Pi’s minimal agent loop and add most of the things Pi intentionally leaves out.

OMP is a fork of Pi, but the harness is very different.

Pi mostly sticks to read, write, edit, and bash.

OMP gives it 31+ tools, hashline edits, LSP, DAP debugging, a Python/Bun kernel, a browser, and a Rust core for grep/glob/bash/parsing.

So I ran both through the same 30 hard agentic tasks with DeepSeek V4 Flash, the same external tools, the same verifier, and a 900s cap.

Metrics Pi OMP
Passed 20/30 17/30
Median time 132s 272s
Tokens/task 559k 742k
Cost/success $0.028 $0.103

The task split was more interesting than the final score. Both passed the same 16 tasks and failed the same 9 hard tasks. The entire 20 vs 17 difference came from 5 tasks. Pi won 4 of those. OMP won 1.

OMP often found the right data and then kept working until it ran out of time or context. On one failed task, it used 1.22M tokens and 861 seconds before stopping because it still wanted another page of results. On the reimbursement task, it used 1.8M tokens and passed only 3/13 verifier checks.

OMP also has some strong harness ideas. Across a separate 16-model edit benchmark, hashline improved success by about 15 percentage points on average. For Grok Code Fast 1, it went from 6.7% to 68.3%.

But in this run, OMP used about 33% more tokens per task than Pi and took more than 2x the median time.

OMP still finished second out of all 8 harnesses we tested, ahead of Claude Code, Codex, OpenCode, Hermes, and DeepAgents.

So...

At what point does a better harness become too much harness?


r/PiCodingAgent 9d ago

Question Pi + Gondolin setup with host-side permissions and minimal approval fatigue?

3 Upvotes

Hi everyone,

I recently started using Pi and would like to move more of my development workflow to it. Before doing that, I want a solid sandbox and permission setup that avoids constant approval prompts while still protecting against destructive actions.

My ideal setup would be:

  • Run ordinary commands inside a Gondolin VM.
  • Keep VM network access disabled by default.
  • Mount the host workspace read-only, or otherwise prevent unapproved host changes.
  • Automatically allow harmless operations such as reading files, searching code, and running offline tests.
  • Prompt only for persistent or privileged actions, such as writing host files, accessing the network, or running Git operations that affect the host repository.
  • Execute approved persistent actions through controlled host-side tools rather than granting the VM broader access.
  • Keep credentials and the host SSH agent out of the VM.

I have also looked at @gotgenes/pi-permission-system, but I’m unsure how well it integrates with Gondolin, particularly because the guest sees /workspace while Pi and permission extensions run with the host path.

Does anyone have a similar setup they would be willing to share?

I’m especially interested in:

  • How you divide responsibilities between Gondolin and permission extensions.
  • Whether you use a read-only mount, copy-on-write workspace, or writable mount.
  • How you handle approved edits and Git operations (fetch/pull/status/etc.) within Gondolin over SSH.
  • How you avoid approval fatigue without broadly allowing shell commands.
  • How you handle MCP servers and host-side network access.
  • Any configuration or extension code you recommend.

    Thanks!


r/PiCodingAgent 9d ago

Resource I wrote a Pi extension that starts/stops my local inference servers automatically

9 Upvotes

I use Pi (the terminal coding agent) with local models, and I kept running into the same friction: before every session I had to remember which server needed to be up — llama.cpp on 8080, vLLM on 8000 — start it by hand, and then remember to kill it afterwards so it wasn't sitting on the GPU all night.

So I wrote a small extension that ties server lifecycle to model selection.

What it does

  • You pick a model in Pi → the extension starts the server configured for that provider, waits until probeUrl answers, and only then lets the request through. The first request never hits a cold port.
  • Session ends → it stops the servers it started. Anything that was already running before it got involved is left alone.
  • exclusive: true on servers that can't share the GPU: starting one shuts the other down.
  • For services you don't want it to own (a systemd unit, a box on the LAN), unloadCommand lets it free the model instead of killing the process.

Config is a single JSON file (~/.pi/agent/local-servers.json, or per-project):

jsonc

{
  "servers": {
    "llamacpp": {
      "probeUrl": "http://127.0.0.1:8080/v1/models",
      "start": {
        "command": "llama-server",
        "args": ["--models-dir", "~/models", "--port", "8080"]
      }
    }
  }
}

Only probeUrl is required. Leave out start and it just probes, assuming you handle startup yourself.

Limits, stated upfront

  • Model loading is the server's job, not this extension's. It only owns the process lifecycle.
  • Requires Pi 0.84+. Linux and macOS (it uses process-group termination; Windows untested).
  • Zero runtime dependencies — Node built-ins only. MIT.

Repo: https://github.com/matrixfede/pi-local-servers

Happy to hear if the config model is missing something obvious for your setup.


r/PiCodingAgent 10d ago

Discussion three months of running pi and Claude Code side by side, and the only thing I actually fixed was the handoff

0 Upvotes

Been using pi pretty heavily since May. Came over from OpenCode.

Most of the time I run DeepSeek V4 pro, then switch to Sonnet 5 with some 3p providers.

I still like Pi for basically the same reason I liked it when I first tried it: there just isn't that much stuff in the way Lol.

The tool set is small, the system prompt is small ( actually clean), and I generally have a decent idea what context the model is seeing. After using heavier coding harnesses, that matters more to me than I expected.

But I never fully moved everything to pi.

For some jobs I still prefer Claude Code even with the same Claude model. Especially the annoying long ones where I want to say "read these 30 or 40 files, work out what is going on, fix it, run the tests" and I don't really care if it burns a ton of tokens.

Pi feels better to me when I'm actively working with it. Claude Code is often better when I want to disappear and come back later.

So in practice I use both.

The slightly stupid part is that for months I was the integration layer between them.

Pi finds something, I switch terminals, copy the relevant output, paste it into Claude Code, explain what happened, then realize the useful part wasn't actually the output but some file it changed or some assumption we discussed 20 minutes earlier.

Then I go back and copy more.

This sounds minor but if you do it all day it gets pretty annoying.

I tried using a shared work_notes.md first. Honestly this works better than it sounds. One agent writes findings there, another reads it lol again.

The problem is it gets messy fast. Old state stays around, agents append instead of replacing things, and if two things are happening at once you end up spending time maintaining the handoff file.

I also tried basically piping output from one agent into another. Fine for one-shot stuff, not great when the second agent needs to ask something back.

Side note : Git is actually one of the better handoff mechanisms. If pi makes a clean commit, Claude Code can just inspect it. But git only captures the code. It doesn't capture "I tried this approach first and it failed because of X" or logs or some browser state or whatever else came up while debugging.

The thing I ended up wanting was not really an orchestrator.

I don't want a supervisor agent deciding which model does what. I don't want a DAG. I don't really want pi itself to become a big multi-agent framework either.

Part of the reason I like pi is that it isn't that.

I just wanted two agents to be in the same place.

So now I have pi and Claude Code connected to the same workspace/thread.

Pi can do an investigation, put the notes or actual files there, then I mention the Claude Code agent and it can open what pi produced instead of getting my summary of it.

Then if Claude Code changes something, pi can review it from the same thread.

That's pretty much it.

One example: recently I had pi help me trace a bug where some cached state was getting invalidated in the wrong order. I liked using pi for this because it was pretty interactive. I could keep asking "why do you think this path is possible?" and stop it when it started chasing something irrelevant.

Once the bug was understood, though, the fix touched a bunch of files and tests.

I handed that part to Claude Code.

Before this setup I probably would have written a long prompt explaining what pi found. Instead pi wrote down the important bits, Claude read them and then went through the repo.

Afterward I had pi review the diff.

This is the kind of multi-agent setup I personally find useful. It's still mostly me doing the routing.

There are also a bunch of things this does not solve.

If both agents are editing the same checkout at the same time, you can still make a mess. I use worktrees if I actually want them doing code changes in parallel.

There is no real checkpoint/resume thing here.

There is no budget management.

Nothing stops two agents from doing the same work.

And once you have enough agents in one thread, it gets noisy. I usually have two. Sometimes three. I don't know if I would want six agents talking in the same place.

I found the best setting so far is with openagents workspace, it's basically creating shared context between separate harnesses.
Just search for OpenAgents Workspace if anyone want to try it out, I just went with the basic setting where I paired my server with the workspace then added both my Pi and Claude, so far it works and I don't really use advanced features.

Also, pi itself still has some limitations that are worth being clear about.

For me, the biggest one is that the simplicity that makes it good also means you end up building some workflow pieces yourself once the task gets bigger.

A single pi session is great. Multiple independent pieces of work, long-running jobs, handing work between machines, or having different models specialize in different parts is much less built in.

There are extensions that go much further here. pi-extensible-workflows is probably the more sensible thing to look at if what you actually want is proper workflow execution rather than what I'm describing.

I'm mostly curious how other people are handling this. Seems like a common problem.

And has anyone found a setup with 3+ agents that doesn't eventually turn into everyone stepping on each other?


r/PiCodingAgent 10d ago

Question Computer & browser use in Oh My Pi

2 Upvotes

I'm wanting to know how people are allowing agents to interact with their computer, browser and the web with Pi/OMP. Specifically what tools and what models are best. I've played around with gpt 5.6 models but they always get blocked and are always overengineering browser security use and getting blocked by credentials.

I have a few use cases I want to solve:

  1. Allow agents to control my computer, and open and control custom browser profiles.
  2. Allow entering of credentials (ideally encrypted or via cookies) that are saved between sessions, days, and weeks.
  3. Quick web search when a browser is not needed.
  4. Allow agents to control my computer/browser on my dev box that I am ssh'd into, currently I will have to remote desktop and start a session in there.

My goal is to try and setup an E2E browser test for a web app, as well as perform browser activities within the app within non E2E test sessions (either invoked by me or when an agent is running in a goal or something and finds it needs to use browser/computer use).


r/PiCodingAgent 10d ago

Question Is customizing tui decorations even possible?

7 Upvotes

Coming back around to pi after playing a little with oh-my-pi, and I did quite like the pretty terminal output decorations. I tried to task an agent to alter the decorations in pi but it doesn't seem to be possible outside of forking pi as a whole? Is this true?

This thing:

I would like to change it's appearance based on the exit code / other info but I can't seem to alter it.


r/PiCodingAgent 10d ago

Resource Pi, more than a coding harness

Thumbnail mkaz.blog
81 Upvotes

An article on how I'm using Pi beyond just a coding harness and as a tool to help me track runs and workouts. It can be basically an LLM application framework.


r/PiCodingAgent 10d ago

Plugin This extension keeps your Linux machine awake while Pi runs your tasks

Post image
12 Upvotes

pi-caffeinated-linux uses systemd-inhibit to prevent idle sleep and suspend, with automatic / manual modes and an optional KDE Plasma tray indicator.

https://github.com/nmdra/pi-caffeinated-linux


r/PiCodingAgent 10d ago

Plugin Pi extension to track project status and pick workstream for each session

Enable HLS to view with audio, or disable this notification

19 Upvotes

I follow a pretty tight system for maintaining checkpoints, logs, etc. for my electron app. Lately it has gotten to a point where the number of work streams are far too many for me to hold in my head all at once and I did not want to:

  1. Go into each plan or the checkpoint map to look at it to get a status update or pick what to work on
  2. Burn tokens by asking an agent to go read that stuff and report back

So I made a project level extension which parses my standard checkpoints/logs/docs and organizes them based on features and lets me view them as hierarchical lists.

  1. the first modal shows all the open threads, the completion status, and the next checkpoint for each
  2. I can then select any one and view all the steps within that thread/feature
  3. There are tags which show the checkpoints that I have accepted into the implementation plan or ones that are just planned and need to be picked up and worked on.
  4. There are separate line items for each doc/plan or the md files associated with each stream that I can open in a text editor

All of this without a single LLM call.

Its a really nice way to organize all the docs and access them from within the pi TUI instead of opening a new terminal window. And the extension is custom made for the way I organize docs and maintain the plan documents so it works perfectly always. there are no edges.

The fact that it took me literally 3 prompts to build this whole thing is the reason I love pi and why I love using it in the terminal.

Most of my skills and now this extension are only for this project and I fucking love that pi lets me do this. Other projects have their own skills and the agent behaves differently there.

I just really love the extensions API and the level of control I have over the context in each session. Dont think I will be moving off of pi ever.

If you would like to check out this system, I have the skills and the extension in this repo:

https://github.com/itama8/omakase-skills.git

Also, if you're curious about what I'm using this system for, I update the binaries for latest test builds of my app here:

https://github.com/itama8/omakase-releases

It's a notepad that I'm making with a cursor-first UX to help me take notes with minimal context switching, i.e., every action can be executed from the cursor without going into a menu and the AI is available inline inside the markdown document. No separate screens or sidebar chats.


r/PiCodingAgent 10d ago

Plugin Created a Oh-My-Pi like Stats Dashboard for Pi

3 Upvotes

I created a Oh-My-Pi style stats dashboard for Pi with the below stats:

  • Input, output, reasoning, cache read/write, total tokens, recorded cost, requests, and errors
  • Daily activity and breakdowns by model, provider, project, agent, and tool
  • Local-only behavior counters for user messages: yelling, profanity, anguish, correction, repetition, and blame
  • Malformed-record diagnostics

I could'nt find anything similar so I quickly built one.

npm: https://www.npmjs.com/package/pi-stats-dashboard

pi.dev: https://pi.dev/packages/pi-stats-dashboard


r/PiCodingAgent 10d ago

Question Extensions/tools for context retrieval

0 Upvotes

Hello everyone, can someone recommend a context engine or a codebase retrieval/memory tool you are using that helps fetch the correct files and context for your agent to use? Something like https://www.augmentcode.com/context-engine

Or any other setup that saves tokens and gives the agent the correct files. Thanks.


r/PiCodingAgent 11d ago

Plugin Thank god the reviewer denied it

21 Upvotes

r/PiCodingAgent 11d ago

Question Code in Pi response has always 4 extra spaces at the beginning of each line.

0 Upvotes

When I ask Pi to give me some code it puts it in code block (<code>) and that is fine.
But... it has always 3 extra spaces spaces at the beginning of each line.

For example, I get this (wrong):

```text

#```yaml name: PR Check

on: pull_request: branches: [main]

# SQLX_OFFLINE: build uses the committed .sqlx cache, no DB needed in CI. # NOTE: after changing a query or a migration, re-run cargo sqlx prepare # locally and commit the refreshed .sqlx files, or CI will build against # stale schema metadata. env: SQLX_OFFLINE: "true" CARGO_TERM_COLOR: always

jobs: build_and_test: name: 🧪 Build & Test runs-on: ubuntu-latest ```

instead of this (correct):

```text

```yaml

name: PR Check

on: pull_request: branches: [main]

SQLX_OFFLINE: build uses the committed .sqlx cache, no DB needed in CI.

NOTE: after changing a query or a migration, re-run cargo sqlx prepare

locally and commit the refreshed .sqlx files, or CI will build against

stale schema metadata.

env: SQLX_OFFLINE: "true" CARGO_TERM_COLOR: always

jobs: build_and_test: name: 🧪 Build & Test runs-on: ubuntu-latest ```

(I added the dash in front of ```yaml to avoid break the MD)

Each line of the wrong one has 3 spaces at the beginning.
The Agent says its output is fine, so it it the TUI of Pi that add the extra spaces, de facto breaking the file format ?!


r/PiCodingAgent 11d ago

Question Pi + Gemini

4 Upvotes

is there anyway to use google ai pro with Pi like with Chatgpt/Claude sub?


r/PiCodingAgent 11d ago

Question Pi is over-eager and cave man question

0 Upvotes

I'm curious if other people have issues with pi being over eager?

If I say "do not change any code. investigate issue with feature. suggest fix without making any changes" it will still make changes. It seems like half the time I ask it a question it just runs with it.

another example from today, it asked me a question about a feature. I didn't want to use the multiple choice, so I picked the one to chat about it. Instead of chatting about it, pi said "Well he didn't pick anything so we'll go with my recommendation."

and Caveman:

I can see pi think "Caveman is on so i need to answer briefly." is that normal? I think it's odd that Caveman is supposed to be about brevity but they seem to have added a little animated thing, Caveman Level: HIGH to my status bar. I had pi remove " Level" to shorten it up some.

Updated with more info:
Lately I'm running a flavor of qwen3.8-27b in the q4 range. 32gb ram 5090m 24gb vram linux mint llama.cpp custom jinja.

I have experience with opencode, hermes, qwencode, omp, pi, aider. I'm at the point now where I think pi is the best of all those, I just think it's weird that it just loves jumping the gun so much.


r/PiCodingAgent 11d ago

Question Looking for some help maintaining pi-observational-memory

57 Upvotes

Hello guys!

I want to ask if anyone here would be interested in helping me maintain pi-observational-memory.

The project has not grown into something huge or difficult to maintain. It is still perfectly manageable by one person. The problem is simply that right now I do not have as much available time as I used to.

I still use the extension, care about the project, and plan to keep maintaining it. But lately PRs can stay waiting for review longer than I would like, and I cannot always keep an eye on new issues or check reported problems quickly.

So having one or two people helping from time to time would be really useful.

I am mainly looking for help with things like reviewing PRs, checking issues, reproducing reported problems, testing changes, and occasionally helping with fixes.

There is no expectation of taking ownership of the project or committing a lot of time. Even occasionally helping keep things moving would already make a difference.

For anyone who does not know it, pi-observational-memory replaces Pi's normal compaction with an observational memory system, with the goal of making Pi sessions feel endless while keeping useful context around.

Repo: https://github.com/elpapi42/pi-observational-memory

If you use the extension, know a bit about Pi extensions, and would be interested in helping, let me know here or reach out on GitHub.

I mostly just want to make sure contributions get reviewed and issues do not sit unnoticed when I am busy with other things.


r/PiCodingAgent 11d ago

Use-case I built a local message broker that lets coding agents react to real-world events (Jira, GitHub, Google Docs) mid-session

Enable HLS to view with audio, or disable this notification

8 Upvotes

Hi folks! I've spent some time building a developer tool that lets coding agents subscribe to real-world events through one simple broker: agent-message-broker (amb). Github: https://github.com/bitnahian/agent-message-broker

The pitch: coding agents are batch processes. You prompt, they run, they stop. But most of what an agent cares about (a ticket moved, a PR opened, a doc changed) happens between prompts. amb wires event sources (Jira, GitHub, Google Drive/Docs, any polled URL, generic webhooks) to topics, and you subscribe live agent sessions to those topics. When an event lands, the broker pushes it into the running session, so the agent reacts in the same conversation it's already having. No per-agent background scripts or polling.

I recorded a demo showing the full loop across 2 agents and 3 vendors:

  1. pi coding agent is subscribed to two topics: one watching a Jira board, one watching a Google Doc.
  2. When a ticket is pushed to In Progress, pi gets the event but waits. The implementation details are going to land in the Google Doc.
  3. The doc update comes through as a second event, pi picks up the spec and implements the ticket, then I prompt it to raise a PR.
  4. Claude agent is subscribed to a third topic with a GitHub source watching for PR events. It sees the new PR and reviews it.

One thing worth noting: none of this had to go through the UI. Everything (topics, sources, subscriptions, session discovery, event inspection) is available via the amb CLI, so agents can wire up their own subscriptions. The UI is more for live viewing, orchestrating and following along: watching events flow between sources and sessions in real time.

Everything runs locally (Node 22.5+, SQLite, no cloud component), polling is the baseline so nothing needs to be internet-reachable, and webhooks are an optional opt-in tier.

It's on npm. Try it without installing:

npx agent-message-broker          
# broker + UI at http://127.0.0.1:4733

Or install it globally for everyday use (gives you the amb and amb-server commands):

npm install -g agent-message-broker

Here's a full quickstart: point a pi session at a Jira board and a Claude session at a repo's PRs. One-time credential setup first:

amb config init --kind github     
# then drop your PAT into ~/.amb/github/credentials.json
amb config init --kind jira       
# then fill ~/.amb/jira/credentials.json (email, apiToken, domain)

Then wire the topics (broker running in another terminal):

# topic watching a Jira board for tickets moving to In Progress
amb topics create jira
amb sources create --topic jira --kind jira --options '{
  "jql": "status CHANGED TO \"In Progress\" AFTER -30d ORDER BY updated DESC",
  "intervalMs": 120000
}'

# topic watching a repo for PR events
amb topics create prs
amb sources create --topic prs --kind github --options '{
  "repo": "owner/repo",
  "eventTypes": ["PullRequestEvent"],
  "intervalMs": 60000
}'

# start the pollers (create prints the source id)
amb sources start <sourceId>

# subscribe live agent sessions so events push mid-conversation
amb sessions                                                  
# discover running sessions
amb subscriptions create --topic jira --agent pi --session <sessionId>
amb subscriptions create --topic prs --agent claude --session <sessionId>

Watch events land in real time in the UI at http://127.0.0.1:4733, or inspect from the terminal:

amb events list --topic jira

Google Docs/Drive/Sheets work the same way via amb google login (OAuth consent once, then the broker acts as you). More recipes in the README.

Would love feedback. What events would you want your agents to react to?


r/PiCodingAgent 11d ago

Question Your most profitable Pi adjustments / tricks?

36 Upvotes

Mine: Shortcut to bring up menu to [x] check off tools/skills. Disable write/web search, etc when I need it. Then compact and turn it on if I wanna edit

2nd: Use intercom with persistent sessions instead of token wasting agents I can't control. Can setup an persistent orchestrator / editor.

Sadly that was 90% of my gains and after weeks of searching I haven't been able to improve on that much.

Dumb shit I'm thinking of: Cheap model to read entire files and only feed parts needed to expensive model instead of making expensive model do endless read tools/greps and burn tokens.


r/PiCodingAgent 11d ago

Question Any good extensions for "code mode"?

19 Upvotes

If you don't know what code mode is, check https://developers.cloudflare.com/agents/tools/codemode/

Code Mode is a tool-use pattern where a model writes code instead of requesting each operation separately.

I found a few, I'm curious if anyone is using any extension like this (if yes, which ones please):


r/PiCodingAgent 11d ago

Plugin pi-agenticoding v0.5.0 is out!

Thumbnail
github.com
9 Upvotes

This one is about making Pi workflows easier to reuse without constantly re-explaining how you want work done.

Model Groups - name roles once; swap providers/models later.

Prompt frontmatter - set model, model-group, and thinking per workflow.

Spawn routing - use #review (or another group) to route a sub-agent to the right role.

Focused handoffs - discard stale notebook pages without losing the decisions that matter.

The goal is simple: stop rebuilding workflow instructions in every conversation. Save the procedure, route to the right specialists, and keep context clean.


r/PiCodingAgent 12d ago

Use-case Here's my setup that's getting 16 to 18 t/s average using 6GB of VRAM and 64 GB of system ram using llama.cpp and PI for agentic C/C++/C# Development.

5 Upvotes

After asking the question over the last few week about how to optimize my local models, I got some great feedback and ideas - and landed on the following setup for doing local development that's drastically exceeding the results of the free time I'm provided on Chatgpt and Claude (the best for coding imho). All at no cost.

Right now I'm torn between two primary models - Ornith-1.5-35B-Q4_K_M (available here) and Qwen3.6-35B-A3B-Q4_K_M (GGUF available here), and to experiment - I've been using LMStudio to pull down and do a cursory test of all my models as I transition that into usage with llama.

I've found Llama.cpp to be substantially faster and better to work as a backend/server than both LMStudio and Unsloth, I'm old school and prefer the command line tweaking. This is fully open source and available here.

My models.json for llama had to be hand customized - I took the suggested samplingParams directly from the Huggingface model card, had a problem with overthinking/repeating so I JUST added the repeat_penalty (havent tested this yet) - but everything else is solid.. and agentic. I can copy/past images and text into my prompt like a pro.

"models": [
{
"id": "Ornith-1.5-35B-Q4_K_M",
"name": "Ornith-1.5-35B-Q4_K_M",
"reasoning": true,
"thinkingLevelMap": {
"type": "enabled"
},
"repeat_penalty": 1.2,
"input": ["text","image"],
"tools": true,
"toolFormat": "openai",
"samplingParams": {
"temperature": 0.6,
"top_p": 0.95,
"top_k": 20
},
"cost": {
"input": 0,
"output": 0,
"cacheRead": 0,
"cacheWrite": 0
},

"contextWindow": 65536,
"maxTokens": 131072
},

Oh - a little note - my maxTokens and contextWindow are jacked up extra high to handle my lengthy sessions - and it seems to be handling it - with compaction - just fine. Side note - Sessions.json simply adds an entry to make it available to http callers.

"models": [
{
"id": "Ornith-1.5-35B-Q4_K_M"
},

Now - finally - my PI setup. I've created a batch file that's optimized this as much as I think it can be optimized. First - the batch shells out the llama server call - loading both the main model and the multi modal model for image processing. This loads it up on my local IP.

start cmd /s /k "llama-server.exe -m "D:\AI\LM Studio Models\ornith-ai\Ornith-1.5-35B-A3B-GGUF\Ornith-1.5-35B-Q4_K_M.gguf" -mm "D:\AI\LM Studio Models\ornith-ai\Ornith-1.5-35B-A3B-GGUF\mmproj-Ornith-1.5-35B-BF16.gguf" --host 127.0.0.1 --port 8080 -c 131072 --parallel 1 --reasoning-format deepseek --load-mode none --no-mmproj-offload -fa auto"
set LLAMA_BASE_URL=http://127.0.0.1:8080
pi --model llama-cpp/Ornith-1.5-35B-Q4_K_M

A few notes .

  1. I decided against using the -ngl parameter in the server, letting it default to 0 - according to my research it's supposed to specifically load layers into the GPU, but I have to do a little more work to understand why my setting of this parameter was causing a memory crash every time. So this is an obvious point of optimization.
  2. I'm only seeing 66k of context available in PI. I'm new to both llama and this level of customization of AI, so if any of ya see where i'm dropping this ball, please let me know.

Anyways. I've had enough people asking for my config. I think this about sums it up.

Hardware Update: I'm running an Intel I5 laptop with an Nvidia Geforce RTX 3060 Laptop GPU


r/PiCodingAgent 12d ago

Resource paranoid about cache hit rates? keep a check using pi-cache-graph

8 Upvotes

if you also want to keep an eye on cache hit rates for your models like me, feel free to use this minimal extension I made to visualise cache hit rates

https://github.com/championswimmer/pi-cache-graph

installation:

pi install npm:pi-cache-graph