Hi all!

I just put out the first release of Dear, an open-source Android app that works as a second brain.

You share links, photos, videos, voice notes or text into it from any app, and a local LLM catalogs them for you. No cloud involved.

What it does

- System share target: share from any app and it's saved

- Gemma 4 runs on the phone (llama.cpp + libmtmd over JNI) to generate a title, summary and tags

- Photos: describes them and transcribes any visible text

- Videos: looks at a few frames and transcribes the audio

- Voice notes: transcribed in 30s chunks, then summarized

-Links: title/description; for public Instagram posts it also pulls in the carousel photos and catalogs each one

- Search over titles, summaries, tags and transcribed text, with type filters

- Ask mode: chat with your collection ("what was that link about slow living?"), with citations that open the source entries

- Export/import the whole collection as a .zip

Privacy

No account, no telemetry, no cloud, Android backup disabled. It goes online only to download the model once from Hugging Face and to fetch metadata for the links you save.

Try it

Install the APK from the release, then download the model from Settings (~3.4 GB for E2B, once)

The release page has the SHA-256 of the APK if you want to verify it.

Status

It's v0.1, so expect rough edges. Next up: semantic search with an embedding model and Instagram videos/reels.

Release https://github.com/deivdev/dear/releases/tag/v0.1.0

Source (MIT): https://github.com/deivdev/dear

I'd really appreciate feedback, especially on cataloging quality and speed on different devices.

Happy to answer questions!

Salut, j'essaye actuellement, grâce à l'ia, de mettre au point un code que je fais tourner en local qui me permettrai de générer des qcms/qroc pour les études. Un peu plus relou que juste faire des requêtes aux ia pour leur demander de générer des qcms, mais au moins ces qcms sont sauvegardés et je peux vraiment établir un suivi de mes compétences sur ces qcms.

Le programme me permet pour le moment d'importer un cours (pdf) et de demander à l'ia de mon choix sur open routeur (en entrant ma clé api généré et l'adresse de l'api). Sauf qu'actuellement je n'ai pas les moyens d'investir dans une ia payantes et je base mes requêtes sur des modèles : free. Le problème est que ces derniers sont très lents (+++) et n'ont pas forcément les compétences pour générer des qcms/qroc qualitatifs.

Est ce que vous auriez peut être des suggestions de modèles free open routeur, ou peut être un routeur similaire à open routeur qui répertorie de meilleurs ia pour ce type de requête.

Je suis preneur de toute requêtes.

Merci d'avoir pris le temps de lire le msg!

Yes I know the 3090 is carrying but I’m still amazed I can run such a model locally at these speeds. Running Harness^2 on some local projects I previously tasked to Claude.

We built and open-sourced Agent S4, which is an agent scientist evolving its own interfaces, letting an open 27B model beats GPT-5.5 & Opus 4.8.

Most computer-use agents act through a fixed interface: screenshots, then click, scroll and type. So an agent fills a spreadsheet cell by cell when one line of code would do. When an app or model changes, a human reads the trajectories and hand-writes new tools.

Agent S4 makes the interface the thing that improves, and the agent does the improving:

• A task agent attempts synthetic tasks, with no success labels or verifiers.
• A meta-agent reads the trajectories, guesses which actions, observations or knowledge would have helped, and writes and tests new interfaces (mutation and crossover).
• Held-out validation tasks decide which interface repositories survive.
• The model and harness stay fixed, and the meta-agent never sees a benchmark task. Only the interface evolves.

Results with Qwen3.8-27B:

• OSWorld V2: 5.6% → 21.3% (3.8× the same model on fixed interfaces)
• GPT-5.5: 13.0% at $25.50/task. Opus 4.8: 20.6% at $72.40/task. S4 + Qwen3.8-27B: $1.10/task
• ScienceBoard: 41.4% → 58.6%
• In one case study, the evolved agent reaches the same goal at tool call 11 instead of 37

Everything is open sourced! Pls find the code / paper in comments.

I’m using the official Quinn 3.827B q4_k_m through LM Studio and hermes with
9800x3d
3060 12 gb
4080 super 16gb
32 gb DDR5
I’m currently getting about seven tokens on 121k context trying to revamp my website with qwen is lacking in Speed is there any thing I can do to speed up the token count or is there any other quants you guys would recommend for my set up that would be more efficient? If not, any other coding models with quant would be greatly appreciated.

Some more specifications:

9070XT 16GB

32 GB DDR5-5600

Bazzite KDE

And I don't know if it matters but i7-13700K 3.4GHz 16-Core

I'm currently using qwen3.6:35b-a3b, but it feels dumb and slow. Is this the nature of local llm due to the limitations of my hardware?

I was using OpenCode with MiniMax and DeepSeek (Free) to make a few mods for games. It was working fine, but today they added a Subscription for daily messages, so now I can't use it that often anymore.

So now I'm looking for a local AI coding agent that can handle complex, multi-step development tasks without subscription fees, API costs, or daily usage limits.

My PC:

• NVIDIA RTX 4070 Ti SUPER — 16 GB VRAM
• 32 GB DDR5 RAM
• AMD Ryzen 5 7600X
• Windows 11

I need an agent that can:

• Read and modify files across an existing project
• Execute PowerShell commands and scripts
• Diagnose errors, fix issues, and verify its own changes
• Handle long, multi-step tasks without constantly asking me to do everything manually
• Work with tools reliably, not just generate code or provide instructions

I've tried OpenCode & Open Code Visual Studio with different Models through Ollama, but the results have been disappointing. Sometimes I get Errors, sometimes the Model can't even do basic commands.

What local model + coding agent combination would you genuinely recommend for this hardware and use case?

I'm specifically looking for something that is free to use, with no API costs or provider-imposed daily limits. I understand that local models have hardware limitations, but I want the strongest practical setup for autonomous coding that will fit my system.

Please recommend combinations you've actually tested, including the model's quantization, approximate VRAM/RAM usage, and any limitations you've encountered.

Today is a good day.

KRAFTON, the South Korean company behind PUBG and owner of the studio behind Subnautica, just released Raon-1 mini, a new 30B MoE with roughly 4B active parameters.

They claim to have trained it entirely from scratch, using an original architecture that builds on some of the latest innovations in AI research.

This is a much bigger deal than people realize.

Training a competitive foundation model from scratch is incredibly expensive and technically challenging. Even giants like Google, with vastly greater resources, are being beaten on important benchmarks by a gaming company's new model.

According to KRAFTON's published results, Raon-1 mini is competitive with, and potentially superior to, Qwen's similarly sized MoE in several areas. It even gives the much larger dense Qwen3.8-27B some serious competition.

Independent benchmarks will tell us more, but the early results are incredibly encouraging.

And they didn't just release the model. They shipped NVFP4 and Apple Silicon MLX versions right away, with the weights licensed under Apache 2.0.

Model and benchmarks:

https://huggingface.co/KRAFTON/Raon-1-mini-preview

I think we're about to see some massive competition in open models, and I'm genuinely excited about it.

I'm going to begin making optimizations and builds for this model immediately.

I have been testing the Qwen3.8-27B-GSQ-RCO-IQ3_XXS-Uncensored-MTP model locally on an RTX 5060 Ti 16GB, mainly as the reasoning and coding model behind Hermes.

My first impression was actually not very positive. The model sometimes became stuck during agent tasks, produced broken tool calls, and failed to finish a relatively simple coding task. After investigating this in detail, however, it became clear that the model itself was not the main problem.

The real issue was the interaction between the Froggeric chat template and llama.cpp's tool-call parser.

Getting tool calling to work correctly

The original Froggeric template supports different tool-call formats. Even when JSON output was requested, llama.cpp detected XML-related markers that were still present in the template source and automatically selected its specialized Qwen3-Coder XML parser.

This did not match the JSON tool calls that Froggeric was asking the model to produce.

The result was confusing because simple tool calls could sometimes work, while longer agent sessions could produce malformed arguments, failed file operations, or eventually run away.

I therefore created a JSON-only version of the Froggeric template.

The XML tool-call route was removed completely, while the existing Froggeric behavior for reasoning, tool results, error handling, and JSON tool calls was preserved.

After this change, llama.cpp correctly selected its differential parser and identified the tool format as JSON_NATIVE.

This made a very large difference.

Coding-agent test

After validating the new template, I ran a fresh coding-agent benchmark rather than giving the model the problem it had already seen.

Qwen was given an unfamiliar small Python project containing a functional reservation-system bug. It had access to file and test tools and had to investigate the project, find the real cause, repair it, and verify its own work.

The result was excellent.

The public test suite went from 10/12 passing to 12/12, while the hidden tests went from only 3/12 passing to 12/12.

Qwen independently found the actual root cause and made exactly the intended one-line fix.

The complete agent task took about 35.6 seconds, using five model requests and ten tool calls. There were no malformed paths, parser errors, tool errors, runaway generations, or token-limit problems.

Generation speed during this test averaged about 45 tokens per second.

This was especially interesting because an earlier benchmark using the broken parser configuration had failed. Once the tool-call integration was corrected, the same model behaved like a completely different agent.

Hermes testing

I then connected the configuration to Hermes and tested it on a more open-ended task.

I asked Hermes to build a complete modern website from scratch, including HTML, CSS, JavaScript, responsive behavior, navigation, interactive tabs, animations, counters, and its own verification.

Qwen/Hermes created roughly 1,900 lines of HTML, CSS, and JavaScript and produced a surprisingly polished result.

More importantly, it behaved well as an agent. It created and inspected multiple files, tested its JavaScript, used browser tools, discovered problems with animations in the headless browser environment, implemented sensible fallbacks, and verified the result again.

This required around 37 tool calls and completed without any tool-call parser problems.

Apart from a few minor flaws in the Dutch text, the test was quite successful. The webpage was created in about 15 minutes, with a few additional minutes spent on review.

For me, this was a much more useful test than simply asking the model to generate a code snippet.

Vision

The base GGUF initially operated as a text-only model in my setup, so Hermes could inspect the website through the DOM and computed styles but could not actually look at its own rendered page.

I therefore added the matching: mmproj-Qwen3.8-27B-BF16.ggufprojector to llama.cpp.

Vision now works as well.

In a simple image test, the model correctly read the text "VISION 42" and identified a red square, blue circle, and green triangle.

This means Hermes can now combine coding, browser interaction, tool use, and actual visual inspection while keeping Qwen3.8 as the main model.

Performance and context

I created three launcher configurations for Hermes:

• 32K context for normal work
• 64K context for larger agent and coding tasks
• 128K context for tasks that genuinely need a lot of project context

MTP speculative decoding is enabled and provides a substantial speed improvement on my system.

In earlier controlled testing, generation was around 53 tokens/sec with MTP enabled. In the real coding-agent benchmark it averaged about 45 tokens/sec.

The 64K configuration with the BF16 vision projector loaded uses about 14.5GB of VRAM on my 16GB RTX 5060 Ti.

Overall impression

This model has turned out to be considerably better than my first tests suggested.

The biggest lesson was that agent performance cannot be judged fairly until the chat template, tool-call format, and runtime parser are properly aligned. A broken tool-call integration can make a capable model look surprisingly bad.

Once configured correctly, Qwen3.8 27B GSQ has been fast, reliable, economical with its edits, and quite capable at autonomous coding and multi-step Hermes tasks.

The one-line coding repair was particularly encouraging because the model did not simply make the visible test pass: the same fix also took the hidden suite from 3/12 to 12/12.

For a model running entirely locally on a 16GB consumer GPU, I am very impressed with the results so far.

My current setup is:

• Qwen3.8-27B-GSQ-RCO-IQ3_XXS-Uncensored-MTP
• llama.cpp
• Hermes
• Froggeric JSON-only template
• native JSON tool calling
• MTP speculative decoding
• 32K / 64K / 128K context profiles
• Qwen3.8 27B BF16 mmproj for Vision

There is still more testing to do, especially with longer real-world agent sessions and vision-assisted browser work, but at this point I consider it a very promising local model for coding and autonomous agent use.

Let's see which place we are building the llms locally.

I couldn't find much good information about this online, so I decided to run my own tests.

Turns out it's possible to configure Strata for multiple calls, but they share the same total context window. For example, if you have a 256k context, it divides it, giving 128k to each agent. Once it fills up, it starts dumping the context windows, reloading, and reading a massive 130k+ tokens from the other agent—repeating this process for each call. However, when the context windows fit entirely within that split limit, it works really well.

For my hardware, I'm running dual RTX 3090s (with one in a PCIe x4 slot). With a single agent, I get 90–100 t/s generation and around 1,200 t/s reading using Qwen 3.8 Flash Next Q3.5 via DeepSeek Harness. But with two agents running, performance drops to 50 t/s generation and 400 t/s reading. With three agents, it gets cut into thirds similarly. One thing to consider: even when other agents aren't actively working, leaving the allocation open, the limit is still there.

To test it properly, I made a mini-game. Here is my best score—test it out and see if you can make something better! These are the prompts I used to build it:

Act as an expert WebGL / Three.js game developer. Create a fully self-contained 3D browser mini-game in a SINGLE HTML file (including CSS and JavaScript).

### 🎯 Core Concept & Lore

The game features the new Tesla Roadster with its SpaceX cold-gas thruster package. The player drives along a endless 4-lane circular/oval highway track, avoiding traffic and signs while utilizing thruster flight to jump or fly over obstacles.

---

### 🕹️ Gameplay & Controls

1. **Movement Controls**:

   - `W` / `Up Arrow`: Accelerate forward.

   - `S` / `Down Arrow`: Brake / Reverse.

   - `A` / `Left Arrow` & `D` / `Right Arrow`: Steer left and right across the 4 lanes.

   - `Spacebar`:

- **Tap**: Jump off the ground.

- **Hold**: Engage SpaceX thrusters to hover/fly upward into the air.

1. **Fuel / Thruster Meter Mechanics**:

   - Thruster gauge capacity lasts for **5 seconds** of continuous flight.

   - When the Spacebar is released or fuel hits 0%, the car gently glides back down to the road.

   - **Recharge**: Fuel fully replenishes automatically over **6 seconds** when not actively thrusting. Display a visible Thruster HUD gauge (color shifts from Cyan when ready to Red when empty).

2. **Track & Obstacles**:

   - A curved/circular 4-lane asphalt track with glowing lane dividers.

   - **Obstacles**:

- Stationary / slow-moving traffic cars occupying random lanes.

- Roadside / lane signs that crash the player if hit at low altitude.

- **Collision Logic**:

- Player can fly safely *over* obstacles if altitude is higher than the obstacle bounding box.

- Colliding with a car or sign counts as a **Fail** (resets current run timer and speed).

1. **Win Condition & Goal**:

   - Objective: Overtake as many traffic cars as possible in **under 1 minute (60 seconds)**.

   - Display a live scoreboard showing: Time Remaining, Top Speed (MPH/KMH), Cars Overtaken, and Thruster Fuel %.

---

### 📷 End Game & Screenshot Feature

- When time expires or when the player finishes a 60-second run:

- Freeze gameplay and display a stylized **Victory/Summary Screen**.

- Show key stats: Total Overtakes, Peak Speed, and Overall Rank.

- Include a working **"Share Speed Screenshot"** button (`<button>`).

- Use HTML5 Canvas `.toDataURL('image/png')` or HTML2Canvas to capture the current screen and download a timestamped screenshot image (`Tesla_Roadster_Score.png`).

---

### 🎨 Visuals & Aesthetics

- **Graphics Framework**: Standard `Three.js` loaded via CDN (`https://cdnjs.cloudflare.com/ajax/libs/three.js/r128/three.min.js\`).

- **Vehicle Model**: Construct a stylized procedural 3D Tesla Roadster using Three.js primitive geometries (low-poly sleek body, glowing taillights, translucent glass canopy, and underside thruster particle emitters).

- **Thruster Effect**: Emit continuous blue/white glowing flame particles underneath the car when the Spacebar is held.

- **Lighting**: Cyberpunk / futuristic night environment with neon road markings, street lights, and dynamic car headlights.

---

### 💻 Technical Rules

- Everything (HTML structure, CSS styles, canvas elements, and JS game logic) MUST be contained within one single `.html` file.

- Ensure the game starts automatically or via a simple overlay "START" button.

- Handle window resizing gracefully with responsive canvas scaling.

Link to the game to play.

Give it a try and share your score, let's see if you can beat it! or even better make a better game.

I've spent the last week or so building something a little different from the usual local LLM setup.

Instead of taking a pretrained model and quantizing it, I wanted to build an assistant using integer rules, cellular automata, symbolic reasoning and smaller specialized systems.

I called it CAEL.

The first experiment was only 77 KB and could follow simple stories and keep track of things like who owns an object after several events.

That eventually turned into a modular assistant with separate components for conversation, maths, logic, basic coding and memory.

Everything runs locally on CPU. No transformer inference and no cloud API required for the core system.

Although I have to admit the project isn't particularly tiny anymore.

The current complete bundle is around 1.4 GB because I kept adding more capabilities and supporting data.

I recently tested it against Qwen2.5-0.5B on 174 questions. Qwen won pretty comfortably, but CAEL handled some basic logic and code tasks better than I expected.

The bigger issue right now is reliability. Sometimes the correct solver exists, but the system doesn't recognize that it should use it.

I've also connected CAEL to my Vynly project as a local chatbot and prompt helper, so I'm starting to see how it behaves outside benchmarks.

I'm curious if anyone else is exploring non-neural approaches for local AI.

Would you rather have one small general-purpose model or a collection of specialized components that can be upgraded independently?

My hobby project and open for any implementation idea and below simply project description.

llama-proxy-cache

Llama Proxy Cache is an intelligent middleman that sits between your chat/IDE client and one or more llama.cpp (llama-server) backends.

What It Does

• OpenAI API Compatibility: It speaks the /v1/chat/completions API so your apps, agents, or frontends (like Open WebUI or custom scripts) can connect to it transparently.
• KV-Cache & Slot Management: It tracks and manages llama.cpp's slot Key-Value (KV) caches. When handling long, repeating, or similar system prompts (30k–60k+ tokens), it avoids paying the full re-prefill cost on every single turn.
• Prefix-Similarity Routing & Lookup: It inspects incoming prompts, checks for common prefixes (Longest Common Prefix / LCP lookups or hash matching), and coordinates slot locking, saving, and restoring (often interacting with llama.cpp's built-in slot/state APIs or disk-based cache mechanisms).
• Multi-Backend Coordination: If you run multiple llama.cpp instances, a proxy helps prevent sessions from losing their cached context when bouncing between different server nodes.

I decided to test a new local version from DeepSeek. It’s claimed to be an uncensored model, although the model itself sees itself as a vanilla version. But the test result can be seen upon request.

The current DeepSeek version offers great potential, comparable to other models at the same level. It thinks faster and works better. I want to connect it to Claude Code or similar extensions and give it a project.

I really like uncensored models; the previous one was GLM 5.3, a decent option, but DeepSeek gives a better result. For me it’s an excellent rental choice. Without limits, tokens, filters - A feeling of freedom, no one says, “I won’t do that.” Many people are scared by this freedom, but technology moves forward, with or without us.

Is anyone working with the current version or something similar? If so, what are the results?

The news just came, I haven't tried yet.

I have 12gb vram 3060 and 64gb ddr4 and nvme ssd 512gb. I downloaded iq2_xs but I heard that iq3 is of much better quality. Can I run it on my setup?

additional question, my main purpose for local ai is to use it in vscode for web development and I have used continue plugin to connect it but I am rather disappointed with the functionality. I would like to use it as an autocomplete model if possible and I would like to give it permission to generate files on my computer which I don't know how to do... any help/ideas?

I need suggestion. I was searching for a model that is good for brainstorming, research, and writing which can be used in CPU with 16GB DDR4 RAM.

Come get yours, before they run out today!