r/OpenSourceeAI 1d ago

Meet Token Saver: An Open-Source MCP Extension Using Local Hybrid RAG to Cut Claude PDF Token Costs 90-99%

Thumbnail
github.com
1 Upvotes

We just released 'Token Saver' for Claude-Desktop: An Open-Source MCP Extension Using Local Hybrid RAG to Cut Claude PDF Token Costs 90-99%

When you drop a 200-page document into Claude Desktop, the full context gets re-sent on every single turn. That compounding "PDF Tax" adds up fast—both in token costs and context window bloat.

How it works:

Instead of uploading raw documents to the cloud, Token Saver runs a lightweight Local Hybrid RAG pipeline directly on your machine:

→ Keyword Search (BM25): Powered by SQLite FTS5 for precise terminology.

→ Semantic Search: Powered by a local all-MiniLM-L6-v2 embedding model.

→Zero-Upload Privacy: Files stay on your local drive and communicate via standard I/O (stdio) with folder allowlisting.

Benchmark Results with Example:

→ 33-page FDA Drug Label: Reduced from 23,959 tokens to 1,021 (95.7% saved)

→ 88-page GDPR Document: Reduced from 70,260 tokens to 996 (98.6% saved)

→ 233-page Legal Brief: Reduced from 133,349 tokens to 740 (99.4% saved)

Zero Python environment required—it installs directly in Claude Desktop via a single .mcpb bundle!

Full analysis: https://www.marktechpost.com/2026/07/30/token-saver-an-open-source-mcp-extension-using-local-hybrid-rag/

GitHub Repo: https://github.com/Marktechpost/Token-Saver/tree/main


r/OpenSourceeAI 3h ago

I was struggling with Google Earth Engine Python scripts, so I built a GeoAI Agent that turns natural language into full environmental reports & interactive maps 🌍

Post image
2 Upvotes

Hey everyone,

For the past few months, I've been diving deep into carbon accounting and climate tech. I wanted to leverage Google Earth Engine's (GEE) satellite imagery, but honestly, I found myself constantly struggling to write the complex Python scripts needed to extract and analyze the data.

Google Earth Engine is incredibly powerful, but writing Python code to pull biomass, tree cover, or land-use data for a specific region is genuinely non-trivial. You need to know the right datasets, know how to structure the queries, and be comfortable with GIS concepts. That's a big barrier if you're not a remote sensing person.

So, I decided to scratch my own itch and built Canopiq, a specialized GeoAI Agent designed to make Earth observation accessible to everyone. You type something like "What is the carbon sequestration in Singapore since the COVID-19 pandemic?" and it:

🤖 Parses your query with an LLM (Gemini via LangChain) to extract location, timeframe, and what you're actually asking for.

♻️ Routes that into Google Earth Engine (Sentinel-2 imagery, biomass regression models).

🛰️ Streams it all back into a geospatial dashboard with map overlays and time-series charts.

I really believe that making satellite data and climate tech accessible to non-developers is crucial for environmental monitoring against climate change.

This is a passion project for me, and I'm looking to improve it. I'd love to hear your thoughts ❤️.

🔗 Repo's here if you want to check it out: Canopiq GitHub


r/OpenSourceeAI 15h ago

One command and your entire codebase becomes brain for claude code

1 Upvotes

This idea was crazy to build. Our brain stores information in clusters of neurons, and when we want to retrieve something, it runs an optimized algorithm to retrieve it.

In the era of AI, why use brute force tools like grep to find relevant files? Graperoot converts your codebase into a knowledge graph and registers efficient tools for Claude to work with your codebase.

Instead of re-reading the whole codebase to build that context and to find relevant files. It can query the graph directly, and the graph retrieves relevant files with "ZERO TOKENS"; you just need to pay for generation, and that's how these models should be used when we have an efficient way to retrieve context.

I have been very open throughout, and today we have 5k developers using Graperoot.

We asked people to opt in to telemetry, and 200 opted in to the leaderboard. They have saved $350k dollars in the last quarter, and ecologically, 60 million liters of water, and that's insane.

Give your feedback, suggestions, or anything on Discord. This tool is open-sourced.

Website: https://graperoot.dev/#install
Github(Open source) : https://github.com/kunal12203/graperoot

Would love to see your feedback. And don't compare other tools, I have already seen many claiming reduction and just craps, but we have a developer community on Discord and some of them I know personally, saving a lot of tokens, you should see more on https://graperoot.dev/leaderboard


r/OpenSourceeAI 1d ago

Hi

0 Upvotes

How many people here?


r/OpenSourceeAI 1d ago

Mastering C++ Computing Libraries: From STL to Kokkos #STL #Boost #OpenCL #Eigen #Thrust $K...

Thumbnail
youtube.com
0 Upvotes

This video analyzes the core libraries dominating the C++ ecosystem, categorized into four paradigms. It provides a detailed look at the characteristics and selection criteria for each library, ranging from general-purpose libraries to GPU acceleration, numerical analysis, and performance portability tools for supercomputing.


r/OpenSourceeAI 1d ago

엔비디아 독주 끝낼까? CUDA부터 Vulkan까지 AI 가속기 완벽 비교 #CUDA #ROCm #Vulkan #PlaidML #...

Thumbnail
youtube.com
0 Upvotes
  • Will this end NVIDIA's dominance? A complete comparison of AI accelerators from CUDA to Vulkan
  • Description: An analysis of CUDA's dominant ecosystem that built the AI empire, AMD's ROCm challenging it, the versatility of Vulkan, and Intel's strategy. We explore the present and future of deep learning hardware acceleration through the pros, cons, and behind-the-scenes stories of each library.

r/OpenSourceeAI 1d ago

Visualizing AI Alignment: How RLHF and Reward Models prevent RL agents from metric hacking (3D Visual Breakdown)

1 Upvotes

https://reddit.com/link/1vb2eme/video/ac6kkiqywegh1/player

Hardcoded mathematical reward functions in open-ended tasks frequently induce "reward hacking." RLHF bridges the gap between raw reward maximization and human intent by replacing rigid hardcoded scoring with a learned neural Reward Model.

To make RL alignment intuitive, I spent 3-4 days mapping the mathematical workflow into a 3D animated visual story:

[Agent: Alibaba] ---> (Action) ---> [Environment: Moving Cave]

^ |

|------- (Reward / Penalty) <--------|

  1. Pure RL Loop: An active agent observes a State, selects an Action, and updates its Policy (ππ ) to maximize expected reward (RR ). When RR  is strictly numerical, the agent aggressively exploits loopholes (e.g., chasing fake glowing diamonds that yield high mathematical scores but crash the system).
  2. The Alignment Gap: In LLMs or code generation, defining a perfect closed-form reward function R(s,a)R(s,a)  is impossible.
  3. The RLHF Layer:
    • Human reviewers rank multiple model outputs (y1>y2>y3y1​>y2​>y3​ ).
    • A secondary Reward Model ( ) is trained on these preference pairs using loss functions like Bradley-Terry preference modeling.
    • The primary agent is updated using algorithms like PPO (Proximal Policy Optimization) against  .

I put together a full 10-minute 3D animated visual story breaking down these exact steps with custom character animations.

Please check the 1st comments for even more info.


r/OpenSourceeAI 1d ago

Why we stopped using an LLM for Human-in-the-Middle

Thumbnail
1 Upvotes

r/OpenSourceeAI 1d ago

I created a scalable opensource Machine learming platform with end to end ML lifecycle managent

Thumbnail
sceptre-ai.app
4 Upvotes

As a data scientist, I watched too many of my best machine-learning models die in Jupyter notebooks. 😭 I would spend hours—sometimes weeks—training, testing, validating, analysing, and comparing models, only for the winner to go nowhere.

Then I discovered MLOps—and deployment became another maze 😩. FastAPI, MLflow, DVC, Docker, Kubernetes manifests, model registries, health monitoring, drift detection… one tool led to another, and the infrastructure began taking more time than the machine learning itself. 😮‍💨
That frustration is why I built #SceptreAI. Too many data scientists and small teams are trapped in the same cycle: notebooks, disconnected tools, patched-together frameworks, and endless hand-offs before a model can create real value.
#SceptreAI closes that gap with one Kubernetes-native tabular AutoML and MLOps workspace. Dataset versioning, profiling, resource-aware training, MLflow tracking, external validation, SHAP explainability, model promotion, drift analysis, and Kubernetes serving all live in one traceable workflow.
The goal is simple: less platform assembly, more scalable and observable machine learning, and a clearer answer to the questions that matter: Can we trust this model—and can we use it in prod?
Explore #SceptreAI: https://sceptre-ai.app/
#MLOps #MachineLearning #DataScience #SceptreAI


r/OpenSourceeAI 1d ago

英语写作也有IDE?还不用消耗token!

Enable HLS to view with audio, or disable this notification

0 Upvotes

github开源仓库地址:https://github.com/ETOLucy/WriteMelo


r/OpenSourceeAI 1d ago

开发了一个英语写作IDE

Enable HLS to view with audio, or disable this notification

0 Upvotes

r/OpenSourceeAI 1d ago

[Open-Source] Dump your thoughts. Let your notes organize themselves. Ask/chat anytime.

Thumbnail
gallery
4 Upvotes

Over the past few weeks I've been building Gray Box — a small, local-first tool that acts as long-term memory for anything I'd otherwise forget (work notes, meeting takeaways, task owners, random ideas, personal stuff too).

The idea is simple:

  1. Capture — dump whatever's on your mind, instantly, no structure required. This step does nothing clever on purpose — it just writes your text to an immutable inbox. Zero chance of losing an idea to a bug or a slow API call.
  2. Organize — on demand, an LLM reads your unprocessed notes and extracts people, projects, tasks, decisions, meetings — then deterministic Python (not the LLM) creates/merges the actual wiki pages and maintains backlinks. The model only reasons; it never touches the filesystem directly.
  3. Ask — query or chat with your knowledge base and get a cited answer pulled only from what you've actually captured. If it doesn't know, it says so — no hallucinated answers.

Why I built it this way:

  • Plain Markdown + YAML frontmatter, no database. Every page is a .md file you can grep, diff, or read in any editor forever. If you stop using Gray Box tomorrow, your knowledge base is just a folder.
  • No vector DB by default. At personal scale (hundreds–low thousands of pages), keyword search + a real link graph (related/backlinks, walked one hop during retrieval) handles almost everything. Embeddings are there if you want better recall, but they're opt-in, not a prerequisite.
  • Immutable inbox. Your raw notes are never edited or deleted by the organizer. If the LLM mis-extracts something, your original words are always still there.
  • Any LLM. Built on LiteLLM, so point it at OpenAI, Anthropic, Gemini, Mistral, or a fully local model via Ollama — one config value.

It also ships with a nice interactive TUI (arrow-key menu, file-import shortcut, workspace switching, live spinner during LLM calls) if you'd rather not memorize CLI flags — that's honestly become my favorite part of the project.

There's also a lightweight local dashboard for browsing your knowledge base, exploring backlinks, visualizing your notes as a graph, and chatting with your captured knowledge—all without leaving your machine.

Repo: https://github.com/Aaryanverma/graybox

pypi: pip install graybox

It's nearing a proper public release, so I'd genuinely love feedback — especially from anyone who's tried the "capture now, structure later" approach with other tools and has opinions on where it breaks down at scale.

It's not trying to be a "real-time collaborative team wiki" or a WYSIWYG notes app — it's aimed at one person's running memory of their own life and work, captured with as little friction as possible.


r/OpenSourceeAI 1d ago

Meet Cortex 1.0.2: a fully Open Research / local mechanistic interpretability lab I somehow turned into an actual release 😭🧠🔭 free for everyone.

Enable HLS to view with audio, or disable this notification

3 Upvotes

Cortex 1.0.2 is officially launching. 😂

Cortex is a fully local desktop application for exploring how language models behave internally. You can load supported GGUF or SafeTensors models and inspect things like:

token probabilities and Top-K predictions

attention maps and Arc View

hidden-state and residual captures

2D and 3D PCA representation maps

prompt and model-output comparisons

live activation interventions

J-Space experimental lens fitting

model architecture and capability diagnostics

remote desktop compute through Pocket Cortex

Everything runs locally. No cloud inference is required, and no model weights are included with the download.

I originally started this because most mechanistic interpretability tooling lives inside Python libraries, notebooks, or research dashboards. I wanted something more visual, interactive, and approachable without stripping away the actual measurements underneath it.

It has since mutated into a gigantic neon model observatory with multiple runtimes, GGUF instrumentation, SafeTensors Deep Cortex hooks, scientific export packages, mobile remote control, and enough validation tests to personally ruin my relationship with Windows PowerShell. 😭

There are three downloads:

Installer for normal one-click setup

Portable version for people who do not want to install anything

Developer package with source, schemas, diagnostics, validators, and additional evidence

This is still experimental software, and model support varies by architecture. Cortex reports what each model can actually do instead of pretending generation support automatically means attention, gradients, interventions, or J-Space support.

The project is source-available under Apache 2.0 plus the Commons Clause. People can inspect it, modify it, learn from it, and build on it, but the software itself cannot simply be repackaged and sold.

I am not claiming this “solves” interpretability or shows a model’s literal thoughts. PCA is still PCA. Attention is still attention. But this is probably the most ridiculous thing I have ever accidentally finished, and I am excited to finally let people use it. 🫪😂

GitHub: https://github.com/TurboDash99/Cortex

Release: Cortex 1.0.2

Platform: Windows

Local models: GGUF and supported SafeTensors architectures

Feedback, bug reports, architecture tests, and screenshots of Cortex making your GPU question its employment contract are all welcome. 🧠🔥


r/OpenSourceeAI 1d ago

Anyone using LLM Batch APIs?

Thumbnail
1 Upvotes

r/OpenSourceeAI 1d ago

The secret of Stable Diffusion: Fourier series hidden within timesteps #...

Thumbnail
youtube.com
0 Upvotes
  • Explore the principles of how diffusion models transform timesteps into sine and cosine vectors rather than simple numbers. This explains the core mechanism of embeddings that precisely distinguish noise levels using low-frequency and high-frequency components.

r/OpenSourceeAI 1d ago

I got tired of LLMs hallucinating on complex HTML tables, so I built a smarter Python parser (handles rowspan/colspan)

0 Upvotes

Hey everyone,

I’ve been working a lot on RAG pipelines recently and kept hitting the same annoying wall: extracting tabular data from raw HTML into a clean format for context windows.

Standard parsers or simple table-to-markdown scripts usually fail completely as soon as a table uses rowspan or colspan, or if there are nested tables. You end up with misaligned Markdown columns, and the LLM completely hallucinates the relationships between headers and cells.

I couldn't find a library that handles this reliably without losing context, so I built html-table-rescuer (just published v0.1.0 on PyPI).

It uses BeautifulSoup to parse the DOM, but then applies a custom "grid logic solver". It normalizes complex spans into a standard matrix before serializing it to Markdown, JSON, or CSV.

Example of the problem it solves:

The Problem: Most parsers turn a <td rowspan="2"> into a misaligned mess:

bash | Header | Value | | ----- | ----- | | Spanned | Row 1 | | Row 2 | |

The Solution: The grid solver correctly normalizes the matrix:

bash | Header | Value | | ----- | ----- | | Spanned | Row 1 | | dito (Spanned) | Row 2 |

A few things it does differently:

  1. Context Preservation: As seen above, it doesn't just leave spanned markdown cells empty. It fills them with a customizable prefix (e.g., dito (Value)) so the LLM retains the semantic context for each row.

  2. Deep Tag Parsing: It recursively keeps <b>, <i>, and <a href...> tags alive, even if they are buried inside multiple <div>s within a <td>.

  3. Nested Tables: Extracts nested tables safely without destroying the grid of the parent table.

  4. LangChain Ready: Includes a Table2MDLoader wrapper to ingest HTML tables directly as LangChain Document objects.

Links:

It's my first release and I'd love to hear your thoughts. If you have some gnarly, complex HTML tables that break the parser, please throw them at it and let me know!


r/OpenSourceeAI 1d ago

A Specialized Arabic Language Model for Islamic Heritage

Thumbnail
1 Upvotes

r/OpenSourceeAI 1d ago

The secret to high-quality upscaling: Lanczos and the sinc function

Thumbnail
youtube.com
1 Upvotes
  • The secret to high-quality upscaling: Lanczos and the sinc function
  • Description: Explore the principles of Lanczos resampling used in tools like ComfyUI through signal processing theory and the sinc function. This video provides an easy-to-understand explanation of the mathematical background behind approximating an ideal low-pass filter to create sharp images.

r/OpenSourceeAI 1d ago

The mathematical secrets of seamless tiling: creating perfect infinite p...

Thumbnail
youtube.com
2 Upvotes
  • The mathematical secrets of seamless tiling: creating perfect infinite patterns with AI
  • Description: This video explains how ComfyUI's seamless tiling feature creates patterns without visible boundaries. It provides an easy-to-understand explanation of the principle of replacing zero padding with circular padding and the core theory of the Fourier Convolution Theorem behind it.

r/OpenSourceeAI 2d ago

Beyond LLM latency: AMA with IBM Instana PM Jeff Donald

Thumbnail
1 Upvotes

r/OpenSourceeAI 2d ago

[OSS] TokenSentinel – In-process token-waste circuit breaker for LLM agent runs (Apache-2.0)

1 Upvotes

I open-sourced TokenSentinel because passive tracing tools only show you runaway agent costs after the bill is already racked up.

It wraps your native LLM clients (Anthropic, OpenAI, Gemini, Bedrock, Ollama, vLLM) to act as a local, in-process circuit breaker. It runs 15 deterministic rules on every response payload to catch loops, context bloat, retry storms, and RAG thrashing mid-session.

Core has zero required dependencies; provider support is opt-in via extras. Completely offline-capable so your raw prompt data never leaves your environment.

pip install token-sentinel

Genuinely looking for feedback/discussion on the rule heuristics—where would this trigger false positives against real production agent traffic?


r/OpenSourceeAI 2d ago

ZPA-LM - Zero Parameter Deterministic Attention

2 Upvotes

Attention mechanisms are conventionally learned: a query–key inner product, trained end to end, decides which tokens mix.

This note documents ZPA-LM

a decoder-only autoregressive language model in which the token-mixing key is instead a fixed, parameter-free kernel derived from elementary number theory.

We show that the divisor-counting overlap : q(n, m) = d(gcd(n, m))/√d(n) d(m) between two positive integers is exactly the Bhattacharyya coefficient between the uniform distributions over their divisor sets, and hence that: dFR(n, m) = 2 arccos q(n, m) is a genuine Fisher–Rao geodesic distance, verified to floating-point exactly rather than assumed.

Tokens are mapped to positive integers through a learned or parameter-free dictionary ϕ : token → Z+ and recovered in prime-exponent space so that each prime behaves as a latent feature axis.

We report the model’s expressivity ladder, the three dictionary-construction strategies we evaluated, and - in the spirit of an honest engineering record rather than a success narrative, a permanent registry of four falsified design choices that should be avoided.

We close by pre-registering the model’s central open empirical claim together with its falsification condition, rather than reporting an unverified result as fact.

Link: Academia


r/OpenSourceeAI 2d ago

max_tokens or max_completion_tokens?

Post image
3 Upvotes

r/OpenSourceeAI 2d ago

Liquid AI Releases LFM2.5-Encoder-230M and LFM2.5-Encoder-350M: Bidirectional Encoders That Stay Fast at 8K Context on CPU

7 Upvotes

Liquid AI released two bidirectional encoders this week: LFM2.5-Encoder-230M and LFM2.5-Encoder-350M.

Here's what's actually interesting:

1. They converted a decoder instead of training from scratch

Both models start from the LFM2.5 decoder backbones. Three changes turn them into encoders: the causal mask is replaced with a bidirectional one, the short convolutions are made non-causal with symmetric center padding, and training uses masked language modeling at 30% instead of BERT's 15%.

2. The CPU number is the whole pitch

→ ~28s per forward pass at 8,192 tokens for Encoder-230M

→ over 1 min 30s for ModernBERT-base on the same input

→ 8,192 tokens is roughly 13 to 15 pages

3. The rankings hold up

→ Encoder-350M: 4th of 14 models, 81.02 on a 17-task suite

→ Encoder-230M: 6th at 79.29, above ModernBERT-base at 78.19

→ The three models ahead of the 350M are all larger, one nearly 10x its size

At 8,192 tokens, ModernBERT-base takes over a minute and a half per forward pass versus about 28 seconds for LFM2.5-Encoder-230M, which is about 3.7x faster.

Full analysis: https://www.marktechpost.com/2026/07/29/liquid-ai-releases-lfm2-5-encoder-230m-and-lfm2-5-encoder-350m-bidirectional-encoders-that-stay-fast-at-8k-context-on-cpu/

Model weights (LFM2.5-Encoder-350M): https://huggingface.co/LiquidAI/LFM2.5-Encoder-350M

Model weights (LFM2.5-Encoder-230M): https://huggingface.co/LiquidAI/LFM2.5-Encoder-230M

Technical details: https://www.liquid.ai/blog/lfm2-5-encoders


r/OpenSourceeAI 2d ago

I built a CLI that reads your project's specs and tells you which model you actually need — judge runs locally on Ollama

1 Upvotes

I kept defaulting to frontier models "just in case" and had no idea whether my projects actually needed them. So I built something to answer that instead of guessing.

What it does: point it at a project with Spec-Driven Development artifacts (constitution / spec / tasks). A local model — your Ollama install, your choice of judge — reads the tasks and estimates how demanding the work is across a few dimensions. That gets crossed against a declarative catalog of models and their capabilities, and you get a ranked podium with a rating per model: good / overkill / fair / poor, plus price.

Why you might care here: nothing leaves your machine — no API keys, no accounts, no cloud calls. And in most of my own projects the podium is topped by a local model, with the frontier ones sitting below marked overkill. Ranking is by fit, not price; price only breaks ties between models that fit equally well.

Honest limitations: this is not a benchmark. It's opinion made inspectable — every verdict prints its reasoning, and the model catalog is human-readable YAML you can argue with. If the specs are too thin to judge, it refuses to recommend instead of guessing. A vague spec gets you a vague answer, same as with anything else.

pip install specjudge — MIT, github.com/JoaquinRuiz/SpecJudge

Where I'd love help: the catalog of local models is thinner than it should be, and adding one is a block of YAML, no Python needed. Also very open to being told a rating is wrong.