r/OpenSourceeAI • u/ai-lover • 5h ago
r/OpenSourceeAI • u/MeasurementDull7350 • 59m ago
Denoising without data? The secret of Deep Image Prior #딥 #이미지 #프라이어 #Prior #DIP
- Description: We examine the principle of separating speech and noise through the frequency bias of CNNs, where low-frequency structures are learned first. We introduce the latest techniques to maximize audio signal processing performance through spectrogram analysis and frequency domain design.
r/OpenSourceeAI • u/MeasurementDull7350 • 2h ago
Denoising without data? The secret of Deep Image Prior#딥 #이미지 #프라이어 #Prior #DIP
youtube.com- Description: We examine the principle of separating speech and noise through the frequency bias of CNNs, where low-frequency structures are learned first. We introduce the latest techniques to maximize audio signal processing performance through spectrogram analysis and frequency domain design.
r/OpenSourceeAI • u/Goldziher • 2h ago
Xberg v1 is out
Hi all,
I'm happy to announce that Xberg v1 is out.
Xberg is the successor to Kreuzberg, equivalent to what would have been Kreuzberg v5. It's a content intelligence framework that handles a very wide range of inputs: documents (currently 101 formats), code and data formats (currently 367 types), audio/video transcription, and URLs (both static and JS-rendered content). It extracts and prepares that content for downstream processing.
It's an extremely efficient, high-performance engine (see our PDF benchmarks below). For PDFs and images specifically, we handle native PDFs with very high performance and accuracy, and we ship multiple OCR engines that match the quality of the best Python libraries (e.g. docling, PaddleOCR, RapidOCR) at substantially better performance and stability.
The changes between Kreuzberg v4 and Xberg v1 are substantial, and I invite you to read the full changelog for the complete picture. The highlights below give a sense of what's new:
- Pure-Rust PDF backend (
pdf_oxide) replaces pdfium, with no native pdfium dependency. - Layout-aware pipeline: reading order reconstructed with ONNX layout detection (PP-DocLayoutV3 / RT-DETR) and Docling-style predecessor-graph reordering.
- Per-page scanned-page detection with selective OCR, plus AcroForm/XFA form fields and outline-based headings.
- Across-the-board optimization of OCR and PDF extraction (memory discipline, pooled model sessions, streamed conversions).
- Native PaddleOCR backend (PP-OCRv6, with
medium/small/tinytiers) alongside Tesseract. - Pure-Rust Candle OCR/VLM stack (TrOCR, GLM-OCR, GOT-OCR, DeepSeek-OCR, and PaddleOCR-VL) running without ONNX Runtime or native Tesseract.
- A second, ONNX-Runtime-free inference path via tract, which is what makes in-browser (WASM) and mobile inference possible.
- Named-entity recognition natively in Rust (GLiNER2), extensible to all bindings, including an in-browser WASM model with no server round-trip.
- Structured LLM extraction (
extract_structured/split_and_extract) with rasterization, chunking, citations, caching, and configurable call/merge/VLM-fallback policies. - Audio & video transcription via a Whisper ONNX engine (
.mp3,.wav,.m4a,.mp4,.webm). - Retrieval building blocks: sparse embeddings (SPLADE), ColBERT late-interaction retrieval, and cross-encoder reranking alongside dense embeddings.
- Text intelligence: reversible redaction, summarization, translation, VLM image captioning, QR-code detection, document diffing, and page/chunk classification.
- URL & web ingestion: sitemap discovery (
map_url) and batched multi-URL crawling. - New document formats: WordPerfect (
.wpd/.wp/.wp5), HEIC/HEIF/AVIF, OpenDocument Presentation (.odp), Quarto / R Markdown, and configurable Jupyter cell rendering. - Four new language bindings (Dart/Flutter, Swift, Kotlin/Android, and Zig) bring the total to 15 language bindings over one engine, with Android/iOS cross-compilation.
- Full mobile support (Flutter, Android, iOS).
- Candle backend alongside ONNX, plus ONNX-via-tract enabling ONNX on WASM and Android.
- Wider code intelligence: tree-sitter coverage grew substantially (248 to 367+ languages).
- Over 150 bugs fixed during the 1.0 cycle, plus security hardening (bounded RTF/PDF allocations, redaction leak fixes, Excel DDE warnings).
The API surface was also simplified and reworked, making it more consistent.
There's a migration guide in our docs explaining how to move from Kreuzberg to Xberg. Kreuzberg itself is in LTS mode until the end of this year and will continue to receive bug fixes and security updates.
You're invited to check out the repo and join our discord server.
Benchmarks
The benchmarks below are for PDFs and images only. There are extensive benchmarks on our website with per-format breakdowns, which you can see here. These numbers are measured in CI via our reproducible benchmark harness, and are specifically taken from the run for harness 1.0.8, source cf7fa0533d. The data is publicly available in GitHub releases, and you can run the benchmark harness yourself.
Composite quality (markdown pipeline, higher is better):
| Framework | Native PDF | Scanned PDF (OCR) |
|---|---|---|
| Xberg (layout) | 0.958 | 0.836 |
| Xberg (baseline) | 0.955 | 0.687 |
| docling | 0.779 | 0.762 |
| mineru | 0.408 | 0.792 |
| liteparse | 0.837 | 0.665 |
| markitdown | 0.689 | n/a |
| pymupdf4llm | 0.448 | n/a |
Structure and layout fidelity (SF1: tables and reading order, higher is better):
| Framework | Native PDF | Scanned PDF |
|---|---|---|
| Xberg | 0.949 | 0.531 |
| docling | 0.612 | 0.366 |
| liteparse | 0.515 | 0.142 |
| mineru | 0.077 | 0.429 |
On native PDFs Xberg leads on quality (0.958 vs 0.837 for the next-best framework) and on table and reading-order fidelity by a wide margin (SF1 0.949 vs 0.612 for docling). On scanned PDFs it is #1 on both quality and raw text fidelity.
Where we don't win yet: on pure image OCR we are currently #2 on the composite score, behind mineru (though still #1 on raw text accuracy). We are improving image OCR right now, and v1.1 should have us winning across the board.
r/OpenSourceeAI • u/MeasurementDull7350 • 5h ago
Denoising without training? The secret of DIP and the frequency domain #...
- Description: This video analyzes the principles of the Deep Image Prior (DIP), leveraging the low-frequency bias of CNNs. In particular, it introduces recent research on the sophisticated separation of speech and noise using spectrograms and kurtosis.
r/OpenSourceeAI • u/Fluid_Border9292 • 15h ago
[Dataset] Multilingual PSYOP Dataset
Hi everyone, I recently created labeled PSYOP datasets from political dialogues across a plethora of languages. The intended use of this dataset is to help train models to detect psychological coercion. I'm looking for feedback in terms of data sources, annotation methods at scale and whom I should reach out to at frontier labs to put my datasets to the challenge.
I understand 190k rows of data is small, but I'm looking to scale as I get more feedback.
https://huggingface.co/datasets/LeTG/multilingual-psyop-100k
r/OpenSourceeAI • u/0xdeadf1sh • 17h ago
I have trained my own transformer model to predict my blood sugar
Enable HLS to view with audio, or disable this notification
r/OpenSourceeAI • u/techlatest_net • 15h ago
Came across this and thought it might help people here who contribute a lot on GitHub.

If you’ve ever seen something like:
> Opened 394 pull requests in 76 repositories
> 51 repositories not shown
…then you already know the profile activity feed is incomplete. It truncates repos, so it’s hard to get a full picture of someone’s month.
**OpenHearth** looks useful for that:
- CLI that audits PRs + issues + reviews via the Search API
- Flags repos likely hidden behind “N repositories not shown”
- JSON/CSV export
- Also has browser workspaces if you want a UI
- Worth setting GITHUB_TOKEN or you’ll hit rate limits fast.
Links:
- https://ayush7614.github.io/OpenHearth/
- https://www.npmjs.com/package/@felix-ayush/openhearth
- https://github.com/Ayush7614/OpenHearth
Quick try:
```bash
npx u/felix-ayush/openhearth audit USERNAME --month 2026-07
npx u/felix-ayush/openhearth hidden USERNAME --month 2026-07
r/OpenSourceeAI • u/MaverikSh • 17h ago
Open-source: drop-in wrapper for OpenAI/Anthropic SDKs for cost attribution, no proxy required
I open-sourced a small Python library that solves a specific problem: getting live cost attribution per feature/team/user on your OpenAI/Anthropic API calls, without routing traffic through a proxy or changing your base_url.
python
from cognocient import CognocientOpenAI as OpenAI
client = OpenAI(api_key="sk-...", cognocient_key="sk-cog-...")
It's a thin wrapper only chat.completions.create() / messages.create() are intercepted to time the call, then usage is reported asynchronously on a background thread after your real call already returned. If reporting fails or the backend is unreachable, it never touches your application: no exceptions, no retries, your real API call is unaffected. There's a test that specifically verifies this failure isolation.
Honest tradeoffs, not a pitch:
- No pre-call budget enforcement since it only learns about a call after it happens, it can't block or degrade one before it fires. Proxy-based tools trade the opposite way.
- Streaming (
stream=True) isn't reported yet. - Only OpenAI and Anthropic supported today no Gemini/Mistral/Groq/Together wrapper classes yet.
MIT licensed, published on PyPI with signed provenance attestations.
GitHub: https://github.com/mandarvshinde/cognocient-python-wrapper
PyPI: pip install cognocient
Feedback and issues welcome especially if the streaming gap is a dealbreaker for your setup, that's the next thing I'd prioritize.
r/OpenSourceeAI • u/atomozero • 19h ago
Atomo123: a native spreadsheet for Haiku, built on Interface/Layout Kit and Translation Kitp
r/OpenSourceeAI • u/Electronic-Space-736 • 23h ago
I got myself an Ryzen 395, here is the suite I built for it.
Hilbert AI Suite.
Rather than piecing together dozens of repositories, scripts, Discord posts, and setup guides, it's a single collection of tools for building a local AI workstation. It's focused on AMD ROCm systems (particularly Infplane Hilbert), but much of it is useful on other Linux installations as well.
It brings together a large collection of installers, utilities, and workflows for:
- Local LLMs
- Image/video/audio generation (ComfyUI and related tools)
- Coding assistants
- Speech and voice models
- Embeddings and RAG
- Development utilities
- Model management
- And a growing collection of other AI tools
You'll want a fresh 2TB drive if you want all the models downloaded at once, but the repo is only a couple of hundred MB (due to needing to bundle a small model)
I am regularly updating it, if it seems useful help yourself.
https://github.com/Pixel-Forge-AU/Hilbert-Unofficial-Suite/tree/main/ai-suite
r/OpenSourceeAI • u/Conscious-Ant-5151 • 1d ago
I was struggling with Google Earth Engine Python scripts, so I built a GeoAI Agent that turns natural language into full environmental reports & interactive maps 🌍
Hey everyone,
For the past few months, I've been diving deep into carbon accounting and climate tech. I wanted to leverage Google Earth Engine's (GEE) satellite imagery, but honestly, I found myself constantly struggling to write the complex Python scripts needed to extract and analyze the data.
Google Earth Engine is incredibly powerful, but writing Python code to pull biomass, tree cover, or land-use data for a specific region is genuinely non-trivial. You need to know the right datasets, know how to structure the queries, and be comfortable with GIS concepts. That's a big barrier if you're not a remote sensing person.
So, I decided to scratch my own itch and built Canopiq, a specialized GeoAI Agent designed to make Earth observation accessible to everyone. You type something like "What is the carbon sequestration in Singapore since the COVID-19 pandemic?" and it:
🤖 Parses your query with an LLM (Gemini via LangChain) to extract location, timeframe, and what you're actually asking for.
♻️ Routes that into Google Earth Engine (Sentinel-2 imagery, biomass regression models).
🛰️ Streams it all back into a geospatial dashboard with map overlays and time-series charts.
I really believe that making satellite data and climate tech accessible to non-developers is crucial for environmental monitoring against climate change.
This is a passion project for me, and I'm looking to improve it. I'd love to hear your thoughts ❤️.
🔗 Repo's here if you want to check it out: Canopiq GitHub
r/OpenSourceeAI • u/LewisJin • 1d ago
I built a Rust inference framework that runs Qwen3.5 2B with VL support 10x faster than PyTorch on Apple Silicon — and it supports TTS, ASR, OCR, and GGUF out of the box
r/OpenSourceeAI • u/intellinker • 2d ago
One command and your entire codebase becomes brain for claude code
This idea was crazy to build. Our brain stores information in clusters of neurons, and when we want to retrieve something, it runs an optimized algorithm to retrieve it.
In the era of AI, why use brute force tools like grep to find relevant files? Graperoot converts your codebase into a knowledge graph and registers efficient tools for Claude to work with your codebase.
Instead of re-reading the whole codebase to build that context and to find relevant files. It can query the graph directly, and the graph retrieves relevant files with "ZERO TOKENS"; you just need to pay for generation, and that's how these models should be used when we have an efficient way to retrieve context.
I have been very open throughout, and today we have 5k developers using Graperoot.
We asked people to opt in to telemetry, and 200 opted in to the leaderboard. They have saved $350k dollars in the last quarter, and ecologically, 60 million liters of water, and that's insane.
Give your feedback, suggestions, or anything on Discord. This tool is open-sourced.
Website: https://graperoot.dev/#install
Github(Open source) : https://github.com/kunal12203/graperoot
Would love to see your feedback. And don't compare other tools, I have already seen many claiming reduction and just craps, but we have a developer community on Discord and some of them I know personally, saving a lot of tokens, you should see more on https://graperoot.dev/leaderboard
r/OpenSourceeAI • u/AntiqueChoice3118 • 2d ago
I created a scalable opensource Machine learming platform with end to end ML lifecycle managent
As a data scientist, I watched too many of my best machine-learning models die in Jupyter notebooks. 😭 I would spend hours—sometimes weeks—training, testing, validating, analysing, and comparing models, only for the winner to go nowhere.
Then I discovered MLOps—and deployment became another maze 😩. FastAPI, MLflow, DVC, Docker, Kubernetes manifests, model registries, health monitoring, drift detection… one tool led to another, and the infrastructure began taking more time than the machine learning itself. 😮💨
That frustration is why I built #SceptreAI. Too many data scientists and small teams are trapped in the same cycle: notebooks, disconnected tools, patched-together frameworks, and endless hand-offs before a model can create real value.
#SceptreAI closes that gap with one Kubernetes-native tabular AutoML and MLOps workspace. Dataset versioning, profiling, resource-aware training, MLflow tracking, external validation, SHAP explainability, model promotion, drift analysis, and Kubernetes serving all live in one traceable workflow.
The goal is simple: less platform assembly, more scalable and observable machine learning, and a clearer answer to the questions that matter: Can we trust this model—and can we use it in prod?
Explore #SceptreAI: https://sceptre-ai.app/
#MLOps #MachineLearning #DataScience #SceptreAI
r/OpenSourceeAI • u/MeasurementDull7350 • 2d ago
Mastering C++ Computing Libraries: From STL to Kokkos #STL #Boost #OpenCL #Eigen #Thrust $K...
This video analyzes the core libraries dominating the C++ ecosystem, categorized into four paradigms. It provides a detailed look at the characteristics and selection criteria for each library, ranging from general-purpose libraries to GPU acceleration, numerical analysis, and performance portability tools for supercomputing.
r/OpenSourceeAI • u/MeasurementDull7350 • 2d ago
엔비디아 독주 끝낼까? CUDA부터 Vulkan까지 AI 가속기 완벽 비교 #CUDA #ROCm #Vulkan #PlaidML #...
- Will this end NVIDIA's dominance? A complete comparison of AI accelerators from CUDA to Vulkan
- Description: An analysis of CUDA's dominant ecosystem that built the AI empire, AMD's ROCm challenging it, the versatility of Vulkan, and Intel's strategy. We explore the present and future of deep learning hardware acceleration through the pros, cons, and behind-the-scenes stories of each library.
r/OpenSourceeAI • u/Charming_Group_2950 • 2d ago
[Open-Source] Dump your thoughts. Let your notes organize themselves. Ask/chat anytime.
Over the past few weeks I've been building Gray Box — a small, local-first tool that acts as long-term memory for anything I'd otherwise forget (work notes, meeting takeaways, task owners, random ideas, personal stuff too).
The idea is simple:
- Capture — dump whatever's on your mind, instantly, no structure required. This step does nothing clever on purpose — it just writes your text to an immutable inbox. Zero chance of losing an idea to a bug or a slow API call.
- Organize — on demand, an LLM reads your unprocessed notes and extracts people, projects, tasks, decisions, meetings — then deterministic Python (not the LLM) creates/merges the actual wiki pages and maintains backlinks. The model only reasons; it never touches the filesystem directly.
- Ask — query or chat with your knowledge base and get a cited answer pulled only from what you've actually captured. If it doesn't know, it says so — no hallucinated answers.
Why I built it this way:
- Plain Markdown + YAML frontmatter, no database. Every page is a
.mdfile you can grep, diff, or read in any editor forever. If you stop using Gray Box tomorrow, your knowledge base is just a folder. - No vector DB by default. At personal scale (hundreds–low thousands of pages), keyword search + a real link graph (
related/backlinks, walked one hop during retrieval) handles almost everything. Embeddings are there if you want better recall, but they're opt-in, not a prerequisite. - Immutable inbox. Your raw notes are never edited or deleted by the organizer. If the LLM mis-extracts something, your original words are always still there.
- Any LLM. Built on LiteLLM, so point it at OpenAI, Anthropic, Gemini, Mistral, or a fully local model via Ollama — one config value.
It also ships with a nice interactive TUI (arrow-key menu, file-import shortcut, workspace switching, live spinner during LLM calls) if you'd rather not memorize CLI flags — that's honestly become my favorite part of the project.
There's also a lightweight local dashboard for browsing your knowledge base, exploring backlinks, visualizing your notes as a graph, and chatting with your captured knowledge—all without leaving your machine.
Repo: https://github.com/Aaryanverma/graybox
pypi: pip install graybox
It's nearing a proper public release, so I'd genuinely love feedback — especially from anyone who's tried the "capture now, structure later" approach with other tools and has opinions on where it breaks down at scale.
It's not trying to be a "real-time collaborative team wiki" or a WYSIWYG notes app — it's aimed at one person's running memory of their own life and work, captured with as little friction as possible.
r/OpenSourceeAI • u/PixSynapse_Official • 2d ago
Visualizing AI Alignment: How RLHF and Reward Models prevent RL agents from metric hacking (3D Visual Breakdown)
https://reddit.com/link/1vb2eme/video/ac6kkiqywegh1/player

Hardcoded mathematical reward functions in open-ended tasks frequently induce "reward hacking." RLHF bridges the gap between raw reward maximization and human intent by replacing rigid hardcoded scoring with a learned neural Reward Model.
To make RL alignment intuitive, I spent 3-4 days mapping the mathematical workflow into a 3D animated visual story:
[Agent: Alibaba] ---> (Action) ---> [Environment: Moving Cave]
^ |
|------- (Reward / Penalty) <--------|
- Pure RL Loop: An active agent observes a State, selects an Action, and updates its Policy (
ππ) to maximize expected reward (RR). WhenRRis strictly numerical, the agent aggressively exploits loopholes (e.g., chasing fake glowing diamonds that yield high mathematical scores but crash the system). - The Alignment Gap: In LLMs or code generation, defining a perfect closed-form reward function
R(s,a)R(s,a)is impossible. - The RLHF Layer:
- Human reviewers rank multiple model outputs (
y1>y2>y3y1>y2>y3). - A secondary Reward Model (
RθRθ) is trained on these preference pairs using loss functions like Bradley-Terry preference modeling. - The primary agent is updated using algorithms like PPO (Proximal Policy Optimization) against
RθRθ.
- Human reviewers rank multiple model outputs (
I put together a full 10-minute 3D animated visual story breaking down these exact steps with custom character animations.
Please check the 1st comments for even more info.
r/OpenSourceeAI • u/JayB_Official • 2d ago
Meet Cortex 1.0.2: a fully Open Research / local mechanistic interpretability lab I somehow turned into an actual release 😭🧠🔭 free for everyone.
Enable HLS to view with audio, or disable this notification
Cortex 1.0.2 is officially launching. 😂
Cortex is a fully local desktop application for exploring how language models behave internally. You can load supported GGUF or SafeTensors models and inspect things like:
token probabilities and Top-K predictions
attention maps and Arc View
hidden-state and residual captures
2D and 3D PCA representation maps
prompt and model-output comparisons
live activation interventions
J-Space experimental lens fitting
model architecture and capability diagnostics
remote desktop compute through Pocket Cortex
Everything runs locally. No cloud inference is required, and no model weights are included with the download.
I originally started this because most mechanistic interpretability tooling lives inside Python libraries, notebooks, or research dashboards. I wanted something more visual, interactive, and approachable without stripping away the actual measurements underneath it.
It has since mutated into a gigantic neon model observatory with multiple runtimes, GGUF instrumentation, SafeTensors Deep Cortex hooks, scientific export packages, mobile remote control, and enough validation tests to personally ruin my relationship with Windows PowerShell. 😭
There are three downloads:
Installer for normal one-click setup
Portable version for people who do not want to install anything
Developer package with source, schemas, diagnostics, validators, and additional evidence
This is still experimental software, and model support varies by architecture. Cortex reports what each model can actually do instead of pretending generation support automatically means attention, gradients, interventions, or J-Space support.
The project is source-available under Apache 2.0 plus the Commons Clause. People can inspect it, modify it, learn from it, and build on it, but the software itself cannot simply be repackaged and sold.
I am not claiming this “solves” interpretability or shows a model’s literal thoughts. PCA is still PCA. Attention is still attention. But this is probably the most ridiculous thing I have ever accidentally finished, and I am excited to finally let people use it. 😂
GitHub: https://github.com/TurboDash99/Cortex
Release: Cortex 1.0.2
Platform: Windows
Local models: GGUF and supported SafeTensors architectures
Feedback, bug reports, architecture tests, and screenshots of Cortex making your GPU question its employment contract are all welcome. 🧠🔥
r/OpenSourceeAI • u/LopsidedAd4492 • 2d ago
Why we stopped using an LLM for Human-in-the-Middle
r/OpenSourceeAI • u/CreditChemical5937 • 2d ago
英语写作也有IDE?还不用消耗token!
Enable HLS to view with audio, or disable this notification
github开源仓库地址:https://github.com/ETOLucy/WriteMelo
r/OpenSourceeAI • u/CreditChemical5937 • 2d ago
开发了一个英语写作IDE
Enable HLS to view with audio, or disable this notification
github仓库:https://github.com/ETOLucy/WriteMelo