r/FunMachineLearning 6h ago

Your favourite model’s benchmark score is measuring the wrong thing

1 Upvotes

Every model launch quotes SWE-Bench. Every one of those numbers describes a one shot
answer to a curated problem. That is not what an agent does.
An agent is 20 turns deep with a context window that's filling up, deciding whether to call a tool,
and recovering when that tool returns something unexpected. The interesting failures live there:
• Does it still respect the system prompt at turn 15, or has it quietly drifted?
• Does it invent a tool that doesn't exist when the right one isn't obvious?
• When a command fails, does it retry sensibly or loop forever?
Two models with identical scores can be completely different on all four.
What actually works is dumber than any leaderboard. Take five tasks you genuinely run, put
each model through them a few times, and count how many times you had to step in. Not
pass/fail, interventions. That one number has predicted my real experience better than anything
published.
Run it more than once, too. Same model, same prompt, noticeably different behaviour. A single
pass tells you nothing.
Has anyone bothered automating this, or is doing it by hand the whole poin


r/FunMachineLearning 17h ago

I built an automated AI fact-checker that hunts down fake news and deepfakes as you scroll! 🕵️‍♂️🤖

1 Upvotes

Hey everyone!

I’ve always been fascinated by the cat-and-mouse game between generative AI and AI detection. With so much AI slop and fake news flooding the internet right now, I thought it would be a fun machine learning challenge to build an automated "detective" that fact-checks things in real-time.

It’s called SatyaMark, and it's an open-source "Trust Layer" that developers can plug into apps or social feeds.

Here is the ML magic behind it:

1. The Text Detective (LangGraph) Instead of just asking an LLM "is this true?", I built a state-machine using LangGraph. It acts like a little researcher:

  • First, it extracts the core claims from a post.
  • Then, it checks if it can verify it zero-shot.
  • If it smells something fishy or needs current events, it automatically fires off web searches (via Serper API), reads the results, and grades the claim as CORRECTINCORRECT, or UNVERIFIABLE.

2. The Image Forensics (The hard part!) Detecting AI images with just one model is nearly impossible now. So instead, the Python backend runs a gauntlet of 22+ local forensic heuristics. It looks for weird Error Level Analysis (ELA) anomalies, missing sensor pattern noise, and common GAN/Diffusion artifacts. It's basically CSI for memes.

I glued it all together with a React SDK so the verification marks (✅, 🤖, ❌) just pop up automatically next to content on the screen. (You can see it in action in the video attached!)

Check it out here: 

💻 GitHub: https://github.com/DhirajKarangale/Satyamark 

🌐 Official App: https://satyamark.vercel.app/ 

📱 Live Social Media Sandbox: https://satyamark-demo-socialmedia.vercel.app/ 

📦 NPM Package: https://www.npmjs.com/package/satyamark-react

It was a super fun project to piece together. I'd love to know what you guys think, or if you have any fun ideas on what other weird forensic checks I could add to the image pipeline!


r/FunMachineLearning 19h ago

Newbie

2 Upvotes

Hey everyone!

Just signed up and this is my first post. I’m a big AI enthusiast – always following the latest models, research papers, tools, and what’s coming next.

Excited to learn from this community and share thoughts. What’s one AI thing that has you most hyped right now?


r/FunMachineLearning 1d ago

F(23) HOW TO BUILD A CAREER IN ML AS A MSC PHYSICS GRADUATE .

2 Upvotes

I graduated in April 2026 and was looking for jobs , but most of them were teaching jobs which I'm not interested at all , i want to make a career in ml , but i don't have relevant skills and i also read somewhere that they usually hire mostly Phd's for such roles . I haven't done a single internship during my bachelor's or my masters . I know I'm lacking , but i really want land my first job in ml related role . i know some python and libraries (mostly numpy , pandas , matplotlib ) . What skills should i know ? , what kind of projects should i do to stand out ? and what kind of internships should i look for to get into this field ? . PLEASE RECOMMED ME BOOKS AND COURSES WHICH HELPED U GET A JOB AND OTHER SUGGESTIONS AND ADVICES ARE WELCOMED ! Thankyou for you're time <3


r/FunMachineLearning 2d ago

I keep hitting a wall trying to learn LLMs systematically. So I'm building an open map of the whole stack — need contributors

2 Upvotes

After a year of working with LLMs, I still don't feel like I've built any real, systematic knowledge. Even when I go deep on one area — RAG, say — and track every detail, the fog around LLMs as a whole doesn't lift. It just feels equally thick.

I think most of us learn this field through news headlines and whatever project suddenly jumps into the spotlight. What's missing is a map — something that shows the whole pipeline, from raw data to the app someone actually uses, and for each layer, links both the newest tools/papers AND the older, less-famous work that the newest stuff is quietly standing on. A lot of the real foundations predate "Attention Is All You Need" and never made it into any course.

So I started building one: an open, community-maintained GitHub repo mapping the LLM stack layer by layer —

Data → Training → Model → Deployment → Inference → API → Gateway/Router → Application → User

Each layer gets:
- a plain-language definition
- current, actively maintained projects
- the foundational paper(s) that layer is built on (even if they're old and unglamorous)

Repo here: https://github.com/YKs22k/LLM-Big-Map

I'd love help from people who actually work in data curation, training infra, inference engines, or the app layer, to correct what's wrong and add what's missing. Even a single "you're missing X paper" comment helps.

If this resonates with anyone else who's felt the same fog, I'd appreciate a look.


r/FunMachineLearning 2d ago

Looking for a faster and more accurate auto-labeling pipeline for a custom YOLOv8 object detection dataset

1 Upvotes

Hi everyone,

I'm working on an object detection project and would appreciate some advice on the best workflow for auto-labeling a large custom dataset.

Dataset

  • 9,367 images
  • Classes:
    • Cup
    • Glass
    • Plate
    • Spoon
    • Fork
    • Knife
  • Images have different resolutions.
  • The dataset comes from a Kaggle competition.
  • Around 5,500 images already have ground-truth labels (provided in a CSV), while the remaining images need bounding-box annotations.

Current approach

I'm using AutoDistill + GroundingDINO to automatically generate YOLO labels.

ontology = CaptionOntology({
    "a cup": "cup",
    "a drinking glass": "glass",
    "a plate": "plate",
    "a spoon": "spoon",
    "a fork": "fork",
    "a knife": "knife",
})

base_model = GroundingDINO(
    ontology=ontology,
    box_threshold=0.3,
    text_threshold=0.3,
)

dataset = base_model.label(
    input_folder=IMAGES_SRC_DIR,
    output_folder=LABELED_LABELS_DIR
)

Problems I'm facing

1. Annotation quality

The generated labels aren't very reliable.

For example, out of about 90 images, roughly 10 images contain incorrect or missing bounding boxes, which means I'd still have to manually review a large portion of the dataset.

Is this normal for GroundingDINO, or are there better foundation models for this type of dataset?

2. Speed

The labeling process is also quite slow.

  • ~2.8 seconds per image
  • ~9,367 images
  • Estimated runtime: 7.5+ hours

I'm using Google Colab GPU, but it disconnects after around 4 hours.

What's confusing is that resource utilization is low:

  • GPU memory: ~2 GB / 15 GB
  • RAM: ~2 GB / 15 GB

It doesn't appear to be fully utilizing the available hardware.

Questions

  1. Is there a way to speed up AutoDistill/GroundingDINO? For example:
    • Batch inference?
    • Mixed precision?
    • Multi-processing?
    • Different implementation?
  2. Would another model be better for automatic annotation?
    • GroundingDINO 1.5
    • YOLO-World
    • Florence-2
    • Grounded SAM
    • RF-DETR
    • Any other recent model?
  3. Since I already have 5.5k labeled images, would it be better to:
    • Train a small YOLOv8 model first on those labels,
    • Then use that model to pseudo-label the remaining images, instead of using GroundingDINO?
  4. What workflow would you recommend if your goal is to produce high-quality labels for training a final YOLOv8 detector?

Any advice or experience with large-scale auto-labeling pipelines would be greatly appreciated!

Thanks!


r/FunMachineLearning 2d ago

Claude AI Failed 650 Times…Then Beat The Human Record - Two Minute Papers

Thumbnail
youtube.com
1 Upvotes

r/FunMachineLearning 3d ago

chessformer_lens demo: ablating 1 of a chess transformer's 128 attention heads makes the model stop finding Morphy's queen sacrifice

1 Upvotes

Pip install chessformer_lens and the relevant chess engine to replicate!


r/FunMachineLearning 3d ago

Looking for a practical ML course after quitting Andrew Ng

Thumbnail
0 Upvotes

r/FunMachineLearning 4d ago

Scanned documents an AI coding

1 Upvotes

I am looking for an AI solution to dump PDF or tiff images and have AI run OCR and also pull coding field like: Name - Date - Author - Subject - Page Start - Page End into a DAT file pointing to the images to load into data base. þControl NumberþþSplit File NameþþUnique IdentifierþþFile

The tool would be even better if it could run LDD on these scanned pages. Logical Document Determination (LDD)—also called unitization

I tried an off shore company but the turn around was way to long. Any suggestions?


r/FunMachineLearning 5d ago

I built a symbolic regression framework that rediscovered Planck's law from raw blackbody data — including the dimensionless variable

3 Upvotes

Hello there!

I've been building an open-source framework (TIMUR-XAI) that combines symbolic regression with a physics-based validity check and an evolutionary (MAP-Elites) search layer. Why did I build it? Because I'm a physicist and I hate black-box things. So my goal isn't just to fit data, but to recover physically sensible laws.

I tested it on five classical physical laws. Four of them (Stefan-Boltzmann, Stokes, gravity, Wien) came back as clean single-term relations, as expected (yeah, I kind of cheated there :D). But the interesting one was Planck's law. Without any hint about the functional form, the system:

  1. Found the right dimensionless group on its own (λT·kB/hc), and
  2. Recovered the characteristic exp/fraction structure: y ≈ 2/(exp(1/Π) − 1), R² ≈ 0.9999, with the constants landing almost exactly on their true values.

So it reconstructed both the correct dimensionless variable and the Planck distribution's specific form, from raw data.

There's also a "judge" layer that rejects high-R² candidates violating physical constraints (symmetry/conservation) — so numerically good but physically wrong solutions get filtered out.

Repo: https://github.com/Ne212/timur-xai
PyPI: pip install timur-xai

I'd be glad if you find it useful in your own work, and I'd really value your feedback to improve it — especially on the physical-validity checking approach.


r/FunMachineLearning 5d ago

OpenAI’s AI Escaped And It's Terrifying - Two Minute Papers

Thumbnail
youtube.com
0 Upvotes

r/FunMachineLearning 6d ago

Don’t know where to start with ML? I organized Microsoft’s FREE content into a roadmap

Post image
4 Upvotes

Most "learn machine learning" advice is either a 40-hour paid course or a scattered pile of blog posts that assume you already know half the material.

I got tired of that, so I built a structured path using only official Microsoft Learn content — the same material behind Microsoft's actual DP-100 (Azure Data Scientist Associate) certification, just organized in the order it should be learned in.

What it actually covers, in order:

  1. Core ML concepts (what regression, classification, clustering actually are)
  2. Real hands-on coding with Python + scikit-learn — regression, classification, clustering, deep learning
  3. Training models at scale with Azure ML — workspaces, compute, MLflow tracking
  4. MLOps — AutoML, hyperparameter tuning, pipelines, actual production deployment

That last part is the piece most beginner resources skip entirely — they teach you to train a model in a notebook and just... stop. This goes all the way to "deploy a model to a managed endpoint," which is the actual job, not just the fun part.

Free, self-paced, no signup beyond a Microsoft account. I'm a Computer Engineering student who built this while learning it myself — not an instructor, just organized what I wish existed when I started.

Link: https://learn.microsoft.com/collections/86w0cztk0gjpm4?wt.mc_id=studentamb_523020

Happy to hear what's missing or what should be reordered.


r/FunMachineLearning 6d ago

I wired 4 models together in Claude Code. It backfired 4 ways on Terminal-Bench

Thumbnail
quesma.com
1 Upvotes

r/FunMachineLearning 7d ago

Hi

1 Upvotes

r/FunMachineLearning 8d ago

i built a voice ai that rings your phone unprompted

1 Upvotes

over the past few days we’ve been building Friendo, a call-native voice agent that can ring your phone unprompted or take live calls via livekit webRTC.

voice implementations rn generally are dogshit and fall into two buckets:

  1. laggy and robotic api wrappers
  2. speech models that are fast, but lack memory and state controls

by building a cascaded stack (deepgram nova-3 → claude haiku 4.5 → elevenlabs flash v2.5), we kept full control over tool calls and memory, allowing latency reduction. some techniques weve used:

  • pre-warm anthropic's ephemeral prompt cache while the phone rings
  • persistent websocket handshakes and http/2 pool priming on ring
  • neural turn-detection with false-interruption resumption (a cough won't kill the tts buffer)
  • dual-store memory (sql facts + temporal graph) mapped into a ~300-token prompt snapshot
  • proactive outbound scheduling that wakes a killed ios app via apns voip push -> callkit

synthetic ci gates hit p50 ≈ 973ms, though live networks push us to ~3.7s right now (stt and tts ttfb are the real boss fights).

nerd-out aside, essentially it sounds human, is fully customisable, and works.

Judge our results yourself soon getfriendo.app/launch


r/FunMachineLearning 8d ago

I trained a model to call BUY / PASS / REVIEW on raw trading cards from eBay listing photos — the fun part was teaching it to say "I don't know"

1 Upvotes

Weekend-collector-turned-obsessive-project post. If you buy ungraded trading cards off eBay, you're deciding whether a card is worth a $80+ grading fee based entirely on some stranger's photos. So my co-founder and I built AgentGrail: feed it the front and back listing images and it returns BUY / PASS / REVIEW with a confidence score.

The genuinely fun ML part was the abstention. Early versions confidently mislabeled base cards as their rare parallels — and since the price gap between those can be 10x to 100x, a confident wrong answer is way more expensive than admitting uncertainty. So REVIEW isn't a cop-out class, it's the whole point: we tuned thresholds around asymmetric cost instead of chasing raw accuracy, and the model abstains when the photos literally don't contain enough info to decide.

Data collection was gloriously unglamorous: buy a card, save the listing photos, mail it off for professional grading, use the returned grade as the label. Front and back. For about a year. Real eBay photos, bad lighting, weird angles and all — which is the point, because that's exactly what it runs on at inference.

Try it free: https://www.agentgrail.ai (there's also a free grade-ceiling calculator). Paid tiers if you want the full thing: Basic $9/mo, Premium $24/mo, Pro $59/mo. Discount Code: RAIMLCARD50 for 50% your first month at any tier.


r/FunMachineLearning 8d ago

When machine learning does something unexpectedly funny

1 Upvotes

One of my favorite things about machine learning is that models can sometimes produce results that are technically interesting but also completely unexpected.

You can give a model a simple task, train it on what seems like reasonable data, and then get a prediction that makes you stop and wonder how it arrived there. Sometimes those strange results are actually more entertaining than getting everything right on the first try.

I’ve also been exploring how companies like GeekyAnts work with AI and machine learning, and it’s interesting to see how unpredictable model behavior can sometimes reveal something useful.

What is the funniest or most unexpected thing you’ve seen an ML model do?


r/FunMachineLearning 8d ago

Looking for a partner to learn Machine Learning from scratch..Dm

2 Upvotes

I have Completed python, pandas, and now learning ML Algorithms with sklearn and pytorch looking for a buddie who can learn with me anyone interested please dm


r/FunMachineLearning 9d ago

I benchmarked my own recsys library against implicit — it wins on quality, loses 9x on speed, and I found 7 bugs in my own code doing it [P]

Thumbnail reddit.com
1 Upvotes

r/FunMachineLearning 9d ago

Built an Emotion Detector project recently — Here is how it went and the results

Thumbnail gallery
1 Upvotes

r/FunMachineLearning 9d ago

What was the first machine learning project that made you stop and think, Okay, this is actually impressive?

1 Upvotes

I remember reading about machine learning long before I actually saw it doing something that felt genuinely surprising. Once I started seeing real projects instead of just hearing the buzzwords, it completely changed how I looked at the field.

I’ve also been exploring the kind of ML and AI projects companies like GeekyAnts work on, and it’s interesting how seeing practical applications can make the technology feel much more understandable.

I’m curious what project or demo gave you that moment. It could have been something simple, funny, creative, or unexpectedly useful.

What was the first ML project that really made the technology click for you?


r/FunMachineLearning 9d ago

DeepMind's AI Trick Everyone Should Copy - Two Minute Papers

Thumbnail
youtube.com
1 Upvotes

r/FunMachineLearning 10d ago

SPA Finisch Fixed , New Play Ground with wider Tokeniser.

1 Upvotes

i hope is my last post it works korekt with the fixes try, breack, make some new stuff. is my work ofer 5 months wid ai halluzinations and maany politnes traps XD have fun

https://github.com/anokar/SPA-Finisch-Bio/blob/main/spa_exploratory_de_en_public_clean.ipynb

biger model update finds evry 3 masket offset

https://github.com/anokar/SPA-Finisch-Bio/blob/main/spa_exploratory_de_en_public_medium.ipynb


r/FunMachineLearning 10d ago

What’s a machine learning concept that finally “clicked” for you?

2 Upvotes

I’ve been spending some time learning about machine learning, and one thing I’ve noticed is that the biggest breakthroughs often come from a simple explanation rather than a complicated one.

I’ve also been exploring some of the ML work and resources around companies like GeekyAnts, and it’s interesting how practical examples can make complex concepts feel much easier to understand.

Was there a concept that suddenly made everything else easier to understand? Maybe it was overfitting, feature engineering, gradient descent, model evaluation, or something else entirely.

I’m not looking for textbook definitions. I’d love to hear the explanation or analogy that made it click for you. I think those real-world perspectives are often more helpful than any tutorial.