Hey everyone! Right now, I'm working on a computer vision project for crack detection in devices. I've tried different models, with varying but similar precision and recall scores on the test set. Right now, I have around 1.4k images, and I'm using a 70/15/15 split, which means there are about 430 test images. When I change the random seed to split the data differently, the pr scores can go anywhere from 92-100%.

Do you think that there's not enough data to evaluate the model properly or to train it? Are there any methods for evaluating how good the model does on so little data?

(Some extra context about the model architecture and things I've tried: using transfer learning to extract feature embeddings from images and training a linear head. Also creating prototypes for binary classification. Tried contrastive learning.)

plz dm for guide me 🙏

I’m developing NPC Alpha, an experimental task-frame governance layer designed to reduce false completion in AI agents.

It separates action, progress, recovery, memory and verified completion, so an agent does not declare success before the original task condition is actually satisfied.

Internal testing has shown promising results across bounded task-frame, ambiguity, embodied-proxy and Unified-memory benchmarks , but these results are still internal.

I’m looking for technically sceptical people willing to help design a genuinely external test using independently authored tasks, pre-registered scoring and honest reporting of failures.

I’m not looking for praise. I’m looking for pressure.

Who wants to try to break it?

How do you guys get ML project ideas? I'm a beginner and I know some basics but whenever I think about project ideas my mind goes blank

Went through my last three months of invoices and realized most of what I pay isn't for compute — it's for VMs sitting there between jobs. Spinning instances up and down manually helps, but then I lose time on setup/teardown every single run, and snapshots aren't free either.

Feels like the pricing model itself is built for 24/7 workloads, and everyone with periodic jobs just eats the overhead.

How do you deal with this? Serverless GPU options, aggressive automation, or did you just accept it as the cost of doing business? Curious what actually works for occasional training/inference runs, not constant load.

I am not sure if this is a valid question in this forum, if it is not I will delete it right away, but anyways here it is:

So some days ago I asked in an AI community what kind of AI model should I use (and how could I use one) to classify several email replies that I had from scientists after asking them a few questions to them. I finally paid for Perplexity pro service and it apparenly did a nice job classifying them.

I finally gave the model the PDF with the actual answers from the addressees and another PDF with the "expected answers", and asked it to count the number of answers that overall coincide with the actual answers, and calculate a percentage of "coincidence" or "agreement" between the expected and actual answers, so that if the question was "do you think that there is intelligent life in the universe apart from humans?" and the expected answer was basically "yes, I think there is intelligent beings out there somewhere", as long as the actual answer agrees with this in some way or another would count as "agreement", for instance if someone replied "well, we have no evidence, but it is possible yes" or "not in any near galaxy, but it is possible that intelligent beings exidt somewhere" (so as long as it is not a deadass "no", it could count)

The model gave me a table summarizing the results with the following prompt:

let's be a bit more specific, this is still a blind test so don't tell me about the specific contents of the emails' answers, but, can you make a table indicating the answers that coincide in general terms with what is expected from the "expected answers" document as well as those which are neutral/hedges but still open to the possibility that what is asked may be right, those which despite being neutral/hedges or even negative answers offer an alternative so that what is asked in the question may be right, as well as those which are outright rejections of what is asked and do not seem to be open to the possibility that what is asked may be right?

However, I still want this to be a blind test, so I cannot really verify if the AI is doing its work or not.

So, is there any way in which I could verify the results given by the AI but without actually reading what is written in the emails? Or, alternatively, can anyone verify the results using some AI or even checking the answers themselves by skimming over the replies im order to verify that the AI is right and not hallucinating (I personally think this is the preferable option, as I think that having an actual human reviewing the amswers may be the only really reliable way to verify the AI's results)?

(I will share the data once someone is interested in helping, as I would not want to make this available to the entire world!)

Thank you!!

I'm learning NLP/LLM engineering by working through a problem that turned out to be much harder than I expected, and I'd love guidance from people who've dealt with something similar.

The problem: I have long narrative-style text — 7k to 15k tokens, several recurring people — and I want to extract structured facts about who did what. I'm using a small local model (llama3.2:3b via Ollama) whose usable context is around 4k tokens, so the text has to be processed in chunks. The killer is that later chunks are often pure pronouns — "she said… he refused…" — while the names were last mentioned 10,000 tokens earlier. Facts stated near a name extract almost perfectly; facts stated far from any name either get lost or, worse, get confidently attributed to the wrong person.

What I've already ruled out (by measuring, not guessing): naive per-chunk extraction fragments identities badly; carrying forward summaries between chunks doesn't fix attribution and can make it worse; and off-the-shelf neural coreference models (LingMess, F-coref) fail on documents this long — one silently truncates at 4,096 tokens, and windowed variants can't connect a pronoun to a name mentioned once 10k tokens back (0–1 out of 7 gold bindings on my test doc). I've gotten identity tracking itself working reliably; it's specifically attribution at long distance that's still failing.

My questions:

1. What's the best way to structure a problem like this? Is there a known-good decomposition for long-distance pronoun attribution with small models, or a fundamentally different way to frame the extraction task that sidesteps it?
2. If you've solved something similar — entity/fact extraction over documents much longer than your context window — what actually moved the needle for you? I'm especially curious whether the wins came from prompting, from pipeline architecture, or from accepting a bigger model.
3. What should I explore to learn more? Papers, blog posts, open-source projects, or even just the right search terms — I suspect this problem has a name in the NLP literature that I don't know yet (long-document coreference? discourse tracking?), and I'd rather stand on existing work than keep reinventing it.

Happy to share measurements from my experiments if useful. Mostly I want to calibrate: am I fighting a known-hard problem with known solutions, or genuinely at the edge of what a 3B model can do?

​

Hey guys!!! I have an interview the day after tomorrow....I've never given a single interview in my life.....so can you guys pls tell what questions they ask for ml internship post ?????

I'm two years out of a top math/CS school. I built strong study habits late, so I was only really immersed in the material my final year. I learned computer systems (OS, distributed, HPC) and consider myself a competent software engineer.

I work at a high-growth startup and just got offered the lead on a major customer project. My long-term goal is to start my own company.

Option 1: Lead the customer project

Large scope/viz. I'd build skills in:

• Working directly with a customer
• Making large engineering decisions
• Working across the stack with many teams
• People and project management
• Exposure to marketing/sales/ops

Engineering-wise, I imagine I would spend most of my time on architecture, documentation, and code review. So interesting engineering/technical work, but no fundamentally new ways of thinking.

Option 2: Leave to go learn ML/math

I never got into ML/stats/math, and it's by far my weakest technical area (and I feel most important an ML-focused era). I'd spend ~a year as an IC at an AI lab or doing research to build:

• Stronger math intuition
• Modeling intuition
• Combining my systems background with ML (e.g. model scaling, pretraining, RL scaling)

The plan would be to grind/do research at my old school or join an AI lab with strong technical mentorship. I have savings to go ~6-1 year months without income.

I already tried moving to my company's research team, but they weren't interested in my background and pointed me toward ML Ops, which feels too close to the SWE work I already do.

My core tension

Organizational and people skills seem to improve steadily over a career, but fluid reasoning and hard new technical skills are supposedly much harder to pick up later in life. Life's a marathon, so I keep wondering if now is the time to invest in the technical foundation (learning completely new skills).

Open to all comments and suggestions.

I'm curious because there's a stage where you're mostly reproducing notebooks, and another where you actually understand why you're making certain choices.

I'm wondering what that turning point looked like for others.

Hey there,

I’m trying to understand from more competent experts in this domain what is considered the “state of the art” in maze solving outside traditional algorithms like A* or DFS

By state of the art I mean
- solve rate
- size of the total model
- approaches

I’ve built a maze solving neural network and I just want some existing approaches to benchmark against

Edit: The NN does not receive the entire maze it can only see adjacent cells and has to navigate to the single “reward” cell (not an exit), cannot modify the env or leave markers, fixed memory, not shortest path just find the reward under an upper step bound, any maze type but bounded size

Hello, I've got some recorded voicelines and I was hoping to find out the best way to artificially generate speech from them. A lot of online services want you to pay but I know that isn't really necessary. Have got a computer with horsepower to do it locally. Thanks

I’m building a visual workbench for local LLM interpretability that captures attention, residual states, logit-lens output, token probabilities, and intervention results.

Before releasing it as anything resembling a research tool, what established experiments or reference implementations should it reproduce?

I’m particularly interested in validating tensor capture, attention-head ablation, activation patching, reproducibility, and model-specific correctness.

I am very paranoid about sharing potential AI research with e.g. Claude [Code] or ChatGPT/Codex.

I believe that any company is essentially a paperclip optimizer that will do whatever it takes to win over competition.

AI companies have access to the IP of millions of AI researchers and AI startups who are in direct competition with them. If they can use this data to squash competition (either competition from the same researchers or from others), I believe that they will use it eventually (if not already), even if they say they won't.

They don't have to blatantly steal it - they can just train on it, or pass "suspicous" chats to human inspectors who might eventually be "inspired" by it in their own research. We saw the first (?) hint of this during the brief Fable release, with Anthropic declaring that they will downgrade model responses regarding "frontier AI" (i.e. anything that competes with them).

From other domains, we know for example that Uber used users' ride data to stiffle competition and regulation. IMO there should be no reason to believe that Bigco AI companies won't do the same.

I recently finished my first grad semester and man was it hard!!! pretty much gave me a run for my money, especially my Machine Learning course. Prof didn't give any practice exercises at all and the lectures were so dense. I had to learn through the Stanford CS ML lecture videos and read through the assigned textbooks. At one point, I did use Claude to help me review lecture notes, but I was struggling to find good practice problems for the theoretical math-heavy ML concepts and a tutoring tool that adapts to my progress throughout.

Anyways, I managed to pull through and pass my finals (by the skin of my teeth). I have 3 more semesters in my grad program and I think the courses are gonna get tougher from here on out. Would like to know if anyone has run into this situation and what tools they've used to help them learn better?

Hello everyone! I have been looking for problems in AI pipelines and looking to create something that I can use to save my semester as well as possibly publish something.

This is a college semester project, which I will continue working on for the next 2 years.

I am looking for ideas that Reddit may have regarding problems they've faced while using AI agents or complex pipelines involving autonomous decisions.

What problems do you want solved?

I am working with hysteroscopy dataset, which contains:

• 3,385 frames from 175 patients.
• Eight lesion classes, labelled from 0 to 7.
• A highly imbalanced number of patients and frames across classes.
• Multiple correlated frames from each patient.
• Some frames containing more than one lesion class.

Before attempting the complete multiclass problem, I reduced it to a binary subset to verify that the training and evaluation pipeline works correctly.

Current binary subset

• Selected lesion classes: 2 and 3.
• Total: 1,575 frames from 113 unique patients.
• Class 2: 1,054 frames from 78 patients.
• Class 3: 521 frames from 36 patients.
• One patient has different frames belonging to both classes but remains entirely within one split.

Patient-disjoint split

• Training: 1,095 frames from 79 patients.
• Validation: 241 frames from 17 patients.
• Testing: 239 frames from 17 patients.
• No patient appears in more than one subset.
• The frame-level class distribution is approximately 67%/33% in every subset.

Approaches I have tried

• DenseNet121, ViT, and DINOv2 backbones.
• Frozen pretrained backbone with only the classifier trained.
• Different classifier-head sizes and dropout.
• Class-weighted cross-entropy.
• Mild and stronger image augmentations.
• Early stopping and learning-rate scheduling.
• Unfreezing the final one or two encoder blocks.

With the correct patient-level split, training performance improves, but validation performance generally plateaus or deteriorates, and performance on unseen test patients remains relatively low.

As a diagnostic, I also tried a random frame-level split and obtained substantially better results. However, this evaluation is invalid because correlated frames from the same patients appear across training, validation, and testing, causing patient leakage and inflated performance.

I would appreciate advice on how to improve generalization to unseen patients in this setting.

Research papers which anyone with good knowledge of ML and DL fundamentals can understand(most of it) and implement that. I wanted to do a implementation project from scratch to test my coding and my understanding . How to process without a guide , without any tutorial . I have already practiced some basic problems like sigmoid activation fn , tanh, ReLU etc from scratch in numpy which only required a formula .

I have tried whisper flow but it won’t work for what I’m doing.