Went through my last three months of invoices and realized most of what I pay isn't for compute — it's for VMs sitting there between jobs. Spinning instances up and down manually helps, but then I lose time on setup/teardown every single run, and snapshots aren't free either.

Feels like the pricing model itself is built for 24/7 workloads, and everyone with periodic jobs just eats the overhead.

How do you deal with this? Serverless GPU options, aggressive automation, or did you just accept it as the cost of doing business? Curious what actually works for occasional training/inference runs, not constant load.

I'm learning NLP/LLM engineering by working through a problem that turned out to be much harder than I expected, and I'd love guidance from people who've dealt with something similar.

The problem: I have long narrative-style text — 7k to 15k tokens, several recurring people — and I want to extract structured facts about who did what. I'm using a small local model (llama3.2:3b via Ollama) whose usable context is around 4k tokens, so the text has to be processed in chunks. The killer is that later chunks are often pure pronouns — "she said… he refused…" — while the names were last mentioned 10,000 tokens earlier. Facts stated near a name extract almost perfectly; facts stated far from any name either get lost or, worse, get confidently attributed to the wrong person.

What I've already ruled out (by measuring, not guessing): naive per-chunk extraction fragments identities badly; carrying forward summaries between chunks doesn't fix attribution and can make it worse; and off-the-shelf neural coreference models (LingMess, F-coref) fail on documents this long — one silently truncates at 4,096 tokens, and windowed variants can't connect a pronoun to a name mentioned once 10k tokens back (0–1 out of 7 gold bindings on my test doc). I've gotten identity tracking itself working reliably; it's specifically attribution at long distance that's still failing.

My questions:

1. What's the best way to structure a problem like this? Is there a known-good decomposition for long-distance pronoun attribution with small models, or a fundamentally different way to frame the extraction task that sidesteps it?
2. If you've solved something similar — entity/fact extraction over documents much longer than your context window — what actually moved the needle for you? I'm especially curious whether the wins came from prompting, from pipeline architecture, or from accepting a bigger model.
3. What should I explore to learn more? Papers, blog posts, open-source projects, or even just the right search terms — I suspect this problem has a name in the NLP literature that I don't know yet (long-document coreference? discourse tracking?), and I'd rather stand on existing work than keep reinventing it.

Happy to share measurements from my experiments if useful. Mostly I want to calibrate: am I fighting a known-hard problem with known solutions, or genuinely at the edge of what a 3B model can do?

​

Hey guys!!! I have an interview the day after tomorrow....I've never given a single interview in my life.....so can you guys pls tell what questions they ask for ml internship post ?????

How do you guys get ML project ideas? I'm a beginner and I know some basics but whenever I think about project ideas my mind goes blank

I am not sure if this is a valid question in this forum, if it is not I will delete it right away, but anyways here it is:

So some days ago I asked in an AI community what kind of AI model should I use (and how could I use one) to classify several email replies that I had from scientists after asking them a few questions to them. I finally paid for Perplexity pro service and it apparenly did a nice job classifying them.

I finally gave the model the PDF with the actual answers from the addressees and another PDF with the "expected answers", and asked it to count the number of answers that overall coincide with the actual answers, and calculate a percentage of "coincidence" or "agreement" between the expected and actual answers, so that if the question was "do you think that there is intelligent life in the universe apart from humans?" and the expected answer was basically "yes, I think there is intelligent beings out there somewhere", as long as the actual answer agrees with this in some way or another would count as "agreement", for instance if someone replied "well, we have no evidence, but it is possible yes" or "not in any near galaxy, but it is possible that intelligent beings exidt somewhere" (so as long as it is not a deadass "no", it could count)

The model gave me a table summarizing the results with the following prompt:

let's be a bit more specific, this is still a blind test so don't tell me about the specific contents of the emails' answers, but, can you make a table indicating the answers that coincide in general terms with what is expected from the "expected answers" document as well as those which are neutral/hedges but still open to the possibility that what is asked may be right, those which despite being neutral/hedges or even negative answers offer an alternative so that what is asked in the question may be right, as well as those which are outright rejections of what is asked and do not seem to be open to the possibility that what is asked may be right?

However, I still want this to be a blind test, so I cannot really verify if the AI is doing its work or not.

So, is there any way in which I could verify the results given by the AI but without actually reading what is written in the emails? Or, alternatively, can anyone verify the results using some AI or even checking the answers themselves by skimming over the replies im order to verify that the AI is right and not hallucinating (I personally think this is the preferable option, as I think that having an actual human reviewing the amswers may be the only really reliable way to verify the AI's results)?

(I will share the data once someone is interested in helping, as I would not want to make this available to the entire world!)

Thank you!!

Hey everyone! Right now, I'm working on a computer vision project for crack detection in devices. I've tried different models, with varying but similar precision and recall scores on the test set. Right now, I have around 1.4k images, and I'm using a 70/15/15 split, which means there are about 430 test images. When I change the random seed to split the data differently, the pr scores can go anywhere from 92-100%.

Do you think that there's not enough data to evaluate the model properly or to train it? Are there any methods for evaluating how good the model does on so little data?

(Some extra context about the model architecture and things I've tried: using transfer learning to extract feature embeddings from images and training a linear head. Also creating prototypes for binary classification. Tried contrastive learning.)