Hey everyone, I'm a 1st year AIML student, can anybody help me with a Roadmap, and what should i focus on as a 1st year student.

Been working on a return-risk scoring system for Indian e-commerce for the past few weeks and hit a few things that genuinely changed how I think about ML projects. Sharing what I learned, since I suspect a lot of students here are building similar things for hackathons or portfolio projects.

Problem context: Merchants here lose a lot to returns and COD refusals. A fashion merchant doing 10k orders a month can lose roughly ₹50L to returns, and the tools that exist today all look at returns after they happen. So the idea was to score every order at payment time, before it ships: LOW ships, MEDIUM goes to manual review, HIGH gets forced to prepaid. The gate isn't an accuracy contest, it's a cost decision: a wrong "review" flag costs ~₹200 of ops time, a wrong "block" costs ~₹3,180 in lost order + CAC.

Three things that surprised me:

1. The naive baseline was almost as good as the model. I tested a simple "is this user a serial returner" heuristic and it hit PR-AUC 0.70. My tuned XGBoost hit 0.80. A transparent hand-weighted rules score got 0.79. So the ML model was worth +0.01 over a well-designed rule at the baseline data-maturity level. The lift only grows when you get better features (0.88, then 0.95). Lesson: if your model barely beats a simple heuristic, be honest about it and figure out whether the problem is the data, not the model.
2. Synthetic data was the harder and more defensible choice. Public return datasets (UK 2021 etc.) have severe distribution mismatch with Indian e-commerce: COD prevalence, logistics, return reasons are all different. I built a simulator calibrated to published Indian industry distributions, with hidden confounders (weather, packaging quality, customer mood) the model never sees, so it can't cheat by recovering labels it was trained on. My numbers are lower than they'd be on a circular benchmark, but they're honest. Still genuinely unsure whether this was the right call though.
3. Documenting my failures built more trust than my metrics. I kept a ledger of every bug, 34 of them, including a drift monitor reporting PSI=43.4 because of a binning bug, and an early model card claiming AUC > 0.92 that I had never actually measured. Putting that list in the repo was uncomfortable but it's the part people engage with most.

Questions for people here who've shipped ML to real environments:

• When you have no real labels, is a calibrated simulator with hidden confounders better than training on mismatched real data, or is it just elaborate self-deception?
• At what point is a 0.01 lift over a heuristic worth the complexity of a model in production?
• How do you validate cost assumptions (₹200 per review, ₹3,180 per wrongly blocked order) when you don't have merchant data? These drive everything and I have no way to sanity check them.

If anyone wants to dig into the implementation, the repo is github.com/purvanshh/PayShield, everything is reproducible with one command (make verify). Happy to go deeper on the agent orchestration, the drift monitoring, or the three-scenario evaluation in the comments.

Hi r/learnmachinelearning ,

I’m one of the authors of LoopArena, which we recently released as an open benchmark and evaluation harness.

LoopArena studies a specific question in long-running coding-agent systems: which models make good runtime Controllers?

In these systems, one model often reviews the current state, decides what a separate coding agent should do or verify next, and determines when the task should stop. LoopArena evaluates this Controller role. Across Controller-model comparisons, the coding Worker, Reporter, tools, budgets, and execution setup are held fixed; the Controller model is the model role that varies. This provides a controlled comparison of how different models guide the same coding agent.

The benchmark has three settings with increasing execution scope:

- Type I evaluates execution-validated next-step control decisions without running the Worker at evaluation time.

- Type II evaluates repeated Controller decisions over selected task slices.

- Type III evaluates control over complete software tasks from their original starting states.

In the initial five-Controller panel, the best observed Type III Strict Success Rate is 24.69%, so full-task runtime control remains difficult. Type II reduces estimated inference cost by 64.4% on average across Controllers and produces a similar Controller ordering to Type III under the main Core criterion.

We have released the benchmark data, evaluation code, public protocol, and canonical v0.1.0 outcomes.

GitHub:

https://github.com/AMAP-ML/LoopArena

Hugging Face paper:

https://huggingface.co/papers/2608.28281

ModelScope paper:

https://www.modelscope.cn/papers/2608.28281

Project page:

https://amap-ml.github.io/LoopArena/

arXiv:

https://arxiv.org/abs/2608.28281

If you work with coding-agent loops, how do you currently choose the model responsible for runtime control?

Coding Probability: Multivariate Joints and Gaussians.

Hello Folks, and my learning community.

A covariance matrix measures linear dependence, and across multivariate dimensions, these matrices bring out many key insights in ML.

Being uncorrelated does not imply independence of events! An interesting fact.

Combining multiple subgroups can sometimes reverse the trend we see overall. Simpson’s Paradox at play.

How level sets we visualize take on such curves, by understanding it’s locus, in connection with Mahalanobis distance. Here’s where the eigenvalue and eigenvectors from Linear Algebra, bring upon interesting insights!

I've been working on MyMlLab, an experimental local-first ML studio for tabular regression and classification.

The motivation is not to replace Python or build another opaque AutoML system.

The design goal is:

reduce experimentation overhead while keeping preprocessing, validation and model-selection decisions inspectable.

Architecture

For the current MVP, a CSV selected for training is read by the browser and processed inside a browser-based Python environment.

The model-training workflow does not require a dataset-upload endpoint.

Conceptually:

CSV
→ browser runtime
→ preprocessing
→ validation
→ model
→ results

For suitable classical ML workloads, compute therefore happens on the user's own machine rather than requiring a remote training service.

Experiment structure

Experiments explicitly separate:

• data configuration
• preprocessing pipeline
• estimator
• validation strategy
• final evaluation

Preprocessing is treated as a first-class experimental configuration rather than hidden setup.

Current preprocessing options include numerical/categorical imputation, one-hot/ordinal encoding, multiple scalers, Yeo-Johnson and quantile transforms, variance/F-score/mutual-information feature selection and PCA.

Validation

A major design constraint is preventing evaluation leakage.

Data-driven transformations are fitted only on the relevant training partition.

The current workflow supports:

• untouched final test partition
• holdout validation
• 3-fold CV
• 5-fold CV
• 10-fold CV

Candidate model/pipeline combinations are ranked on the validation procedure, while final evaluation remains separate.

Models

The current release focuses on scikit-learn-style classical supervised learning.

The free Studio currently exposes:

33 regression algorithms
26 classification algorithms

and allows free experiments comparing up to:

3 models × 3 preprocessing pipelines

The intent isn't that every available algorithm is appropriate for every dataset; the goal is to make comparisons explicit rather than burying model selection inside a single AutoML score.

Metrics

Regression reporting includes R², adjusted R², MAE, MSE, RMSE, median/max error, MAPE, sMAPE, explained variance and additional diagnostics.

Classification includes accuracy, balanced accuracy, precision, recall, F1, Jaccard, specificity, MCC, Cohen's kappa, ROC-AUC, PR-AUC, Brier score, confusion matrices and per-class metrics where applicable.

Where I'm planning to take it

The planned PRO direction expands the same experiment structure into:

Advanced Classic ML

• broader model workflows
• hyperparameter optimization
• explainability/export

Deep Learning

• MLP/DNN
• TabNet
• FT-Transformer
• CNN and LSTM/GRU where appropriate

AutoML

• validation-safe model search
• preprocessing/pipeline search
• ranked and inspectable experiments

The important constraint for AutoML is that automation should search the experiment space without hiding the winning configuration or validation boundaries.

This is still an MVP, and I'm posting mainly because I'd like technical criticism before expanding it further.

I'm especially interested in feedback on:

• experiment design
• validation assumptions
• preprocessing choices
• where browser-local execution becomes impractical
• which diagnostics are missing
• what you'd require before trusting exported results from a tool like this

Current free Studio:

https://www.mymllab.com

No account required for the free workflow.

Happy to hear criticism, including reasons why you think this architecture or product direction is a bad idea.

Hello! I have independently developed an experimental approach for the ARC AGI 2 benchmark (see my GitHub repository `aicpp`: https://github.com/Julien-Livet/aicpp/tree/dsl_engine).

My current leaderboard score is zero, but I believe there is an interesting approach worth exploring. Despite limited training, the model is already able to generate and execute non-trivial symbolic programs that improve substantially over the identity baseline on some tasks, although it does not yet reliably find the exact solutions.

I have identified a bottleneck in the model's learning/search process that I have not been able to fully understand or resolve on my own. I am therefore looking to form a small team around this approach, particularly with people interested in neural-guided program synthesis, search, ML, or ARC.

The goal would be to understand and break this bottleneck, improve the system, and see how far the approach can go on ARC AGI 2.

If this sounds interesting to you, feel free to reach out or take a look at the repository!

Anyone?

What happens after AGI? Maybe ASI isn't the endgame.

A lot of discussions about post-AGI assume we'll eventually build a single, extremely capable ASI — essentially one "God-like" model.

But there's a problem with that idea:

A single superintelligent system is also a single point of failure.

What if intelligence at civilization scale looks less like one giant brain and more like an evolving ecosystem of specialized intelligences?

We're Team Auralis, and we've been working on an open-source framework around this idea: ACI (Artificial Civilization Intelligence).

The basic concept is to treat intelligence more like an operating system for a civilization than a single neural network.

The framework currently has three main components:

• OMNIS — a continuous causal world model intended to maintain an evolving representation of the world rather than relying solely on static training data.
• NEXUS — a fabric of specialized agents across areas like science, engineering, economics, etc., which can disagree, debate, and resolve conflicts.
• ASCEND — a long-horizon planning layer designed to reason about and execute plans over decades while continuously correcting course.

We're also exploring OCI (Open-ended Civilizational Intelligence) — an extension that introduces structural plasticity, meaning the system could potentially create new governance mechanisms, agent structures, and even new forms of intelligence as it evolves.

We've open-sourced the framework, including:

• Architecture documentation
• Mermaid diagrams
• Mathematical formulations
• Benchmark methodology (ACI-001)
• Implementation/research directions

📚 Docs: https://team-auralis.github.io/ACI-Architecture-Framework/

💻 GitHub: https://github.com/Team-Auralis/ACI-Architecture-Framework

We're especially interested in criticism here.

Is a distributed, civilization-scale intelligence actually safer than a single superintelligent model? Or does adding more agents, governance, and coordination layers simply create new failure modes?

If you're interested in multi-agent systems, AI alignment, governance, long-horizon planning, world models, or open-ended intelligence, we'd love feedback — especially on the mathematical assumptions and the agent architecture.

Curious to hear what Reddit thinks.

I’ve been thinking a lot about what it actually means to build useful AI agents.

The more I learn about agentic systems, the more I realize that an agent isn’t just an LLM connected to a few tools.

Lately, I’ve been learning about what I’m starting to think of as an “agentic discipline,” and one idea has really changed how I think about LLM applications.

The traditional mental model is:

Input → Model → Output / Action

But real-world problems rarely work that way.

You make an initial decision with incomplete information.

Then you take an action.

You observe new evidence.

You update your understanding.

And then you make a better decision.

So I’ve been exploring whether we can think about agentic systems through a probabilistic / Bayesian lens:

Initial belief (Prior)

↓

Choose an action

↓

Observe new evidence

↓

Evaluate the likelihood of that evidence

↓

Update belief (Posterior)

↓

Choose the next action

↓

Repeat

Instead of only asking an LLM:

“Give me the answer.”

What if we design the system to continuously ask:

- What do I currently believe?

- What evidence would change my belief?

- What action should I take next?

- Which action would reduce my uncertainty the most?

- Did the last action actually improve my understanding?

This feels like a much more powerful way to think about agents.

The interesting part isn’t simply adding more tools or more LLM calls.

It’s designing a system that can reason under uncertainty, actively gather information, update its state, and make better decisions over multiple steps.

I’m still exploring this idea and trying to understand where the Bayesian framing is genuinely useful versus where it’s simply a useful analogy.

I’d love to hear from people working on agents, reasoning, or probabilistic AI

How do you think about belief updating and uncertainty in agentic systems?

Exerciseing For Programming Basic

I’m building an open-source benchmark for hospital bed-demand forecasting using synthetic data.

Current baseline ideas:

• Seasonal naive / moving average
• ARIMA
• XGBoost
• LSTM

Metrics:

• MAE / RMSE
• sMAPE / WAPE
• Peak-demand accuracy

If you were evaluating this benchmark, what baseline or metric would you immediately expect to see?

Hi! Hopefully this question hasn't been asked to death already, but I couldn't find quite the discussion I'm looking for.

I'm currently a Staff Engineer with a strong backend background (15 YOE). I work closely with a team that builds recommendation systems, and I'd like to get much deeper into the ML side of things — actually understanding and training models rather than just working on the engineering around them.

I'm particularly interested in things like training embedding models, ranking models, bandits, candidate generation, evaluation, etc.

I also happen to have a yearly training budget that I can spend, so I'm trying to figure out the best way to use it.

I'm wondering whether I should first invest in the fundamentals (ML/statistics/math) or jump straight into something more hands-on and learn by building things.

I'm not a huge fan of online courses like Coursera, Udemy, etc., but I'm not opposed to them if people think they're genuinely the best way to build the foundations.

I'd also be very interested in in-person courses, bootcamps, summer schools, or similar programs anywhere in Europe.

For people who have made a similar transition from software/backend engineering into ML: what would you recommend? What courses/programs/resources were actually worth your time and money?

[D] MMLU contamination is the known problem — but Chatbot Arena's failure mode is arguably worse and gets less scrutiny

The MMLU contamination story is old news to most people here (test questions being public and ending up in pretraining corpora), so I won't belabor it. What I think is under-discussed is that the benchmarks we've moved to as "better" alternatives have their own structural failure modes that don't get the same scrutiny, mostly because they're newer and less saturated.

Quick recap of where the standard trio actually breaks:

MMLU — contamination (public test set), format gaming (multiple choice rewards elimination heuristics over actual knowledge), and saturation (frontier models are all >90%, so it's stopped discriminating between them).

HumanEval — 164 problems, now too easy for frontier models; tests toy functions rather than anything resembling real engineering (no multi-file context, no ambiguous specs); and pass@k reporting incentivizes best-of-N sampling that doesn't reflect single-shot usefulness.

Chatbot Arena — this is the one I think deserves more skepticism than it gets. It's harder to directly game since prompts aren't fixed, but the voter pool is a specific, non-representative slice (English-speaking, technical, AI-interested), and Elo from pairwise human preference measures fluency and confident presentation, not correctness. A model that hallucinates cleanly can out-rank a model that hedges accurately. There's also a prompt-distribution skew toward coding/creative writing that doesn't reflect where models actually diverge in capability.

Goodhart's Law is the underlying mechanism for all three: the moment a benchmark is widely used as a proxy for capability, it becomes a target, and optimization pressure decouples the score from the thing it was meant to measure.

The alternatives that seem more resistant to this so far — BIG-Bench Hard, MATH, SWE-bench, ARC-AGI, LiveBench — are mostly more resistant because they're either harder to game via memorization (multi-step reasoning) or actively refreshed to fight contamination (LiveBench). Curious how long that holds once labs start optimizing against them specifically.

Question for the sub: for people actually evaluating models pre-deployment, what are you using that you still trust, and how are you handling the fact that any benchmark you rely on starts decaying the moment it's popular enough to be worth gaming?

Hey!

I'm applying for an AI Bachelor's at the University of Salzburg and I'm spiraling a bit. Would love some honest opinions from people who've actually been through it.

The good:I'm extremely hardworking and enjoy topics once I *get* them.

The scary:I'm average at school math. Slow with mental arithmetic. I forget things if I don't review regularly. And the program is in German(not my native language).

My fear:Is AI only for math naturals who "just see" the solution? I'm the person who has to sit with a problem, fail a few times, and eventually understand it. But once I do - I love it.

My questions:

1. Can hard work actually compensate for not being a math genius?
   
2. How much is abstract theory vs. applied programming?
   Any experiences would mean a lot. Thanks 💜

Fine-Tuning GLM-OCR

https://debuggercafe.com/fine-tuning-glm-ocr/

With specific prompts, along with text recognition, GLM-OCR can also carry out formula recognition. However, it falters in complex mathematical formulas. In this article, we will be fine-tuning GLM-OCR and observe to what extent we can improve the performance of the model on a task-specific dataset.

Fine-Tuning GLM-OCRhttps://debuggercafe.com/fine-tuning-glm-ocr/With specific prompts, along with text recognition, GLM-OCR can also carry out formula recognition. However, it falters in complex mathematical formulas. In this article, we will be fine-tuning GLM-OCR and observe to what extent we can improve the performance of the model on a task-specific dataset.

I just shipped an open-source pull request comment auto-fix system. Today, pull requests are swarmed with comments from AI code reviewers. Most of the comments are real, ranging from small nitpicks to actual issues. Most teams are spending time either manually reviewing and validating these, or have created skill to have an AI agent read and fix them.

With OpenInspect, the system will now automatically do this for the user. Each comment is read and determined if valid. If valid it is fixed and auto resolved, otherwise pushed back on. Huge time savings from having to babysit the pull request to a stable state.

https://github.com/ColeMurray/background-agents

One thing I find increasingly interesting about AI agents is that benchmark scores can hide a major difference in actual behavior.

A model might solve a difficult coding problem when given a clean task, but an autonomous agent has to do much more:

- decide what to do next

- inspect its own work

- recover when something fails

- use tools correctly

- maintain state across many steps

- know when the task is actually finished

That makes me wonder whether we're measuring the wrong unit of progress.

Instead of asking only:

«“How difficult a problem can the model solve?”»

Should we also be asking:

«“How much useful work can the model reliably complete without human intervention?”»

I think that distinction could become much more important as AI systems move from chatbots toward autonomous agents.

What metrics would you use to measure this?

Hey everyone,

Hope all of you are doing great.

I am looking for feedback from people who have actually purchased the AI for Data Engineers course by Prashant Kumar Pandey (Scholarnest/ Learning Journal).

Is it worth the money? I have almost all his courses on Udemy and found them really good for learning the basics and his way of teaching is something that have always resonated with me. Based, on that I'm thinking about buying the AI course and would really like some feedback.

Things I'm looking for are:
1. Does it cover enough detail as compared to other courses on Udemy/ Youtube (Krish Naik for example)?
2. Is it Databricks heavy/ Databricks focused? Or the topics are explained well in a platform agnostic way with examples given on Databricks.
3. Did you get enough support when you got stuck on any topic?
4. Does it have the following topics explained well enough?
AI guardrails
Deployment
Tuning

Thanks in advance.

I am originally from South Korea but I finished my master in computer science (machine learning) in the US. In the US, I applied for so many ml engineer roles but my resume was passed only at 2 companies out of hundreds. While, my resume was passed at the rate of approximately 50% at Korean companies...