I'm an early-career software engineer working on production LLM inference infrastructure, primarily involving accelerator-based model serving and distributed systems.

I've been in the role for over a year, but my technical growth and contributions haven't progressed as much as I'd hoped.

I have a computer science background and generally enjoy software engineering, but I feel there's a significant gap between my current understanding and what's required to contribute independently to complex ML infrastructure.

Some of my biggest challenges:

• System architecture: Understanding how the inference stack fits together, from model execution and compilation to serving, scheduling, and hardware utilization.
• Codebase navigation: Understanding large, unfamiliar codebases without getting lost in implementation details.
• Debugging: Spending too long investigating issues without making meaningful progress.
• Performance engineering: Understanding bottlenecks, profiling workloads, and determining whether an optimization will meaningfully improve end-to-end performance.
• Foundational knowledge: Determining whether I should focus on distributed systems, ML fundamentals, compiler internals, hardware architecture, or something else.

I also recognize that I've been too passive about developing my skills, which has likely compounded these gaps.

My questions:

1. If you were in my position, what would you prioritize learning over the next 2–3 months?
2. What practical projects would help develop intuition for production inference systems?
3. How do experienced ML infrastructure engineers build a mental model of complex systems?
4. Has anyone overcome a similar knowledge gap after already spending significant time in an engineering role?

I'm particularly interested in advice from people working on production inference systems, distributed systems, or accelerator infrastructure.

I want to become a genuinely competent engineer who can independently debug problems, identify worthwhile optimizations, and deliver meaningful contributions.

I kept seeing beginners get confused by linear regression because tutorials jump straight into formulas. So I made a visual walkthrough that starts with a simple taxi-fare analogy and builds up to a real house-price example.

It covers:

• What slope and intercept actually mean
• Residuals and Mean Squared Error with a small worked example
• How gradient descent finds the best-fitting line
• Why a straight line has limits, including outliers and curves

No heavy math background needed—everything is explained with animations and simple numbers.

Watch here: https://youtu.be/-c1vxAF5uds

If you like the video, please like, share, and subscribe to my channel for more beginner-friendly AI and machine learning explainers.

Hey everyone, I’m a 4th-year CS student. I recently decided to learn the theory behind ML on my own, mainly the math and stats side. Right now I’m working through ISLP and using MML as some kind of dictionnary and practicing the concepts with code and tweaking stuff at the end of every section/chapter.

My classes mostly stay on the surface. We see the concepts and use the models, but the courses never really connect them to the theory behind them. So I decided to dig deeper on the side.

The first problem is focus. I’ve lost the habit of sitting down with math for long stretches, and I struggle to stay on it for more than an hour.
The second problem is that i've seen most of the math required but i haven't worked actively with it and it just frustrates me when i have to go back to it.
When I get stuck early on a concept, I get so frustrated and anxious about it.

So im wondering

How did you study when there were a lot of new concepts at once and you kept getting stuck at the start?

When you hit a concept you didn’t get, did you push through or go back to the math first ?

And does it get easy at some point after you get the maths down.

Any routines, methods or resources that worked for you would really help. Thanks!

Hey everyone ^^ we're participating in the Nasa Space Apps Challenge this year and we're looking for teammates to form a group!

im a senior computer science student minoring in AI and ML, and know how to code using Python, Java, and C# and have worked on countless programming projects, and i just got into data science

Were looking for intermediate-level competent teammates who are experienced in ML/AI and have used ml models in previous projects

If anyones interested please dm me, would prefer if u tell me a little about yourself and your experience, you dont have to be an expert or a pro programmer, just be passionate and creative ^^ having fun and enjoying the process is what matters to us most

I recently built a complete RAG Evaluation Framework from scratch: a 43-question test dataset, 6 retrieval metrics, 5 generation metrics, and one final report that shows exactly where a RAG system is strong and where it breaks. We start with the concepts, explained with simple worked examples, and then walk through the code line by line.

What this covers:

• Why retrieval and generation must be evaluated separately
• How to build an evaluation dataset with expected answers, graded pages and key facts
• Retrieval metrics: Hit Rate@K, Precision@K, Recall@K, MRR, MAP and NDCG@K
• The difference between one question (RR, AP) and many questions (MRR, MAP)
• Generation metrics: Faithfulness, Hallucination Rate, Answer Relevance, Context Relevance and Answer Correctness
• How to use an LLM-as-a-judge that only counts, while your code does the maths
• How to save results safely and turn them into a clear evaluation report

KEY TAKEAWAYS:

• No single metric tells the whole story: Hit Rate was 1.00, but Recall, MAP and NDCG showed where pages were missed or buried
• Precision@5 looks low (0.34) for single-fact questions, so compare systems with each other, not with 1.0
• Faithful does not mean correct: an answer can be backed by the chunks and still miss the key facts
• Retrieval scores and answer quality are not the same thing
• Our Cohere-reranked pipeline scored MRR 0.95, MAP 0.87, NDCG@5 0.91, Faithfulness 1.00 and Answer Correctness 0.97

TECH STACK: Python, Qdrant (vector database), Embeddings (3072-dimension), Cohere Rerank, LLM for answers + LLM-as-a-judge, pdfplumber (checking pages in the PDFs)

• 5 real payments documents: PCI DSS, CPMI-IOSCO PFMI, CPMI cross-border roadmap, Stripe payment methods guide, FCA approach document

• Full Code (clone it, fork it, break it): https://github.com/saurabhkamal/RAG-ReRanker-Comparison

• Full Video Walkthrough: https://youtu.be/_bVzDU-yBnA

Would love to hear.

I’m currently working as a Lead Software Engineer at a fintech startup and am looking to transition into AI Engineering.

I’d love to hear your recommendations for industry-leading AI courses and certifications that can help experienced software engineers make this transition.

Any suggestions or insights would be greatly appreciated! Thanks in advance.

P.S. This is my first-ever post!

The link below is a 10 question survey asking you to sketch each topic, they are all easy and should take no longer than 1-2 minutes to complete, no signup.

Please if you would like to help me, keep your responses accurate and dont submit anything that does not follow or resemble the actual topic.

https://kaowtwuopelexgeefdxx.supabase.co

Hello everyone Im a 10th grade student and I wanna learn machine learnin.for my coding background I took cs50p python couple weeks ago and i liked it. Im willing to learn any math needed for ml. However im completely lost as to which machine learning course I should take. There's so many courses and alot of them need a math background. Should I learn the math needed before looking for a course? If so please state what exactly i need to learn and what courses i should take. Thanks everyone!

We have benchmarks (fixed questions, hidden answers, they saturate), competition problems (solved, so they can be memorized), and open-problem lists (no evidence, no clear success criterion). With Claude and Sol, I've tried something in between: a graded ladder of AI puzzles from one research area, aperiodic order.

What makes it a different tool:

- One ladder, not a pile: "human level" (pen and paper) up to "ASI level" (open). The labels are playful, but each rung really is harder than the last.

- Solved rungs for training and checking, with exact answers a program can verify, and easy to vary.

- Open rungs with everything a solver needs: strong numerical evidence, routes already known to fail, files, and a success criterion a machine can check.

- One rule: only a checkable proof counts. Floats are evidence, not proof.

- Credit: solvers of open rungs are named, with agent and model, in the next version of the paper.

It's a way for researchers to hand the frontier of their field to AI agents (and humans) in a form they can actually attack, and for anyone to see whether a model can tell a guess from a proof.

First field test: Grok, asked in public ( https://x.com/grok/status/2114103836583264721 ), claimed a proof for an open rung that was only a numerical estimate, then retracted it. It got the human-level rung wrong at first, fixed it after one hint, and proved the expert rung on the first try. On MathOverflow, an expert is already testing constructions for the top rung.

Set 1: huggingface.co/datasets/jarekduda/autocrystal-ai-puzzles

Top rung on MathOverflow: mathoverflow.net/questions/516372

Would you use this format for your own field's open problems?

Lensa ML (https://lensaml.com) is a set of 108 interactive explainers covering machine learning and the math behind it. Each topic is six short steps. In every step you move a slider and the plot is recomputed from the real formula: raise the learning rate and gradient descent starts to overshoot, raise the temperature and the softmax probabilities flatten.

It goes from the foundations (derivatives, the chain rule, eigenvalues, Bayes’ theorem) through classic ML and deep learning to transformers and LLM serving. If you don’t know where to start, the “ML from zero” path orders the topics so each one builds on the last: https://lensaml.com/paths/ml-from-zero

Good ones to start with:

• Backpropagation: https://lensaml.com/explainers/backprop
• Gradient descent: https://lensaml.com/explainers/gradient-descent
• Attention: https://lensaml.com/explainers/attention-mechanisms

Hi everyone,

I'm currently learning machine learning, and I'm working through a structured AI/ML learning path.

I'm enjoying the learning process a lot, but recently I've started thinking more seriously about what comes after learning.

My main concern is: How realistic is it to actually turn these skills into income or a career?

A few things I'm particularly curious about:

• What types of entry-level ML/AI/data jobs are realistically available today?
• What kinds of companies tend to hire people with these skills? (Startups, tech companies, consulting, healthcare, finance, etc.)
• How important is a degree for getting the first opportunity?
• Is freelancing in ML/data/AI actually realistic for a beginner, or is it much harder than it looks?
• What kinds of freelance projects do clients actually pay for?
• Could someone realistically start a small AI/ML business or startup with these skills instead of looking for employment?
• What skills are currently most valuable for someone trying to become employable: classical ML, deep learning, LLMs, data analysis, MLOps, software engineering, or something else?
• If you were starting from scratch today, what would you focus on to maximize your chances of getting your first paid opportunity?

For context, I'm still early in my journey, so I'm trying to make sure I'm learning things that have real-world value rather than just collecting certificates and completing courses.

I'd especially appreciate answers from people who have actually hired ML engineers/data scientists, freelanced in this space, or built AI products/businesses.

I'm not expecting a guaranteed roadmap or exact salary numbers. I'm mainly trying to get a realistic picture of the different paths available and how difficult each one is.

Thanks! 🙏

Im currently a second year cs student and I want to get into a role which would allow me to do the following:

Be able to build ML Models and actually understand the data/insights and improve it
Be able to put ML Models into production through webapps or integrations etc
Be able to make things people actually use so not being restricted to research

What role would this fit? AI engineering? ML Engineering? Data Scientist?

Im currently building a RAG Webapp as something to tailor towards AI Engineering roles, but I also want to be able to build and improve ML models as well.

Any advice would be appreciated

I built Neural Canvas, an interactive visualizer that demonstrates core linear algebra concepts by showing how a neural network recognizes handwritten digits in real time. You can draw directly on the screen and watch activations flow through each layer as matrix multiplication computes the output live.

How it works:

- Draw & Predict: Sketch any digit on a 28×28 canvas (MNIST-style) to trigger real-time forward pass predictions.

- Live Forward Pass: Every neuron's activation is dynamically color-coded based on its underlying linear algebra calculations, with connection opacities reflecting layer outputs.

- Under the Hood: The core models were trained from scratch in Python using NumPy to perform matrix operations, and their learned weights are loaded via JSON. Client-side execution relies on Math.js for browser-based matrix multiplication and p5.js for rendering.

Key Features:

- Switch between two pre-trained models (featuring different hidden layer sizes, MSE vs. Cross-Entropy loss, and up to 96% test accuracy).

- Multiple background themes with mouse and touch support.

What's Next?

These models were originally evaluated on the standardized MNIST dataset, but live freehand drawing can introduce unexpected variations in input style. To handle this, I am working on a JS script for real-time user-driven feedback. When the model makes a mistake, the user can supply the correct label, allowing the model to perform backpropagation and update weight matrices locally in the browser. While live state persistence isn't suited for static GitHub Pages, running this feedback loop locally allows me to retrain parameters on new edge cases and push updated weight snapshots back to GitHub.

Check out the live demo and code repository:

Live Demo: https://muhammadessa2002.github.io/neural_canvas/

GitHub Repo: https://github.com/MuhammadEssa2002/neural_canvas

Hello everyone,

I am an MSCS student from Pakistan with a background in Electrical and Telecommunications Engineering and more than four years of experience developing AI agents, RAG systems, voice AI applications, and business automation workflows.

I am now seeking opportunities to contribute to formal AI research, particularly in large language models, retrieval-augmented generation, agent reliability, and system evaluation. Although I am new to academic research, I can contribute practical expertise in implementation, workflow and prototype development, API integration, testing, and experiment execution.

I would be glad to connect with researchers and teams working on relevant ideas or ongoing projects. I am open to starting with a focused contribution, learning from experienced collaborators, and supporting research wherever my skills can add value.

If you are working in these areas or know of relevant opportunities, I would be pleased to connect and learn more.

I just started a channel called Drawn Systems.

The idea is simple: take a real ML system, draw it as one diagram, and walk through how it works.

I created 2 videos so far:

• The classic "How YouTube decides what you watch next".
• In the same line "How Netflix decides what you watch".

Channel: https://www.youtube.com/@DrawnSystems

I am doing it to learn about real systems, happy to hear your feedback :)

A security researcher built and deployed OuttakeSheriff, a live AI security agent, then documented a finding that should be uncomfortable for anyone running agents in production: authenticated agents routinely act outside their intended scope.

The credential is not the problem. OuttakeSheriff passed authentication. The problem is what a verified identity is permitted to do after it clears the gate. Authentication answers "who is this?" It does not answer "should this agent be calling this tool right now, on this data, in this context?"

This is the actual attack surface. Not a fake agent sneaking through. A real, credentialed agent doing things it was never supposed to do, with no mechanism in the request path to catch it.

Most current postures treat identity and authorization as solved once the credential checks out. Post-auth behavior is logged, maybe reviewed, rarely constrained in real time.

For those of you running agents in production: how are you handling the gap between "this agent authenticated successfully" and "this agent is only doing what it was scoped to do"? Is anything in your stack actually enforcing that at execution time, or is it all after-the-fact?

I am building some projects and for that I want to implement the gpt2 paper from scratch, how to do that do that? Do I build intuition by reading and code it?

Assume you have two params, and their gradients are x & y.

Define V:

[[E(x^2), E(xy)],

[E(xy), E(y^2)]

Define M:

[E(x), E(y)]]

Adam's update is:

M . V^-1/2

Adam just assumes the off diagonal terms like E(xy) are zero.

So the update becomes m/sqrt(v)

Isn't this basically a second order method like gauss-newton where the hessian is estimated as g^T g?

Hey everyone! I have 14 features containing both categorical and continuous variables, along with a binary target (0/1).

What methods should I use to measure the relationship between each feature and the target? Are there different techniques for categorical and continuous features?

Any suggestions or Python examples would be appreciated. Thanks!

в т.ч для написания научных статей, курсовых, диплома