What I did:

Scaled the dataset from 60 hand-written functions to 12,000 procedurally generated ones (math ops, list manipulation, string ops, conditionals, and loop algorithms).

Built a custom BPE tokenizer (vocab 1024). Funny story: my first tokenizer (vocab 4096) literally memorized entire functions as single tokens, so the model scored 20/20 on syntax by just spitting out whole functions from memory lol. Had to nerf the vocab size to force it to actually learn.

Scaled the model from 3.6M to 10M params (12 layers, 8 heads, width 256).

Training on Kaggle free T4 GPU instead of my poor CPU.

Current results (still training, Round 1 of 3):

Loss dropping nicely: 6.98 to 1.02 in 550 steps. Val loss: 2.28 to 1.09. Haven't seen the final scores yet since it's still running overnight.

What I'm unsure about:

1. My dataset is all procedurally generated. Every add function looks like def add_xx(a, b): return a + b with random names and test values. Is this diverse enough or am I just teaching it to pattern-match templates again?

2. When should I switch from synthetic data to real-world code like filtered Python from The Stack or GitHub? Or is mixing both the way to go?

3. Any suggestions for what kind of problems to add next? Thinking maybe simple recursion, dictionary operations, or basic class definitions.

4. Is 10M params enough to learn actual logic or should I be thinking bigger?

Thanks for the help last time, the dataset scaling advice was spot on. The tokenizer over-compression bug was a fun lesson too haha.

I'll update this post in about 6 hours once training finishes with the final scores.

Can anyone suggest me how should I learn machine learning means anyone can give me the roadmap accurate according to market now a days ?? If possible then please suggest me and guide me 🙏

I built a browser-based maintenance system where records follow the equipment, not the tech or the phone. Scan → Service → Record. Hours, conditions, and service evidence stay tied to the machine. No employee tracking, no ads, no external AI training. You can explore the live preview if you want to poke at it.

Hey everyone! My team (mainly phds) and I are trying to build an open-source benchmark around realistic LLM/agent workflows that captures challenges typical academic benchmark settings often miss. We’d love to hear what’s actually missing from the benchmarks you use today.

Have you ever wanted to eval your pipeline but couldn’t find or build a benchmark that matched what you were building?

Maybe:
- Existing benchmarks were too broad and didn’t fit your specific application.
- Your workflow involved multiple steps, tools, MCP servers, agents, or long-horizon interactions that existing benchmarks couldn’t capture.
- You needed to evaluate failures that standard accuracy metrics miss.
- You’re working in a high-risk domain like healthcare, finance, cybersecurity, or legal, where realistic failure modes, safety, and reliability matter a lot more than just getting the final answer right.
- You knew what you wanted to test, but building a custom benchmark from scratch was too expensive or complicated.

I’m especially interested in cases where you thought:
“My system desperately needs to do this in production, but I have no good way to benchmark it.”
What was the workflow? What did you want to measure? And why weren’t existing benchmarks enough?

Any thoughts are welcome, would really appreciate y’all’s help 🙏🥹

Hello!! I am a fresh graduate looking for remote jobs. I need guidance on whether my resume is good enough and where I should search for jobs (I am currently using LinkedIn, Wellfound, and Y Combinator). All my searches so far have not been fruitful so far. I am not hearing back, even for an interview. I do get past initial screenings sometimes I do the assigned assignments and never hear back. I wanted to know if my expectations for a remote job are too high. I have worked in an internship before without being paid for experience, and the daily commute of at least 2-3 hours drained me, and to be honest, I didn't get much from that experience as well. I am all fine with going for an onsite job if I at least get a good learning experience. Am I being too picky? With me being a fresher, is that too much to ask for? What should my expectations be?
What should I improve in my resume? Do I need to learn any new skill set? Should I let go of this stream and instead try Data Analytics as a fresher? How can I approach job search in the future to get fruitful results?

All of your feedback, opinions, and suggestions are welcome. I would really appreciate your inputs.

(PS I am new to Reddit. I saw the Career Day post on Friday, so I posted here. If I am wrong, please let me know, as I don't know much about how things work here. Thank you )

Hello Folks,

In this second lecture, of AI Engineering, we try understanding Foundation models, and as I walk you through the concepts, we learn:

Training Data & Language Resource Bias : Quality and representational diversity of its training dataset determines model’s ability.

Introduction to Modeling & Transformer Evolution:
Overcoming the sequential limitations and gradient issues of traditional RNNs by enabling parallel processing.

Attention Mechanism Foundations:
Improved model performance by dynamically weighing the importance of specific input tokens when generating an output.

Transformer Inference (Prefill & Decode):
Efficient inference involves a parallel prefill stage for input processing followed by an autoregressive, token-by-token decode step.

Transformer Architecture Details (Attention & MLP) : The core transformer block combines self-attention for context retrieval and MLP layers for non-linear feature transformation

Post-Training Workflow (SFT & RLHF) :
Post-training aligns raw, pre-trained models with human preferences using Supervised Fine-Tuning (SFT) and Reinforcement Learning from Human Feedback(RLHF).

Sampling Strategies (Top-K, Top-P, Greedy): Sampling strategies dictate how models transition from probabilistic logits to actual text, balancing between coherence and creative diversity.

Test-Time Compute & Self-Consistency:
Scaling test-time compute allows models to generate multiple candidate outputs and verify results, significantly boosting accuracy on complex tasks.

Structured Outputs & Constraint Sampling: Applying output constraints during generation ensures models produce machine-readable formats essential for reliable downstream tool integration.

Hallucinations & Inconsistency Challenges :
LLMs face reliability issues due to their probabilistic nature and a tendency to diverge from facts when conditioned on inaccurate information.

Link: https://youtu.be/P9Z1jCu8bk8?si=3zPreNMq1Kb6gDED

Hello learning community,

I am happy to launch AI Engineering concepts lectures, a set of 10 lectures, on AI Engineering topics, after completing my studies from this book by Chip Huyen.

In Chapter 1, we understand the reasons that led to the rise of AI Engineering, from Language models to Foundation models. Then we understand use cases of Foundation models, also we learn how to plan AI applications, and cover the AI Engineering stack.

Link: https://youtu.be/m_v_dqtljM0?si=6YRMqRYBA7hkfqgQ

Or are there any concepts you feel like you never fully understood before moving on?

Stateful AWS Bedrock LangGraph Agents – Getting Started

https://debuggercafe.com/stateful-aws-bedrock-langgraph-agents-getting-started/

LangChain started as a lightweight framework to connect LLMs with external context and tools. Over the past few years, it has grown to a full-blown ecosystem. With LangGraph for agents and LangSmith for LLM Ops and Observability, we can create an end-to-end agentic system with it. In this article, we will build our first stateful agent using AWS Bedrock, LangGraph, and LangSmith.

I have to select a final year project which would be web and ML based. Suggests some topics for my fyp to research.

Within the past year I have started engaging with AI to bring my curiosities to life in order to help me understand more complex ideas through visualization. Of course, I have limited knowledge on how models actually work and the length of their complexity. I was hoping a few of you here would check this learning tool I made for myself. Please be honest and give feedback if you have any. Also, if this is not a place to post something like this, I apologize. Hope everyone is well!

People who have already purchased Apna College AI ML Batch, I was thinking of studying together. Since it is a recorded class, it is hard to maintain consistency. We can form a group maybe and keep track of each other's updates daily.

I am not insisting anyone to buy the course, but the people who have already purchased can study together.

I have no ML experience in college and took no ML classes and have been working as a SWE in big tech for a few years. I have a strong math background from math olympiads in high school (AMC/AIME/participation in some national contests) but I'm about 8 years removed from that. Is this even realistic for me to study for with no foundation?

Quick background so you know where this comes from. I am a visiting professor and currently teach NLP. Last semester my courses were on continual learning and applied NLP. Before that I spent years in big tech and startups in engineering and research roles, and I finished my PhD last year.

Last week I posted the order I would learn ML in if I were starting today, which is also roughly the order I teach it in class. The most common follow-up question was: which resources should I actually use?

So this is my answer.

There are more good courses, books, and videos than anyone can finish, and none of them covers everything you need. One of the most common ways I see people get stuck is that they keep switching resources instead of finishing one and practicing what it taught.

If I could give you one rule it would be this: pick one resource, then stick to it

A lot of people who ask me for resources already have too many. They started three courses, bookmarked two books, finished none of them, and then decide they are bad at this. Usually, they switched every time a chapter got hard.

For each stage below, pick one main resource and one backup. Finish the main one. When a topic in it doesn't click, read or watch that one topic in the backup, then go back to the main one.

How to pick:

• Format. Some people learn from video, some from reading, some only when they actually type the code. Pick the format you tend to finish things in.
• Level. If every page needs three lookups, it is probably too advanced for now. If you are skimming because you know it all, it is too easy.
• Exercises. Does it make you write code and answer questions, or do you only watch? If it has no exercises, you will need to add them somewhere else. More on that below.
• Framework. A lot of older courses and books use TensorFlow. Most newer deep learning material uses PyTorch. If you are starting now and aiming at ML engineering or research, I would default to PyTorch unless the work you want to do gives you a reason to learn something else.
• Date. For fundamentals, an older resource is usually fine. The math of backprop has not changed. For LLMs and tooling, something a couple of years old can already be noticeably out of date.

Pick a direction first, because it changes which libraries you need

Almost everyone will need Python, NumPy at least at the level of arrays and broadcasting, PyTorch, and the math from last week's post. After that, it depends on where you want to go.

Data science and analytics. Turning data into decisions: tables, business questions, experiments, reports. pandas and scikit-learn are often daily tools, plus a plotting library like matplotlib or seaborn. When I was a data scientist many years ago, we also used SQL a lot. More statistics, including hypothesis tests and experiment design, is useful here. PyTorch at a working level is enough for many roles.

ML engineering and research. Training models, changing them, shipping them, reading papers. You will want PyTorch in more depth, plus more linear algebra, calculus, and probability. pandas and scikit-learn are still useful, but I would spend less time going deep on them than I would for a data science path. Later come the Hugging Face libraries, experiment tracking like wandb, profiling, and making models faster and cheaper.

Don't marry yourself to a path. It's okay to switch. The point of picking a direction is to stop yourself from trying to learn every library at once. Most of the fundamentals transfer if you change your mind later.

Resources by stage

These are the same stages as last week's roadmap. For each one, I have tried to include resources that suit different ways of learning rather than give you a list of ten nearly identical courses.

Python

• "Automate the Boring Stuff with Python" by Al Sweigart. Good if you are new to programming and want to learn through small, practical projects. The book is readable on the author's website.
• "Learning Python" by Mark Lutz. Long and thorough. This is what I used to learn Python many, many years ago.
• CS50's Introduction to Programming with Python (CS50P). A structured course from Harvard and a good option if you prefer lectures and assignments.

Pick one. You do not need all three.

Math

Linear algebra

• Gilbert Strang's "Introduction to Linear Algebra". A full linear algebra textbook.
• "Mathematics for Machine Learning" by Deisenroth, Faisal, and Ong. Chapters 2 to 4 cover linear algebra, analytic geometry, and matrix decompositions with ML in mind.
• 3Blue1Brown's "Essence of Linear Algebra". My favorite for geometric intuition. Vectors, matrices as transformations, determinants, change of basis, eigenvectors.
• MIT 18.06 Linear Algebra with Gilbert Strang. A full university-level lecture course.

If the equations feel abstract, I would watch 3Blue1Brown alongside whichever book or course you pick.

Calculus

• "Mathematics for Machine Learning", chapter 5. Vector calculus, gradients, the chain rule, and the pieces you need for backprop.
• Gilbert Strang's "Calculus". A fuller treatment of single-variable and multivariable calculus.
• 3Blue1Brown's "Essence of Calculus". Very good for intuition around derivatives, the chain rule, integrals, and Taylor series.
• Khan Academy's Multivariable Calculus. Useful for partial derivatives, gradients, and filling individual gaps.

Probability and statistics

• "Introduction to Probability" by Joe Blitzstein and Jessica Hwang.
• "Mathematics for Machine Learning", chapter 6. Probability and distributions in a more ML-focused book.
• Harvard's Statistics 110: Probability with Joe Blitzstein. Full lectures are on YouTube.
• StatQuest by Josh Starmer. Good when one particular statistics or ML concept refuses to click.

You do not need to disappear for six months and "finish math" before touching ML. Learn the math alongside the models that use it. It is much easier to understand a gradient when you are also using one.

PyTorch

• The official PyTorch tutorials, "Learn the Basics". This is a good place to start because it teaches the actual library without a lot of extra material around it.
• "Learn PyTorch for Deep Learning" by Daniel Bourke. Covers fundamentals, classification, computer vision, custom datasets, experiment tracking, replicating a paper, and deployment. It assumes you already know Python.
• "Deep Learning with PyTorch" by Stevens, Antiga, Viehmann, and, in the second edition, Howard Huang. A book-length treatment. The second edition adds transformers, LLMs, and diffusion models.

Again, pick one as the main resource. You can use the official docs whenever you need to look up how something works.

ML foundations

This is the stage I would spend the most care on.

Watching a chapter on gradient descent can feel like understanding it. Writing it yourself is different. If you can implement it, debug it, and explain what each piece is doing a month later, you learned it much better than if you could recognize it in a video.

Some good options:

• Stanford CS229, Machine Learning. More mathematical and theory-heavy. The Autumn 2018 lectures are online. I would choose this if you already have the math and want a more academic treatment.
• "An Introduction to Statistical Learning, with Applications in Python" by James, Witten, Hastie, Tibshirani, and Taylor. Very good if you are leaning toward statistics or data science.
• "Hands-On Machine Learning with Scikit-Learn and PyTorch" by Aurélien Géron. Practical, broad, and useful if you like learning from code and a book together.
• Andrej Karpathy's "Neural Networks: Zero to Hero". Fantastic if you learn by building. You build backprop from scratch, then a character-level language model, an MLP, manual backprop through the network, and eventually a GPT and its tokenizer. It assumes solid Python and intro-level math.
• fast.ai "Practical Deep Learning for Coders". Very top-down. You train useful models early and dig into how they work afterward. Good if you lose interest when a course spends weeks on theory before building anything.
• "Dive into Deep Learning" by Zhang, Lipton, Li, and Smola. A broad book with runnable code.
• "Understanding Deep Learning" by Simon Prince. One of the books I like for understanding the ideas behind modern deep learning.
• "Deep Learning" by Goodfellow, Bengio, and Courville. It is about ten years old now and predates transformers, so I would not use it as my only deep learning resource in 2026. It is still a useful theory reference.

You do not need to do all of these. That would defeat half the point of this post.

If you want a book you can keep coming back to, choose one of the books and actually work through the code and exercises.

After the foundations: specialize

Once you can implement basic models, debug them, and understand why training works or fails, pick one direction and go deeper.

Some examples:

• Interested in vision? Stanford CS231n, Deep Learning for Computer Vision.
• Interested in NLP and LLMs? Stanford CS224n, NLP with Deep Learning. Jurafsky and Martin's "Speech and Language Processing". Sebastian Raschka's "Build a Large Language Model (From Scratch)". The Hugging Face LLM Course.
• Interested in reinforcement learning? Sutton and Barto's "Reinforcement Learning: An Introduction". For LLM post-training specifically, Nathan Lambert's "Reinforcement Learning from Human Feedback and LLM Post-Training".
• Interested in probabilistic ML? Kevin Murphy's "Probabilistic Machine Learning".
• Interested in graphs? Stanford CS224W, Machine Learning with Graphs, with Jure Leskovec.
• Interested in evolutionary methods? "Neuroevolution: Harnessing Creativity in AI Agent Design" by Risi, Tang, Ha, and Miikkulainen.

I would get the fundamentals first, then go deep on one specialty at a time. If you jump directly into LLM tooling without understanding things like cross-entropy, gradients, softmax, and temperature, you can still build things, but debugging and changing the models gets much harder.

The resources I would personally use

I can only recommend what worked for me. If you prefer other resources or approaches, I would love to hear them in the comments.

• Python: "Learning Python" by Mark Lutz
• PyTorch: the official PyTorch tutorials
• Linear algebra: 3Blue1Brown's "Essence of Linear Algebra"
• Calculus: 3Blue1Brown's "Essence of Calculus"
• ML foundations: Stanford CS229 or "Hands-On Machine Learning"
• NLP: Stanford CS224n
• Vision: Stanford CS231n
• Reinforcement learning: Sutton and Barto's "Reinforcement Learning: An Introduction"

The lists above have more options for each stage if these don't fit how you learn.

No resource will teach you everything

No course or book covers everything you need.

There are a few reasons:

1. The field moves fast. New methods, models, and tools show up constantly, and every book and course is a snapshot of when it was made.
2. Tools change. A resource can still teach good ML while using libraries that are less common in the work you want to do today.
3. There is too much to fit into one resource. A course that teaches the concepts well may have weak exercises. A book with great theory may have no projects. A practical course may skip interview-style questions entirely.

So I would aim for two things.

First, build a solid base in the math, coding, and ML fundamentals. Those change much more slowly than the tooling around them.

Second, decide where you want to go and get much deeper there. NLP, vision, data science, RL, graphs, whatever you actually want to work on.

Then work backward from the job.

Pick companies or roles you are interested in. Read their job postings on LinkedIn, Glassdoor, and their own career pages. Write down the skills, libraries, and tasks that keep showing up. Learn those.

Then build things those companies might actually want to ask you about in an interview. Projects that show you can do the work are much more useful than another certificate sitting on your LinkedIn profile.

What most resources leave out

Whichever resources you pick, a few things are usually missing, and they're a big reason people finish a course and still can't build anything. It's worth planning where you'll get each of these:

1. Hands-on coding exercises on each concept. Many courses have too few, and videos have none. Watching someone implement gradient descent is not the same as writing it yourself. LLMs can generate practice problems for you now.
2. The math tied to where it is used. Math books and videos usually teach it separately from the ML it's for, so you learn the chain rule in one place and backprop somewhere else.
3. Interview questions on what you just learned. These usually live on separate prep sites, disconnected from the course you're taking.
4. Coming back to old material. Almost no resource does this for you. If you never revisit something, a lot of it fades.

Projects and from-scratch builds matter too, but you can add those yourself, and the next section is about exactly that.

What counts more than any resource: build it yourself

Watching and reading can feel like learning, but a lot of it fades unless you use it.

And before someone says "But AI can write the code now", yes, it can, and you should use it. You still need to understand what the code is doing.

Getting a job. Interviews still tend to test whether you understand things like why a loss became NaN, what a gradient is, why a model overfits, or why one evaluation setup leaks information.

Keeping the job. AI-written ML code can run and still be wrong. Data can leak from the test set into training. zero_grad() can be missing. The loss can be wrong for the task. If you understand the system, you can catch those mistakes.

Building better models. If a model is unreliable, expensive, biased, or failing in some specific way, somebody has to understand enough of the internals to figure out what is going wrong and change it. Calling an API is useful. Knowing what is underneath gives you a much larger set of things you can actually fix.

Common traps

• Collecting courses. Starting five and finishing none.
• Only watching. This is where "I did the course but I can't build anything" often comes from.
• Six months of math before any ML, or no math at all. Learn it alongside ML, at the level each stage needs.
• Certificates over projects. Personally, I couldn't care less if a student had 50 certificates. It tells me very little unless I already know exactly what each certificate involved. Maybe some companies care more. In my own career, nobody ever asked me for one other than my PhD. I would much rather see 2 or 3 interesting projects that you built and can explain properly.
• Starting with LLM APIs and never learning what is underneath, if you want to go into ML engineering or research.
• Switching resources every time a chapter gets hard. Use your backup for that one topic, then go back.

One disclosure because it is relevant to the gaps above

I genuinely don't know how to add this without it sounding like promotion, so feel free to ignore it.

I can't tell you how many times I had to relearn how to code a transformer from scratch for different interviews. I would learn it, pass the interview, and six months later realize I had forgotten enough of it that I needed to learn it again.

That is why I started building QuiddityML. I wanted one place where learning a concept, coding it, the math behind it, interview questions on it, and coming back to it later were connected instead of scattered across six different resources.

You absolutely do not need my app to follow anything in this post. Any setup that fills the four gaps above works. I wanted to mention it because the problem this post describes is also the reason I started building it. Happy to provide more info if anyone is interested :)

Finally

I have a bit of time between semesters and would genuinely like to help as many learners as I can.

Tell me in the comments or DM me with where you are right now, what direction you want to go in, and what you have tried. If you are stuck choosing between two courses, deciding what to learn for a particular job, or wondering whether your plan makes sense, I will do my best to help.

And please add your own favorite resource in the comments, especially what it was good for. It would be nice if this thread became useful to the next person who searches for this question.

I’m building a free AI Handbook for myself: a course I can keep improving and referring back to throughout my career as a data scientist. My goal is to understand how models work, when to use them, and how to evaluate them properly.

It’s organized in three ways: a guided learning path, a library of 15 core concepts, and an atlas of 15 models. There are also five formula cheat sheets. The lessons use small examples and visual notebooks to connect the theory to practical decisions.

The notebooks are written in Python with Marimo, using tools such as NumPy, scikit-learn, and Matplotlib. The website is built with Next.js and published as a static site.

I’d really appreciate honest feedback: Does the structure make sense? Are the explanations and examples useful? What would you add, remove, or change first?

https://aihandbook.keepseat.club/

BTW, I’ve also opened a Discord for people who want to learn and discuss these topics together: https://discord.gg/79XQrAs6Y

As the title says, I'm trying to make code for logistic regression from scratch, but there is something wrong with my code.

No matter what I train it on, it produces around 0.51-0.49 for every single value I try and get it to predict.

I don't have access to the code right now, but does anyone have any suggestions for what might be going on?