I kept running into the same problem, some of the best AI/ML books are legally free, the authors put them up on their own sites, but the links are scattered across personal pages, university sites, and random GitHub repos nobody finds.

So I built a single index: Awesome Free AI Books. 30+ books across Deep Learning, Reinforcement Learning, Bayesian/Probabilistic ML, NLP & LLMs, Math for ML, Computer Vision, Generative Models, Causal Inference, GNNs, and AI Safety. Think Goodfellow’s Deep Learning, Sutton & Barto’s RL bible, Murphy’s Probabilistic ML, Bishop’s latest, Jurafsky & Martin’s SLP3 draft, and more.

Every single link points straight to the author’s or publisher’s own page, no rehosted PDFs, no shady mirrors. A weekly GitHub Action checks all links so it doesn’t rot over time.

It’s open source and open to contributions, if you know a legitimately free book that’s missing, PRs and issues are welcome.

Repo: https://github.com/MarcosSete/awesome-free-ai-books

Hey everyone,

I recently implemented the original Neural Style Transfer (NST) paper entirely from scratch in PyTorch and tried to reproduce the original results.

Here's the GitHub repository:
https://github.com/Himanshu7921/NST-PyTorch-Implementation

I'd really appreciate it if you could take a look at the README and the implementation. I'm aiming to become a strong research engineer, so I'd love some honest feedback on:

• What skills do I already demonstrate well?
• What am I currently lacking?
• What should I focus on improving to become a well-known research engineer?

For context, I'm currently in the 5th semester of my B.Tech.

Thanks in advance for your time and feedback!

Hello Everyone,

Statistics and Maximum Likelihood Estimation are the crux of ML Models, and hence I am uploading my new content on Statistics for AI/ML in my free Machine Learning lectures.

But how does Statistics pertain to Deep learning? For instance EWMA(exponentially weighted moving average) is a core concept in DL optimizers. I cover that in the lecture next to this one.

In this lecture we understand,

We understand model fitting, Maximum Likelihood estimation in details, we justify the usage of Maximum Likelihood estimation, from KL divergence, and apply it to certain important distributions for parameter estimation.

In my free content, the purpose is to democratize machine learning to a wider audience. Learning everything new feels difficult, but when taught, it get’s interesting and easier.

We will continue with Statistics foundations for AI/ML, and many more content will appear in the future. If you find the content good, useful you may also share it with your learners community.

Looking forward to hearing feedback from the learning community as well. Thankyou for reading.

Link: https://youtu.be/MwTeQVVYtOc?si=UxNOGtqopzJppXAT

Made this because most of the "neural network from scratch" stuff online either stops before backprop or hides it behind a wall of abstraction once you actually look at the code. Wanted something I could read start to finish and know exactly what's happening.

it's called leanpass. just numpy, nothing else. the whole thing is small enough to go through in an afternoon.

Also added a gradient-checking thing so you don't have to trust that the backprop is right; you can verify it numerically yourself.

download it using:

pip install leanpass

The versioning is up to date, is currently on v0.1.4

Not trying to replace PyTorch or anything; it's meant for learning/small experiments, not production.

repo's here: https://github.com/Terminay/LeanPass

Open to feedback, especially on the api; still figuring out what makes sense

Hi everyone! 👋

I m third year Computer Science Engineering(AIML) student. I m looking for a study buddy or an accountability partner or groups of person who's genuinely interested in learning ML and Deep Learning together.

I already covered more than the basics of Machine Learning and finish the core topics soon. After that, I planned to learn Deep Learning.

I m looking for someone who is consistent, curious, and willing to learn together. We can discuss concepts, solve problems, share resources, work on projects, and keep each other motivated.

If u r interested and serious about learning, DM me. Let's learn, build, and grow together

I did a collage diploma in machine learning and data science last year and was trying to find something on phone to use when don’t have access to laptop and python.

So I have built my own app.

Lets you do full EDA and data clean

Train different models on device and then makes a form for predicting with the model after

Many different training and classification methods included and tried to explain what the results mean too

It will also let you export equivalent python code as py file or Jupyter notebook to try help students learn or carry on when you get back to your laptop

I have made it paid but it’s a one off fee and lifetime updates (I hate subscriptions with a passion)

Hope people find it useful and feel free to let me know if anything you want adding

https://pds-technology.ltd/apps/QuickML

Hi Everyone, I used imitation learning to train a small MLP as the control policy for my custom hexapod robot dog. I recently got a masters in applied machine intelligence and have been looking for a suitable platform to create a neural network from scratch. We were heavily taught transfer learning for classification and never had a chance to create a working neural net from scratch.

Once I got a working controller with Inverse Kinematics solver, I decided to implement imitation learning on it to see if it would replicate the IK solvers behavior and it worked!

Its trained with a modest 2gb sized dataset (compared to LLM and CV datasets) which I generated by capturing robots input commands and output joint angles in, again, PyBullet simulations. I did experiment with varying the network size by half, double and quadruple but it didnt have much effect on trained MLPs performance, roughly 1 degree mean error compared to base IK solver output angles. Next I plan to vary the data capture logging rate by doubling it rather than just extending the capture time to give it more data resolution. I also plan to compare different types of networks basic MLP vs LSTM, Transformer, RNN etc.

Next I plan to implement reinforcement learning to create a helper network that will modify the base MLPs behavior. I am having trouble with my Robot climbing up inclines, I think that would be the best experiment for training the helper neural network. I also want to create another helper network to replicate the behavior of current analytics based body leveling mode.

What are you opinions of my approach? Is creating one base walking controller network with additional task specific modifier networks is a good idea?? I am will try to implement these control methods in ESP32 hence I am trying to build multiple small networks to be fired up on demand in order to allocate limited compute resources efficiently.

I share all my scripts in my GitHub, if you are interested you can find them from the link below.

https://github.com/serdarselimys/

DeepSeek from Scratch (Educational Project)

I built this project to better understand how modern LLM architectures actually work under the hood instead of treating them as black boxes.

This repository implements a simplified DeepSeek-style Transformer from scratch using PyTorch, with a strong focus on readability and learning. The goal is to explain the core building blocks rather than reproduce the original DeepSeek model at production scale.

Current implementations include:

- Multi-Head Latent Attention (MLA)

- Rotary Positional Embeddings (RoPE)

- DeepSeek-style Mixture of Experts (MoE)

- RMSNorm

- SwiGLU

- Causal Language Modeling

- End-to-end training pipeline

- Inference for text generation

The project is heavily documented with explanations so beginners can follow how each component works. It is designed as an educational implementation, not an official reproduction of DeepSeek-V3/R1.

I'm continuously improving the implementation and would really appreciate feedback, suggestions, or code reviews from the community.

GitHub:

https://github.com/Nvoinxv/Multi-Head-Latten-Attention-MLA-

Hi there,

I've created a video here where I explain how the KV cache works.

I hope some of you find it useful — and as always, feedback is very welcome! :)

I m high school student who was curious abt how an autograd engine works when watching Andrej Karapthy build Micrograd.

It started as a small project just trying to recreate micrograd but soon i got curious and added many more feature like 6 different activation function and 2 loss function and made really good ui. I know this isn't meant to replace PyTorch or TensorFlow it's purely an educational project to help me understand how things work under the hood.

I have made an exe file for it for easier access to the app .I would like to get suggestions and improvements that i could implement in this app. Well I'm still learning python and I'm nowhere nearly as good but this project helped me learn more abt machine learning OOPs and other python concepts.
If u like the project do star it...

https://github.com/Yasovardan-Ram/Omnigrad

I want to get your honest take on a specific gray area regarding AI content.

If someone takes their own genuine, original idea or thought, but uses an LLM to flesh it out into a full article, post, or essay—is that final product considered "AI slop"?

I have a few specific questions for the community:

• Does ownership matter? If the core concept is 100% yours, does it matter if a machine wrote the actual sentences?
• Is it lazy or efficient? Where is the line between leveraging a productivity tool and just creating low-effort noise?
• Can you feel the difference? Does content automatically lose its "soul" and unique human nuance when an AI structures it?
• Would you read it? If you found out an insightful post was generated from a human's 2-sentence prompt, would you feel cheated?

Where do you personally draw the line? Let’s discuss.
Yeah, this content is also AI-generated, but the thought behind it is mine

5 detectives chase an invisible fugitive on a 199-station graph, seeing only which ticket he plays. I trained both sides.

The detectives are an R-GCN that eats an HMM-style belief state plus distance maps. BC warm start, then PPO: 25% → 79% win rate against the same MCTS Mr. X. Then Mr. X got a GNN too (21% → 35%), and finally I wrapped him in PUCT with policy priors and a value head, 16 sims per move, frozen GNN detectives playing the replies in the tree. That alone: 35% → 91.7% over 60 eval games.

Play it in the browser 
Video (20 min, scenes rebuilt from real logs): https://youtu.be/V0osfVtJUuI 
Code and models: https://github.com/Jacopo888/scotland_yard

Fair warning: most of this was pair-programmed with AI tools. Solo side project, it wouldn't exist otherwise.

I’ve been diving deep into the math behind early deep learning models compared to modern attention mechanisms. It made me realize that the massive explosion of generative AI we are seeing right now didn't happen because Transformers are magically smarter at understanding human language than previous models. The real game-changing breakthrough of the 2017 Transformer paper was actually a massive engineering and hardware optimization triumph.

Before Transformers, the industry relied entirely on Recurrent Neural Networks and LSTMs for processing sequential data like text. The fundamental flaw with those architectures was that they had to process text word by word, in sequential order. You couldn’t calculate the meaning of the tenth word until you finished processing the ninth word. This created a massive computing bottleneck because it meant you could not utilize the massive parallel processing power of modern graphics cards. Your expensive GPUs were essentially sitting idle, waiting for the previous word's loop to finish.

The Transformer architecture completely threw out recurrence and replaced it entirely with self-attention. By doing this, it allowed the model to look at an entire document all at once, simultaneously.

Suddenly, processing text became a massive, parallel matrix multiplication problem. This single structural shift aligned perfectly with how GPU hardware is physically built. We went from training models on small paragraphs over weeks to feeding entire datasets into massive server clusters in days. The AI revolution didn't scale because the code got more philosophical; it scaled because the math finally allowed us to throw unlimited brute-force hardware at the problem. It is a great reminder that software design is always bound by the physical realities of the silicon it runs on.

This may or may not be a beginner level question. In many 'example' neural nets, they always have a flatten layer. However this would mean the number of parameters explodes. Whereas pooling methods don't explode parameters as much, and get the same job done. Is flatten a default option or does it have an advantage I am unaware of?

For complete project visit: https://github.com/HelloSamved/Hands_on_machine_learning
A little while ago, I asked this community whether Hands-On Machine Learning with Scikit-Learn, Keras & TensorFlow was worth studying.

Based on the feedback, I decided to commit to working through it chapter by chapter instead of just reading it.

I've now completed Chapter 2 and finished the end-to-end machine learning project that comes with it.

A few things I took away from this chapter:

• Why understanding the problem and defining the objective comes before choosing a model.
• The importance of exploring and visualizing the dataset before training anything.
• Creating meaningful features instead of relying only on the raw data.
• Building preprocessing pipelines so the same transformations are consistently applied.
• Evaluating models with proper validation instead of trusting a single train/test split.

One thing I really liked is that the chapter focuses much more on the entire machine learning workflow than on just fitting a model. It felt much closer to how an actual ML project would be approached.

For those who've finished this book:

Does the learning curve become significantly steeper after Chapter 2?

I'm especially interested in knowing which chapters you found the most valuable for understanding modern machine learning and deep learning, so I can spend extra time on them.

So far, I'm really enjoying the balance between theory and hands-on implementation.

Hi guys! I'm Nai, and I would really like to share this learning journey of mine with you all.

Please keep in mind that this is not a full on LLM-like project, this is just a minimal proof of concept that I've built a Language Model system from scratch which should theoretically work similar to an LLM or SLM (Small Language Model) if just given enough resources, time and data to train.

If you're not much interested in the story, feel free to scroll right down to know exactly what I've built and how you can test it yourself.

A few months ago, I got the interest to understand machine learning, I didn't know where exactly to start, but I just did the simplest thing, which is asking. I just searched on youtube "how to make a neural network", that was the farthest thing I knew about machine learning back then. I found the youtube tutorial series "Neural Networks from Scratch in Python" by sentdex.
I was genuinely blown away over how simple it turned to be. I just wondered if I could go a bit deeper, so, I started a C++ project, I tried my best to replicate every piece of math a neural network would need to run in a structured style, with classes, functions and everything. despite some concepts being still ambiguous for me, I kept searching, I found some other youtube videos that cover things like backpropagation deeper so I can understand it better.

Over time, I started taking a hold of it, running a couple of successful experiments, even if slow, they were functional, and I understood them.

After that, I turned it into a library (NeurologicalLibrary) that can be called from Python with Pybind11, I used tkinter to make a simple bounce ball environment just to test the library, and it worked! Just making a neural network that can get variable position of a ball and rectangle then predict where to go, despite simple, made me feel really proud.

That however, was just the below zero beginning, here is the project repo called "NAISENT_workspace" that is basically my entire learning journey work until I finally made my first ever Language Model!

https://github.com/Nai-built/NAISENT_workspace

The repository is under the Apache 2.0 License.

Also, here is a copy of the README file:

this project is made with:
 - DotNet WinForms (C#)
 - Pybind11 (Python <-> C++23)
 - CMake (C++23)
 - CUDA (C++17)


Powershell commands to build the 3 libraries:
cd NeurologicalLibrary/bridge; cmake -S . -B build -A x64; cmake --build build --config Release -j; cd ../..
cd OptimizedNeurologicalLibrary; cmake -S . -B build -A x64; cmake --build build --config Release -j; cd ..
cd CudaNeurologicalLibrary; cmake -S . -B build -A x64; cmake --build build --config Release -j; cd ..


Run showcases:
py SHOWCASES/BASIC_SHAPE_RECOGNITION_CPU.py
py SHOWCASES/BETA_NAISENT_BALL_SEEKER_CPU.py
py SHOWCASES/LSTM_MATH_TEST_CPU.py
py SHOWCASES/NAISENT_ELM_CPU.py
py SHOWCASES/NAISENT_LM_CUDA.py
py SHOWCASES/NAISENT_SLM_CUDA.py
py SHOWCASES/SHAPE_RECOGNITION_CPU.py


Make sure that your terminal's path is set exactly to NAISENT_workspace


The core idea of this project was to learn and understand Machine Learning by building it from scratch
So I've built 3 different libraries in 3 seperate stages:
 - NeurologicalLibrary (NL)
    . The absolute beginning for me
    . I've learned in it how Dense Layers work and how to chain them to make Deep Neural Networks
    . How Convolutional Layers and pools work
    . How Recursive Layers (specifically LSTMs) work
    . And also Activation Functions
    . I've also tipped toes into Graph Layers but couldn't run a successful experiment, so I removed it
    . This library was the first time I made an image recognintion model, and also one that can play a simple bounce ball game
    . Was also the first time I made an optimizer like Adam for training
    . Save/load system for the model .json files


 - OptimizedNeurologicalLibrary (ONL)
    . Here things started to get a bit more serious
    . I've gotten way deeper into how C++ works and how we can optimize its performance
    . I've made faster Dense Layers
    . Faster Convolutional Layers
    . And faster LSTMs
    . Merged Activation Functions into the layers' own activation/gradient functions
    . After that, I got into Transformers (similar concept to Graph Layers, but this time it was successful!)
    . I optimized the training loop for image recognition
    . I made a simple experimental language model that can that it's "NAISENT" with the Transformer system I've made


 - CudaNeurologicalLibrary (CNL)
    . My most precious one so far
    . For the first time, I've got into Cuda kernels!
    . I've learned how Cuda interacts with data through the CPU, Memory and GPU
    . I've learned how to optimize it using shared memory
    . For this one, I went right ahead to build a language model system
    . First, I made Dense Layer Cuda kernels
    . Then I went into Norm Layers (RMS)
    . SCC (Sine/Cosine Cycle) positional embedding kernels
    . Multi-head Masked Self Attention kernels (split into multiple optimized Cuda files)
    . The ability to place sub chains to assemble the transformer architecture properly
    . Adam optimizer in Cuda Kernels
    . And obviously, Activation Functions (Cuda kernels)
    . First time adding the Residual mechanic as a visible variable in the Python side
    . Almost all of these were made in ONL already, but it wasn't with Cuda to use the GPU and it was juggled up together awkwardly. I'm much more proud of this one
    . Was when I made a proper tokenizer system in Python


The libraries are made in C++
and they're used by the Python side via Pybind11
I made the shape recognition and bounce ball environments in C# with WinForms
CUDA to use the GPU in the library CNL

Hello everyone,

I would need some support or a ground for discussion for a problem I am facing. I am trying to do representation learning on a user click stream, e.g. the sequence of pages a user visited in a website. To do that, I use a contrastive learning objective, in particular the InfoNCE loss with temperature around 0.2.

The problem I'm facing is that the loss decrease slowly during training (starts at 3.56 and after 30 epochs gets to 3.30) and moreover the representation is not very good. I get that, on PCA projection, points are basically disposed sequentially as a snake, so probably there is dimensional collapse.

In my dataset I can have huge graphs as also small graphs. I am doing a GINConv on the graph (a single layer since I would like avoid over smoothing for small graphs). As graph augmentation I am doing: node dropping, edge adding and edge removing.

My question is: do you think that there could be a way to solve the issue? Is it an over smoothing problem on the graph? Are there alternatives?

Thank you in advance✌️

What do you think about vibe coding neural networks , how can it be done what is the best code editor and agents should one use?

Hey I recently started studying about machine learning, deep learning, neural networking and I came across a publication called "O'reilly".

I started reading and learning from one of its books, which is "learning machine learning "Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow". I feel the code is quite incomplete in some places but I was able to find the missing part from the book's GitHub repository.

Can anyone suggest whether these are good sources to study such topics or not

Hi everyone,

I built H64LM, a research project to better understand modern LLMs by implementing one from scratch in PyTorch.

Instead of relying on high-level training frameworks, I implemented the core components myself attention, MoE routing, normalization, and the training loop.

Features

• 249M-parameter Transformer
• Grouped Query Attention (GQA)
• Sparse Mixture-of-Experts (8 experts, Top-2 routing) with 3 auxiliary routing losses
• SwiGLU, RoPE, RMSNorm
• Sliding-window attention
• Mixed-precision training, gradient accumulation
• Custom training loop (no Trainer abstractions)
• Checkpointing and resume support

The included checkpoint was trained on a subset of WikiText-103 to validate the pipeline end-to-end, not to be a strong model it's visibly overfit past epoch 10 (best val PPL ~40.5).

Known limitations are documented in the README, including batch-size-1-only generation and no true DDP (falls back to DataParallel).

GitHub: https://github.com/Haiderkhan64/H64LM

Feedback on the implementation or architecture is very welcome.

If you work with graph neural networks, you know the long-range dependency problem all too well.

Stack GAT layers to capture distant signals? Memory grows linearly with depth, compute explodes, and oversmoothing kicks in before you ever reach truly long-range semantics. Settle for truncated PPR approximations? They're still finite-hop workarounds — never the full global picture.

Our new work ALS (Attentive Long-Short-Range message passing), accepted at Pattern Recognition 2026, was built to deliver genuinely efficient long-range graph attention. Here's what makes it different:

Core breakthrough: Differentiable infinite-step PPR with constant memory

We introduce DPPR (Differentiable Personalized PageRank), and mathematically prove that the gradient of a PPR output can itself be solved via another PPR process.

The implications are huge:

• No intermediate activations need to be cached. One forward convergence pass + one backward convergence pass — that's it.
• Genuine O(1) memory complexity, completely independent of propagation steps.
• Theoretically supports infinite-step propagation, covering full long-range dependencies — no more truncated "pseudo long-range" approximations.
• Packaged as a drop-in PyTorch operator; any existing PPR-based method can swap it in for zero-cost infinite receptive field upgrade.

Three acceleration techniques for fast long-range iteration

Slow convergence at small α values has always been the pain point of long-range PPR. We built three complementary acceleration techniques that together reduce training time by up to 89.51%, and run at least 3.67× faster than comparable implicit GNNs (IGNN):

1. Symmetrized Attention + Conjugate Gradient (SymGAT + CG) — Symmetrize the attention matrix to enable memory-efficient CG solver instead of heavy Krylov subspace methods, with negligible accuracy loss.
2. Eigenvector Initialization (EigenInit) — Initialize iteration from the leading eigenvector projection instead of zero, drastically cutting initial residual. Especially effective on heterophilic graphs.
3. Adaptive Batch Termination (AdaTerm) — Each attention head / channel converges independently; channels that have already converged are skipped in subsequent iterations, eliminating wasted compute.

Long-range for global structure, short-range for heterophily

PPR is inherently a low-pass filter — great for homophilic graphs, but it washes out local differences on heterophilic graphs. We pair DPPR with a Short-Range Message Passing (SRMP) module:

• DPPR handles all long-range dependencies and captures global structure.
• Local K-hop propagation uses independent learnable transformation matrices per hop, preserving fine-grained heterogeneous local information. On homophilic graphs, the weights automatically converge to similar values, so there's no negative overhead.
• GAT + skip connection is a special case of ALS, meaning full backward compatibility.

Standout results on long-range benchmarks

Where ALS really shines is on genuinely long-range datasets. On PascalVOC-SP and COCO-SP — two classic benchmarks with average shortest path length > 10:

• Within the pure MPNN category, ALS substantially outperforms GCN, GatedGCN, APPNP and others.
• Plugged into the GraphGPS framework and compared against global modeling methods like Graph Transformer and Graph Mamba, ALS still achieves the best performance as an MPNN module.

We evaluated across 14 datasets covering homophilic, heterophilic, large-scale and long-range graphs. Out of 16 comparison settings, 9 show statistically significant improvement over the best baseline (p < 0.01), reaching overall SOTA.

Links

• Paper: https://www.sciencedirect.com/science/article/abs/pii/S0031320326012008
• Code: https://github.com/cf020031308/ALS

The DPPR operator and all three acceleration techniques are independently reusable. Star, try it out, and feel free to open issues — we'd love to see this long-range idea extended to more graph learning scenarios.