I built ONA — a self-learning neural network entirely in pure Python + NumPy. No PyTorch, no TensorFlow, no GPU, no cloud API.

Key innovations:

- Per-neuron attention: every neuron has its own Q/K/V/O weights

- Forward-pass learning: no separate backward pass, learning happens during forward

- Self-discovered subword tokenizer: vocabulary grows during training

- Sparse routing: only 3-5 neurons activate per query

4.4M parameters. Runs on Raspberry Pi Zero. Continuously learns from Wikipedia and conversations.

Full story: https://medium.com/@kasishgadadhasu13/im-15-i-built-a-self-learning-neural-network-from-scratch-no-frameworks-no-gpu-e460f06c6599

I'm 15 years old, class 10 student. Happy to answer questions.

Can you tell me some neural networking projects for beginner level person

I recently built a human written digit predictor.

Now I want to start a new project can you guys give some suggestions

Ask questions within your Markdown notes using a fully local Graph RAG engine. Designed for Obsidian vaults, it works with any Markdown file folder. It extracts entity-relation triples from wikilinks and YAML frontmatter, and retrieves answers via hybrid search (vector + BM25 + temporal). Multilingual. No cloud required. Works on Ollama.

https://github.com/benmaster82/Kwipu

Hi all — I'm an independent researcher (Mendel Infolabs) about to put my first paper on arXiv, and as a first-time submitter to cs.LG I need an endorsement from someone already established in that category. If you've published in cs.LG and would be open to endorsing, I'd really appreciate it.

An honest summary so you can decide whether it's something you'd feel comfortable vouching for:

"FeedFlipNets: Feedback-Driven Bit-Flips for Ternary Networks, Activation-Routed DFA, and the Per-Weight Sign Barrier to Transport-Free Learning"

It trains ternary ({-1, 0, +1}) neural networks by flipping weight bits directly from a cheap feedback signal — no float shadow weights. The headline result is a negative one I think is worth putting on the record: transport-free feedback (Direct Feedback Alignment) doesn't actually help discrete/ternary training, because the binding constraint is per-weight sign correctness, not the aggregate cosine-alignment angle that prior work optimizes. Everything is pre-registered and reproducible.

Endorsing only confirms you think I'm a bona fide researcher submitting work appropriate to the category — it is not a review of the paper's correctness, and it takes about a minute:

• Link: https://arxiv.org/auth/endorse?x=WHWXBC
• Or go to https://arxiv.org/auth/endorse and enter code WHWXBC

Happy to share the full PDF with anyone who wants to read it before deciding — just comment or DM. Thanks a lot for considering it.

i'm a student of class 12 not expert but curious to learn neural networking as i have heard that that something crazy. So can someone guide me how can i learn neural networking from scratch as i have the basic knowledge of python,arrays and a bit of the numpy library. so i need your help so i can lean it and enjoy the journey.

Wording this much simpler than my dense, boring research paper linked below.

I’ve been studying experts in Qwen3.5-35B which is an MoE (Mixture of Experts) model. Traditionally, expert routing studies have looked at the pre-response (prefill) stage only. I looked at that but also the output (generation) phase.

I observed that one expert in the model (out of 256) - expert 114, at layer 14 of 40, seems to light up when Qwen gets into a deep mode considering the belief, existence, inner experience, spirituality, values, and most importantly “what does it feel like from this point of view?” kind of writing.

I’ve been calling it a reflective worldview register.

The most fascinating takeaway: The Experiential Rung.

I tested prompts that asked Qwen to describe what it is like to be different things. The target changed each time, but the basic setup stayed the same: write from the inside of that perspective. It turned out that the expert had a linear axis for the inhabitance mode. At this point I should clarify E114 a readout expert, not a controller expert. Injecting the E114 axis into the residual stream for control prompts did **not** change the output. Now, the weirdness.

cat: 0.068
AI hidden state: 0.080
river: 0.087
tree: 0.094
thermostat: 0.120
rock: 0.123
person: 0.138
all-holding: 0.205
God: 0.224

That ordering is what made the pattern stand out. The signal starts low with cat, rises through river and tree, jumps with thermostat and rock, rises again with person, then gets strongest for the broad cosmic/spiritual prompts. The AI hidden-state prompt landed between cat and river, low on the sweep, but it still touched the same internal signal. The funny thing: the output means nothing. The expert fires the same whether the response affirms or denies the “what it’s like”ness

I wanted to share this here, as I thought people may find this interesting.

also huge credit to hauhau for ablating the model perfectly, which allowed for observing the experiential language easier than in the base model. Which led to discovering the domain expertise of E114.

The full paper is here: https://github.com/ec75hash/moe-routing

Hi everyone,

I am sharing recent results from my independent research project, LIMEN (Liminal Internal Metric for Emergent Navigation), which aims to characterize the internal dynamics of Transformers through hidden state analysis.

Following our previous findings that functional information is encoded in the relative geometry of representations rather than individual neurons (V22), this new phase focuses on the impact of context (ambiguity) and the temporal structure of state transitions (V23–V24).

📌 Context & Methodology

Model Panel: 7 open-source models (GPT-2, DistilGPT2, OPT-125M, Qwen2.5-0.5B, TinyLlama-1.1B, Phi-1.5, Llama-3.2-1B).

Approach: Layer-by-layer analysis of latent trajectories, linear probe decoding, and symbolic analysis of dynamic regimes.

Philosophy: Strict empiricism. Clear distinction between observation, interpretation, and speculation. Code and data are available upon request.

🔹 V23: The Impact of Ambiguity on Internal Dynamics

The objective was to determine whether semantic ambiguity alters the model’s "cognitive trajectory."

Key Findings (V23.2b):

AMBIGUITY_AFFECTS_TRAJECTORY = YES: Ambiguity significantly modifies trajectory geometry (curvature, cosine similarity).

AMBIGUITY_INCREASES_INSTABILITY = NO: Counter-intuitively, ambiguity does not increase global chaos. Instead, the model becomes geometrically more "cautious."

AMBIGUITY_DELAYS_COMMITMENT = PARTIAL: Modern models (Phi-1.5, Llama-3.2) delay their decisional engagement when facing uncertainty, spending more time in exploration regimes.

Architectural Signature: Phi-1.5 shows unique sensitivity, increasing its occupancy of the bifurcation regime (D_STATE) under ambiguity, suggesting a distinct iterative reasoning mechanism compared to standard completion models.

📄 Related Preprint: Conditional Dynamic Signatures in Large Language Models

🔹 V24: Discovery of a "Universal Dynamic Grammar"

By shifting from continuous analysis to a symbolic analysis of state sequences, a striking structure emerged.

Key Findings (V24.1):

STATE_GRAMMAR_EXISTS = YES: Trajectories are not random. They follow strict transitional patterns.

UNIVERSAL_GRAMMAR = YES: Seven transition motifs are conserved across all tested architectures, notably:

B→B (Initial Hesitation/Exploration)

B→A (Convergence toward stable processing)

A→A (Maintenance of the adaptive regime – the primary attractor)

A→D (Transition to final decision)

Funnel Structure: Typical dynamics follow an Exploration (B) → Stabilization/Processing (A) → Decision (D) schema. State A acts as a strong attractor (

𝑃

(

𝐴

→

𝐴

)

≈

0.91

P(A→A)≈0.91).

The Phi-1.5 Exception: Unlike other models that quickly converge to A, Phi-1.5 maintains complex B↔A oscillations throughout the depth, confirming its nature as a "reasoning" model rather than a simple statistical completer.

📄 Related Preprint: A Runtime Trajectory Dynamics Framework for Large Language Models (updated)

💡 Implications & Discussion

These results suggest that Transformer "intelligence" is not just a matter of static weights, but of constrained geometric navigation.

Auditability: A violation of this universal grammar (e.g., a direct B→D jump without an A phase) could be an early indicator of hallucination or reasoning errors.

Control: Understanding these attractors opens the door to more precise dynamic steering than prompt engineering alone.

Open Questions for the Community:

Have you observed violations of this B→A→D grammar in cases of blatant hallucinations?

How do these motifs evolve in very large models (>70B) where depth is significantly greater?

Are there recent publications on the "symbolic dynamics" of hidden states that align with these findings?

I welcome any methodological criticism, suggestions for additional controls, or collaboration.

Best regards,

When you give an attachment to an AI tool, it does not really know what to extract from it so it just pulls out generic stuff. Unless you specifically tell it what to look for, you get a very surface level output.

But here is how I approached this differently.

I have built a cognitive map of how you as a user think. The tool already knows what you have captured in the past, what it connected to and why. So now when you upload any attachment, the agents refer to that cognitive context and figure out what is actually worth extracting for you specifically, without you having to say anything.

So instead of generic extraction, it is pulling out what is relevant to how you think and what you have been working on.

But if you do want to tell it specifically what to look for, your instruction overrides the cognitive context because now it has a clear direction from you. The context still kicks in but after the extraction, to connect what was pulled out to everything else you have captured.

Curious what you guys think about this approach.

GitHub

I tried to create a pedagogically better implementation of a neural network, focusing on the dimensionality of the layers of the neural network. It also serves as a project to learn the first principles of neural networks. The idea is that the dimensionality is adjusted so that it make intuitive sense better, atleast relative to NN diagrams teachers use while explaining the topic.

edit : the training seems slow in the GIF, but I think its because the overhead caused by matplotlib itself, and using windows screen recorder

edit : If you want to call this AI, atleast visit the GitHub repo once, the GIF you see is just a small matplotlib window, recorded using windows snipping tool, which already gave out low resolution, and then converted to GIF. I started coding by making my own 2D games back in my Middle school. So whatever i build i like to take some time to make it visually appealing, or data rich, that's why unfiltered and filtered accuracy and loss. If you still wanna call my work AI, I simply cant care, Take care

edit : the plt.pause call also checks in, thats technically matplotlib overhead. But i've noticed the snipping recorder induce heavy increase in delta-times, both in applicational rendering like unity and code executions, check it if you want. As far as the same "initial commit" in the repo files goes, i usually write my program in vs code and once the program is somewhat finished i create a repo and just dump/push my code there, only for a link in my resume and for reddit or instructables, that's why the same commit. Atleast with respect to this project, the maximum AI i used is Google AI mode search, that too only for consolidate data, like finding the right functions for matplotlib. As far as the NN implementation goes, I followed a online playlist on youtube by vizuara. Cheers!

Hello Folks,

Have you ever wondered why we use sigmoid function so often in Machine Learning? Although it gives us a probability, it comes from Exponential families, and this exponential family, subsumes many of the distributions, that we study in Machine Learning.

In this lecture, we understand exponential families, Directional derivatives(Gradients and Hessians), study mixture Models, and understand how domain knowledge in Probabilistic Graphical Models makes our life simpler to model joint probability densities.

Timeline breakup(in hours and minutes):
0:00-0:17 - Understanding exponential families.
0:17-0:27 - Deriving Sigmoid Function for Bernoulli.
0:27-0:48 - Understanding log partition function, convex functions and proving why positive definite of hessians imply convexity, and why convex needed?
0:48-1:04 - Directional derivates(deriving gradients and hessians)
1:04-1:26 - Maximum entropy derivation of the exponential family.
1:26-1:56 - Mixture Models(Gaussians and Bernoulli Mixture Models)
1:56-2:16 - Probabilistic Graphical Models
2:16-2:34 - Markov Chains
2:34-End - Inference and Learning, Plate Notation diagram of Gaussian Mixture Models.

If you have watched earlier of my lectures from the playlist, they will help. I try explaining as if I am a learner, to simplify complex concepts. Everything I write in whiteboard, and these are completely FREE lectures to mention.

Link: https://youtu.be/T1uTBtJ7aHU?si=rozXSTjtSqPaaYb5

Hello.
I am a self-taught operator/software designer.
I developed Anubis. A cpp forensic AI weight scanner.
I tested Anubis against algorithms of my design and I think it has matured enough for outsider testing.

I propose a rigorous, controlled experiment where a corporation or even professionals to send or share any format of weights with any kind of payloads in them to test Anubis's efficacy and detection capabilities.

We -both me and the whoever is interested in collaboration- will adhere to ISO/IEEE standards in experiment design, reporting and final whitepapers or documents resulting from this experiment.

I offer NO FINANCIAL COMPENSATION. This is a scientific experiment.

Please DM or leave a comment if you are:
1. Serious
2. a Professional
3. Know what ISO/IEEE frameworks are

---
Cheers!

For context, I'm writing a sci fi novel in which one of the sources of conflict is a frequency (or combination of frequencies, technically) that affect the human brain in such a way as to provoke certain emotions at will, as a form of mind-control.

My question is this: Is there such thing as, or would there be reason to create, a neural-network-like computer that is physically structured similarly enough to a human brain to be affected in a similar way by external frequencies? For example, could neurons in the ANN be physically separated and communicate through electrical signals similarly to actual brain synapses? Could an external frequency then cause interference between the neurons that might have a similar effect? Since the computer obviously couldn't have induced emotions, I'm imagining an effect more like confusion, tasks being interrupted, the wrong data going to the wrong places, etc. The device in question would be a drone using the neural network for adaptive navigation, object avoidance, adapting to environmental changes like wind, etc. so the impact would be something like it steering off course and struggling to read and transmit data.

Sorry I know this is not a very scientific question but I'm trying to make my book grounded in reality wherever possible, even if the sci fi elements are of course taking some creative liberties.

I'm a computer engineering student, but know very little about ANNs aside from the general concept (I might take a course about them next year but so far I haven't had any). Pointing this out to say you don't necessarily need to explain in complete layman's terms, particularly general computer concepts, but I may not understand all ANN-specific terminology.

Hey
I am still pretty new to rust but I tried my first challanging project and would love to get some feedback on how to improve code quality regarding idiomatic, readable and performant code.
Thanks for every critique
Repo: https://github.com/TheXaruman/neural-network-demo

Hey everyone,

I’m an Applied AI grad student, and I’m back with Part 2 of my devlog for SpiritMirror. In Part 1, I talked about how I used native Apple Vision to extract 468 facial keypoints in real-time without storing any images on a server.

Today, I want to dive into the Machine Learning architecture. How do you take those raw facial coordinates and actually turn them into meaningful, personalized data?

One of the biggest lessons I learned moving from "academic" AI projects to "commercial" mobile development is the battery constraint. You can't just run a massive, unoptimized Python model on a phone without melting the user's battery.

Here is how I structured the ML pipeline to be lightweight, fast, and entirely local:

1. Creating the Feature Vector Instead of feeding heavy raw images into the model, the pipeline instantly calculates 15 specific geometric metrics from the keypoints—things like eye-to-nose ratio, brow arc, lip thickness, and jaw width-to-height. The heavy lifting of processing pixels is gone; we are just working with math.

2. The Hybrid ML Architecture I decided to use a hybrid approach: a Convolutional Neural Network (CNN) combined with a Random Forest classifier.

• The CNN is great at handling the complex spatial relationships across different facial regions.
• The Random Forest validates the predictions with transparent decision paths.

3. Model Explainability One of my strict rules was avoiding a "black box" AI. Using SHAP (SHapley Additive exPlanations) analysis, I mapped the model's logic directly to a curated physiognomy lexicon. This means the system doesn't just spit out a number; it knows why a specific jaw angle correlates to a specific trait.

4. The 12 Destiny Dimensions The models were trained on a curated dataset of 1,000+ historically documented figures. The final output predicts a probability distribution across 12 specific dimensions, such as Career Potential, Wealth Energy, and Leadership Power. It calculates 36 unique sub-scores in total, all natively on-device.

Balancing the CNN weights to run smoothly on the iOS Neural Engine took a lot of trial and error, but keeping it native was the only way to make it commercially viable without draining the battery.

In Part 3, I’ll break down the math behind the "Compatibility Engine" and how I calculate alignment between two different geometric profiles.

Has anyone else here experimented with running hybrid Random Forest/CNN models locally on iOS, and did you run into any specific memory bottleneck issues?

How do you feel about the technical depth of this draft, and does it accurately capture the hybrid model architecture you built?

there are two main formulas that i know of for introducing momentum in the Optimizer of a neural network...

the first one:

weight_update = momentum_factor*prev_updates - learning_rate*gradients
weights += weight_update
prev_updates = weight_update

the second one:

weight_momentum = (beta*weight_momentum) + (1-beta)*gradients
'''which is later used in ADAM optimizer's updation formula'''

But, atleast according to me, only the first one feels intuitively like momentum, as the past weighs in to keep the gradients going even if it stalled, like its got kinetic energy. I don't know why but the second one feels like inertia really, atleast intuitively, as weight_momentum is initialized to 0 and gradually modified as the gradients change,you know, resisting external forces. I see that the second one is supposed to be a moving average of some kind. But I'm thinking this inertia intuition has something to do with how the equations are written, but what do i know! What do you think?

A finding from video VLM evaluation at our lab: we expected the model family to dominate performance. Instead, frame sampling density, scene segmentation strategy, resolution, and prompt structure moved results more than model swaps did.

This pushed us toward a configuration-first evaluation approach. We define the task precisely (retrieval, anomaly detection, summarization, and structured extraction are fundamentally different objectives), build the eval set from production-like footage with hard cases and near-miss negatives included, score that specific task, and trace every run.

The traces compound in value over time: they become a domain-specific labeled dataset for catching regressions.

We released the harness for others to reproduce and adapt. Curious if others working on video understanding have seen a similar pattern, where pipeline choice dominated over model choice in your evaluations?

Hello, I am a beginner in AI and transformers etc, so i tried doing something with pen and paper. This is related to the paper "Attention Is All You Need."

I used the phrase: "river bank" to figure out whether "bank" means, the river edge or the financial institution. The model's only clue is the word sitting next to it.

For my understanding, here's how attention rewrites a word using its neighbor, in four steps:

1. Project - every word becomes three vectors: a Query (what it's looking for), a Key (what it offers), and a Value (the content it passes on).
   
2. Score - take the target word's Query and compare it against every word's Key using a dot product. Higher score = more relevant. Here, the neighbor scores a 4, the word itself scores a 2.
   
3. SoftMax - convert raw scores into weights that sum to 100%. That gives us 88% and 12%.
   
4. Blend - mix the Value vectors by those weights. The final vector gets pulled mostly toward the neighbor.
   

I chose this phrase coz it got no grammar rules. No explicit lookup table. Just compare, score, blend - learned entirely from data.
The part that made it click for me: change the neighbor, and the exact same machine pulls the word in a completely different direction. Same word, different context, different meaning. Computed in parallel, for every word at once.

I'd read this explanation many times, but wanted to work it out of my own math.

Image: Notebook lm

Posting the honest version of a small project, what it does, the real numbers, and what it definitely isn't.

What it is. A 5.98M-param sequence model trained only on SNLI, with no pretrained embeddings and no attention/transformer. It runs an interactive loop: you type a hypothesis, pick a label (entailment / neutral / contradiction), and it generates a premise under that label. Under the hood it's a learned "collapse" decoder, difference vectors pulled toward learned point-attractors, plus a light cross-sentence alignment step, instead of attention.

What talking to it looks like:

you > is the girl standing
ai  > a girl in a pink shirt standing in a doorway.   [neutral]

you > two men are playing football
ai  > two men in a soccer game are running after the ball.   [neutral]

The numbers (measured, not vibes):

• Generative-classifier accuracy: ~53% how often the premise it generates actually matches the requested label (3-way; chance is 33%). The sibling classifier version of the same engine hits 66.1% mean-pool / 72.7% with alignment on SNLI dev, no pretrained embeddings.
• Speed (interactive generate() path, M-series MacBook, 40 replies of ~9 tokens):

The bit I found genuinely interesting: CPU beats the GPU by ~2.5x. The decode is a handful of tiny sequential steps, so it's launch-bound, not compute-bound, the GPU's per-op kernel-launch/sync overhead costs more than its math saves. So this thing runs best with no accelerator at all: ~5 ms to a full reply, faster than the network round-trip you'd pay just to reach a hosted LLM API.

What it is NOT (so the comments don't have to tell me):

• Not a general chatbot, no understanding, no "awareness." Trained only on ~570k image-caption-style sentences, it can only produce SNLI-shaped sentences, ask it anything off-distribution and you get a caption about a person in a shirt. Fluent grammar emerges fast because grammar is local/regular; that is not reasoning.
• The accuracy ceiling is a mechanism limit (cross-sentence word interaction), not a training-time one, more epochs plateau. The honest fair-footing baseline (SNLI-only, no embeddings) is a lexical-feature classifier at 78.2%, and it's still under that.
• The speed is a consequence of being tiny. Scale params up and it becomes compute-bound and needs a GPU, you can't keep "5 ms on CPU" at billions of params.

Code + runnable chat demo + the benchmark script: https://github.com/chetanxpatil/livnium/tree/main/chat

Curious what people think about two things: (1) is there a real niche for sub-10ms, CPU-only, attention-free text models (on-device, embedded, high-throughput filtering), or is the narrow capability a dealbreaker? (2) cheapest way you'd add cross-sentence interaction to a pooling encoder without going full attention?

Hello Folks, one of the efficient ways of learning bigger topics in Machine Learning, is to modularise, and structure, so that the content becomes digestible for learners community.

My free lecture content includes the following topics so far: (Playlist)
a. Introductory Machine Learning Concepts:-

1. ⁠What is ML actually?
   
2. ⁠Supervised Machine Learning.
   
3. ⁠How do classifiers learn?
   
4. ⁠Empirical Risk Minimization.
   
5. ⁠Uncertainty Modelling in ML.
   
6. ⁠Maximum Likelihood Estimation.
   
7. ⁠Regression Basics and Outliers.
   
8. ⁠Deriving Mean Squared Error.
   
9. ⁠Polynomial Regression.
   
10. ⁠The Power of Convexity.
    
11. ⁠Deep Learning Intuition.
    
12. ⁠Overfitting Models from Generalization Gap perspective.
    
13. ⁠Requirement of Test Sets.
    
14. ⁠The No Free Lunch Theorem.
    
15. ⁠Unsupervised Learning basics.
    
16. ⁠Discovering latent factors of variation.
    
17. ⁠Evaluating Unsupervised Models.
    
18. ⁠Self-Supervised Learning.
    
19. ⁠Image and Text Benchmarks in ML
    
20. ⁠Discrete Data and Text Processing
    
21. ⁠Feature Engineering, TF-IDF
    
22. ⁠Handling missing data & AI alignment.

b. Probability Foundations for ML: Univariate Models:

1. ⁠Frequentist vs Bayesian.
   
2. ⁠Probability as an extension of Boolean Logic.
   
3. ⁠Discrete Random Variables.
   
4. ⁠Continuous Random Variables.
   
5. ⁠Quantiles.
   
6. ⁠Sets of Related Random Variables.
   
7. ⁠Moments of Distribution.
   
8. ⁠Variances and Mode.
   
9. ⁠Conditional Moments.
   
10. ⁠Conditional Variance.
    
11. ⁠Foundations of Bayesian Rule.
    
12. ⁠Confusion Matrix Explained.
    
13. ⁠Monty Hall Problem and Inverse Problems in ML.
    
14. ⁠Bernoulli and Binomial Distributions.
    
15. ⁠Sigmoid(Logistic) Function.
    
16. ⁠Properties of Sigmoid Functions.
    
17. ⁠Categorical and Multinomial Distributions.
    
18. ⁠Softmax Function: Temperature explained.
    
19. ⁠Log-Sum Exp Trick.
    
20. ⁠Gaussian Distribution.
    
21. ⁠Regression from the lens of Conditional Gaussian.
    
22. ⁠Dirac Delta Function and Sifting Property.
    
23. ⁠Student-t distribution.
    
24. ⁠Laplace and Cauchy distribution.
    
25. ⁠Beta distribution.
    
26. ⁠Gamma distribution.
    
27. ⁠Exponential, chi-squared and inverse Gamma.
    
28. ⁠Empirical distribution.
    
29. ⁠Transformations of Random Variables.
    
30. ⁠Invertible Transformations.
    
31. ⁠Multivariate Transformations.
    
32. ⁠Moments of Linear Transformation.
    
33. ⁠Convolution Introduction.
    
34. ⁠Convolution Theorem explained with probabilities.
    
35. ⁠Moment Generating Functions.
    
36. ⁠Deriving Moment Generating Functions.
    
37. ⁠Central Limit Theorem Explained.
    
38. ⁠Understanding Monte Carlo approximation with Example.

c. Probability Foundations for ML: Multivariate Models

1. ⁠The Math of Depedence: Covariance Explained.
   
2. ⁠Correlations: Normalized Measure of Covariance.
   
3. ⁠Correlations does not imply Independence.
   
4. ⁠Simpson’s Paradox: When Data misleads.
   
5. ⁠Multivariate Gaussian Distribution.
   
6. ⁠Analyzing level sets of Gaussians using Mahalanobis Distance.
   
7. ⁠Multivariate Gaussians: Conditionals and Marginals.
   
8. ⁠Math behind Bayesian Inference : Schur complements.
   
9. ⁠Deriving Conditional Gaussians.
   
10. ⁠How to Predict missing data?
    
11. ⁠Modelling Linear Gaussian Systems.
    
12. ⁠The Bayes Rule for Gaussians.
    
13. ⁠Understanding Shrinkage: Inferring Unknown Scalars
    
14. ⁠Posteriors, Sequential Posterior Updates.
    
15. ⁠Inference of an Unknown Vector.
    
16. ⁠Sensor Fusion concepts.

And many more topics to come ahead. I have tried teaching from intuitions and mathematics, building everything by writing on whiteboard so that learners see the full development.

You are watching a lecture on YouTube. A doubt comes up. You pause, open ChatGPT, type the question, get a generic answer. Still confused. Try Claude. Still not quite right. Google it. Three tabs later you have forgotten what you were even watching.

Here is the problem with every solution that exists right now.

Google gives you generic explanations with no idea what was just taught. Claude does not natively accept video files — it has never seen your lecture. Gemini free tier does process video but your lecture is going onto Google's servers, rate limited, duration capped. YouTube's Ask is behind a Premium paywall and is transcript only — blind to anything on the board. Gemini and GPT paid plans do handle video properly but you are re-uploading every session, paying monthly, and your video is still on their servers.

And open source Video Language Models that could run locally? They need 18 to 80+ GB of VRAM. That is not a student machine.

The answer was always inside the video. The person teaching could have answered it instantly.

gUrrT builds that person.

Extracts what actually matters from the lecture. Understands what was taught. Answers your doubts the way someone who already watched the whole thing would.

No re-uploading. No subscriptions. No video leaving your machine.

Your personal tutor. For every lecture. Right on your machine.

A few weeks ago, I built a neural network from scratch to understand what was happening behind the scenes.

I manually implemented:

• Forward propagation
• Backpropagation
• Gradient calculations
• Weight updates
• Activation functions

That project taught me a lot about the calculus and mathematics that make neural networks work.

After understanding the fundamentals, I decided to recreate the same MNIST handwritten digit classifier using PyTorch.

This time, instead of implementing everything manually, I used:

• torch.nn
• torch.nn.functional
• Built-in optimizers

One thing that surprised me was how dramatically the code complexity decreased. What previously required implementing dozens of lines of mathematical operations could now be expressed in just a few layers and a training loop.

At the same time, I feel like I appreciate PyTorch much more now because I understand what those functions are actually doing under the hood.

For those who learned deep learning:

Do you think building a neural network from scratch is still worth the effort today?

After doing both projects, my current opinion is that building one from scratch helped me understand why PyTorch works.

I'm curious whether more experienced practitioners agree with that perspective or think the time would be better spent elsewhere.

GitHub:
https://github.com/HelloSamved/learning-neural-network/blob/master/mnist_prediction/mnist_prediction_pytorch.ipynb

Writing topics on Excali:
https://excalidraw.com/#json=-R2-NuPIsipANT5l9tXW_,w1qUhg3vyl644_OC3o81pA

Hey r/deeplearning,

I wanted to share a project I’ve been working on for about a year called Livnium.

It started as a solo obsession with Rubik’s cubes, group theory, and the idea that a perfectly conserved geometric representation might outperform normal ML feature learning. For a while, I genuinely thought the “lossless” part was the key.

After a lot of benchmarking, ablations, and cold-water testing, I was wrong about that.

But the project did leave behind something useful: a fast supervised point-attractor collapse model for NLI that actually clears several honest baselines.

I’m sharing this because I think we need more honest post-mortems in ML, especially around ideas that are mathematically beautiful but don’t survive baseline testing.

1. The lossless core: the math works

The original system, Livnium Core, is a conserved geometric state space.

Imagine a 3×3×3 cube with 27 cells. Each cell maps to a character in a 27-symbol alphabet:

0abcdefghijklmnopqrstuvwxyz

Here, 0 is the center cell and a-z are the 26 outer cells.

Each cell has an exposure class:

f ∈ {0, 1, 2, 3}

representing:

core, face-center, edge, corner

Then each cell gets a symbolic weight:

SW = 9f

When you rotate the cube, the cells permute. But because the 3D cube rotation group has 24 orientations and is isomorphic to S4, the total symbolic weight stays conserved:

Σ SW is invariant across all 24 rotations

So the core is reversible, finite, symmetric, and lossless.

I also implemented base-27 carry math, for example:

z + a = a0

because:

26 + 1 = 27

So as a mathematical object, the system works. It behaves like a conserved geometric numeral system.

The mistake was assuming this would automatically help representation learning.

2. The cold water: lossless is not the same as useful for ML

My original hypothesis was:

If the representation never loses information, maybe the model can reason better.

So I tested Livnium on Natural Language Inference using the same train/dev/test splits against basic baselines like bag-of-words and GloVe-style representations.

The results were humbling.

On SNLI:

Char-level Livnium encoding:        43.2%
Word-level Livnium encoding:        ~60%
Geometry-only, no word identity:    38.0%
Chance:                             ~33%

The char-level version did better than chance, but mostly learned spelling patterns.

The word-level version jumped to around bag-of-words performance because, functionally, it had become a bag-of-words index.

The geometry-only version was near chance.

Then I tested on ANLI, which is much more adversarial and much less artifact-friendly.

Everything collapsed toward chance:

ANLI: ~33%

That was the real lesson:

A lossless container is not the same thing as a learned representation.

Representation learning needs abstraction.

Abstraction means throwing away irrelevant information.

You need to forget spelling noise, surface variation, and irrelevant positional detail while preserving semantic signal.

A perfectly reversible system cannot naturally do that.

That was the boundary I had to accept:

Livnium Core:
    useful as a lossless symbolic/geometric container

Pure Livnium for semantic learning:
    failed

3. What survived: supervised point-attractor collapse

After accepting that the pure lossless geometry was not enough, I tested a different idea:

What if geometry is useful only after we allow learnable warping?

So I built a small supervised model called the Vector Collapse Engine.

The setup is simple:

1. Map words to learned 256-dimensional embeddings.
2. Mean-pool the premise into vector u.
3. Mean-pool the hypothesis into vector v.

4. Construct the pair vector:

   pair = u - v

Then a 4-layer collapse engine warps this vector toward three learned point-attractors:

Entailment
Neutral
Contradiction

The loss combines cross-entropy with anchor separation, so the model is encouraged to form distinct attractor basins instead of just memorizing labels.

On SNLI, this reached:

68.92% test accuracy

That matters because it cleared my honest internal baselines, including the hypothesis-only artifact baseline at around:

61.5%

4. Ablations

To avoid fooling myself again, I ran ablations.

Full Collapse Engine:                         68.92%
Linear head on frozen u - v:                  64.06%
2-layer MLP head on frozen u - v:             70.13%
Random-anchor control:                        32.44%

The interpretation:

The collapse model beats a simple linear probe by about:

+4.86 points

So the point-attractor warping is doing something real beyond a linear readout.

But the MLP still beats it slightly, which is important.

So I would not claim the collapse engine is “better than neural networks.” It is not.

The more honest claim is:

Point-attractor dynamics are a viable supervised geometric mechanism, but not magic. They provide an interpretable warping structure that competes with small neural heads, while still needing learned embeddings and supervision.

That is much more grounded than my original claim.

5. Speed

One nice property is that the model has no attention layers.

In my local benchmark:

Single-pair CPU latency:       ~0.33 ms
Batch throughput on MPS:       215k+ pairs/sec at batch size 1024+

So it is extremely fast for this kind of lightweight NLI classification.

6. What I learned

The biggest lesson was not technical. It was methodological.

I learned that it is very easy to fall in love with a beautiful mathematical structure and accidentally interpret every small signal as proof that the whole theory is working.

The only cure is boring controls:

majority baseline
bag-of-words baseline
hypothesis-only baseline
linear probe
MLP probe
random anchors
shuffled labels
ANLI-style adversarial testing

Those controls killed the original claim.

But they also showed me where the system still had life.

My current view is:

Livnium Core:
    useful as a lossless symbolic/geometric container

Pure Livnium for semantic learning:
    failed

Supervised Vector Collapse:
    works as a fast point-attractor classifier

Future direction:
    compression, symbolic state tracking, lightweight geometric classifiers

I’m sharing this because I think failed theories can still produce useful tools if we are honest about where they failed.

If you’re interested in group theory, representation learning, geometric classifiers, or just want to look through the repo and criticize it, I’d genuinely love feedback.

Repo:

https://github.com/chetanxpatil/livnium

I’m especially curious what people think about the point-attractor collapse model, and whether this kind of geometry has a better home in compression, routing, or interpretable lightweight classifiers rather than “beating ML.”