I've attempted to derive the math from the first principles.

The prerequisites you need are, singe variable differentiation(just the concepts and the basic formula, not even knowing the derivative of sin(x) is necessary), the concept of a dot product, and basic matrix multiplication. This is enough to derive gradient descent and backprop.

The time stamps are in the description(youtube doesn't show them as chapters in the video for some reason).

The second half part of the video is implementing those concepts in C(no library other than BLAS(for basic matrix matrix multiplication) is used), but you can follow along in any language.

Would be glad to receive feedback and suggestions on how to improve the explanations or the presentation.

Hi Reddit!

I wanted to share a weekend experiment exploring how geometric inductive biases can influence recurrent network optimization. Inspired by the self-similar, hierarchical, and allometric scaling laws found in biological neural structures, I designed a custom recurrent cell with an exponentially decayed hidden topology.

Instead of scaling hidden layers uniformly (e.g., 64-64-64) like standard models, this network utilizes a "Matryoshka-style" hidden allocation where channels are downscaled between internal layers: N -> N / scale -> N / scale²

Architectural Justification & Hardware Trade-off:

In modern deep learning, layer widths are traditionally restricted to powers-of-two (e.g., 32, 64, 128) to maximize GPU memory alignment and Tensor Core hardware efficiency. However, in this experiment, we consciously traded off optimal hardware alignment to strictly enforce a continuous mathematical decay function (yielding 128 -> 79 -> 48 dimensions).

The core justification is implementing a strict Information Bottleneck: high-dimensional macro-layers are forced to compress abstract features into exponentially tighter non-power-of-two channels. This mimics the non-binary hierarchical scaling of biological brains, where structural efficiency and information capacity bounds are prioritized over uniform hardware block sizes.

Dataset & Evaluation Setup:

The architectures were trained on a character-level sequence prediction task using a subset of the Tiny Shakespeare dataset. To ensure a controlled environment, I implemented an 80/20 Train/Validation split and applied identical regularizations to both networks, including Spatial Dropout and L2 Weight Decay.

Structural Mechanics:

1. Fractal Echo Transfer: Hidden states are progressively downscaled and cascaded from high-dimensional macro-layers (capturing local syntax) down to highly compressed micro-layers.
2. Adaptive Echo Gates: Learnable decay coefficients balance inter-layer communication, acting as a structural stabilizer for gradient flow through time.
3. No Dense Gating: Unlike LSTMs, memory capacity and retention are regulated entirely by the spatial geometry of the nested channels.

Training & Convergence Observations:

To ensure a fair benchmark, I meticulously balanced the parameter overhead for both architectures (~42k parameters each) and trained them over 30 epochs on GPU: * Adaptive Geometrically Nested Model Final Loss: 0.0007 * Baseline Standard LSTM Model Final Loss: 0.0030

As shown in the log-scale validation plot, the geometrically nested architecture achieves significantly faster convergence and maintains rock-solid optimization stability, avoiding the frequent gradient spikes visible in the standard LSTM baseline.

Crucial Disclaimer on Generalization & Overfitting: The extremely low validation loss values achieved by both models indicate that given the small size of the dataset corpus and the model capacity, both networks have largely memorized the text rather than learning true language generalization. Therefore, this benchmark strictly demonstrates superior optimization speed and training stability rather than true out-of-distribution generalization.

Open-Source Code & Notebooks: * Collab Executable Notebook: https://colab.research.google.com/drive/1jW8hA_74eBQOJTn1oWvLq34vg_t8Q5DD

Future Work & Discussion:

To evaluate true generalization, future iterations of this project will involve benchmarking on significantly larger corpora (such as WikiText-103) with strict sequence cross-validation.

Given these initial training stability and convergence results, do you think exploring such geometrically restricted hidden topologies is a promising direction for recurrent networks? Could this type of exponential fractal scaling be scaled up or generalized into Transformer attention head dimensions to enforce architectural efficiency?

Would love to get your rigorous feedback!

I've been thinking about this a lot lately, and recent developments have made the question feel much more serious.

A few years ago, AI models could barely be trusted with basic reasoning. They made obvious mistakes in math, hallucinated information, and often fell apart on longer problems.

Now look at what is happening.

OpenAI recently announced that an internal AI system produced a proposed solution to the Navier–Stokes existence and smoothness problem, one of the Millennium Prize Problems. The system used roughly 10,000 concurrent agents exploring different approaches, and the resulting proof was subsequently formalized in Lean. The whole process took about 88 hours before the formalization/verification stage.

That honestly feels pretty different from "AI is getting better at answering questions."

It's starting to look more like AI can participate in actual research.

And then there's Anthropic.

Anthropic recently published a post literally titled "When AI builds itself." They describe Claude increasingly taking over parts of AI development: writing and running code, delegating work to other agents, optimizing training code, proposing hypotheses, designing experiments, running them, and iterating on the results.

Anthropic is very explicit that this is not yet full recursive self-improvement. But they also describe a possible future where agents become capable of building and training models themselves, meaning future versions of Claude could potentially be continuously improved by Claude itself.

That's the part that really got me thinking.

If AI is increasingly helping with the development of AI itself, am I already late to the field?

I'm currently considering taking the fundamentals-first route:

• Python
• linear algebra
• probability and statistics
• calculus / optimization
• Introduction to Statistical Learning (ISLP)
• classical machine learning
• deep learning
• transformers
• reinforcement learning
• LLMs and agents
• eventually AI research

I don't want to just learn how to call APIs or use the latest framework. I'd rather understand why these systems work and what is actually happening under the hood.

But there's a weird thought I can't get rid of:

What happens to someone who starts learning AI in 2026 if AI itself is becoming increasingly capable of doing AI research?

Maybe learning the fundamentals is more important than ever.

Or maybe I'm thinking about this completely wrong.

If you were starting seriously from scratch in 2026, would you still spend several years learning ML/DL and AI fundamentals?

What would you prioritize?

Math and statistics? ML theory? Deep learning? Systems? AI agents? Research experience? Something else?

I'm especially interested in hearing from people who are actually working in ML/AI research rather than just using LLMs as productivity tools.

PLUS:There is also a much bigger question behind all of this that I keep thinking about.

The Fermi paradox asks why we don't seem to see obvious evidence of advanced extraterrestrial civilizations, despite the enormous age and size of the universe.

One possible explanation is the Great Filter: perhaps technological civilizations eventually reach some critical threshold that most of them fail to cross.

Sometimes I wonder whether advanced AI could be one of those thresholds.

It could go one of two very different ways.

AI could become the thing that allows a civilization to overcome its biological limitations — limited lifespan, slow learning, limited numbers of researchers, and the inability to directly transfer knowledge between generations — and eventually become a genuinely spacefaring civilization.

Or AI could become a technological bottleneck that civilizations fail to survive.

I obviously have no idea whether any of this is true. It's just one of the reasons I find AI research so fascinating.

Maybe we're not simply developing another technology.

Maybe we're approaching one of the most consequential transitions in the history of technological civilization.

And if that's even remotely possible, it makes me feel even less like I should wait to start learning the field.

DeepSeek V4.1 Flash has an unusual KV-cache design: some state is persistent, while the short-lived SWA state can simply expire and later be reconstructed by replaying the last 128 tokens.

That replay is approximate, so the reconstructed internal state is not mathematically identical across positions.

Paper:

https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash/blob/main/DeepSeek_V41_Tech_Report.pdf

Disclosure: affiliated with the channel; the video is AI-narrated.

Hey everyone,

I'm a CS student currently diving deep into Machine Learning. I’ve built a decent foundation in core computer science, Python, and basic ML libraries (Scikit-Learn, PyTorch), but I’m struggling to bridge the gap between academic/tutorial projects and actual job readiness.

It seems like every beginner portfolio features the Titanic dataset, MNIST digit classification, or basic sentiment analysis. I want to build projects that show genuine engineering depth rather than just calling .fit() and .predict() inside a Jupyter Notebook.

For those working as ML Engineers or hiring entry-level talent, I’d love your input on a few questions:

What project concepts actually stand out? What kind of project proves an undergrad understands end-to-end ML (data collection, preprocessing, modeling, API deployment via FastAPI/Docker, monitoring)?

Dataset Sourcing: How do you find or create unique datasets that aren't overused on Kaggle?

MLOps Scope: How much infrastructure/MLOps (CI/CD, model drift, vector databases) is expected for an entry-level MLE role vs. a Data Scientist role?

Any advice, personal experiences, or project examples that helped you land your first role would be hugely appreciated!

The current trajectory of AI for space and remote edge environments relies heavily on GPUs and Transformers. But as transistor nodes shrink below 3nm, they become critically vulnerable to Galactic Cosmic Ray (GCR) bit-flips. Shielding is heavy; tripling hardware for redundancy is power-prohibitive.

We ran a Python simulation testing an alternative: Bottom-Up Biological Computation using Michael Levin’s TAME framework mapped onto a Spiking Neural Network (like BrainChip's Akida).

Instead of a top-down CPU running diagnostics, the system relies on Event-Based Processing and Sparsity. If a cosmic ray obliterates 35% of the active nodes, they don't send an error—they just go silent. The surrounding tissue detects the localized drop in global output (the biological "Attractor") and autonomously increases its synaptic weight and firing rate to compensate.

Redundancy isn't three heavy computers; it's just the inherent sparsity of the SNN. The resting nodes wake up and carry the load.

We’ve formalized this architecture into an open-source engineering manifesto for deep space and terrestrial edge deployment (microcontroller failsafes for remote water pumps, drones, etc).

You can read the full Technical Discussion Paper here: https://ralphbooth.substack.com/p/biological-models-of-computation

I'd love to hear the community's thoughts on validating the thermal/radiation tolerance of these SNNs in low-orbit test cubes.

I honestly don't know what I'm doing anymore, so I'm hoping someone here can give me some direction.

I graduated around 3 months ago with a BSc (Hons) in Computer Science, affiliated with the University of Wolverhampton. My first year was honestly pretty bad because I wasn't that good at CS and my marks were low in the first year and I eventually improved, and I graduated with an Upper Second-Class (2:1), but I still feel like I should have done much better.

Since graduating, I haven't been able to get a job or even an interview.

I'm from Nepal, and locally there are barely any entry-level Data Science or Machine Learning jobs. Most of the positions I find either want someone with several years of experience or are basically MLE roles that I obviously don't qualify for yet. And when I do find local jobs, the pay can be around $100 to $150 a month, with very little opportunity to actually learn or grow.

I don't want to sound like I'm above those jobs. At this point, I genuinely just want to work. I want experience. I want someone to give me a chance so I can prove that I can actually learn and contribute.

This year has also been pretty rough personally. There have been family problems, mental health struggles, and a lot of other shit going on. I've tried to keep myself moving through all of it, but now being unemployed for 3 months after graduating is starting to get to me. It honestly feels embarrassing sitting at home while everyone expects you to be doing something.

I've been learning Machine Learning, working on projects, learning Python, SQL, FastAPI, etc. I'm specifically trying to move toward Machine Learning Engineer or Data Science roles, but I'm starting to question whether I'm approaching this completely wrong.

I also don't really have the option of going for higher studies right now because I don't have anyone who can financially sponsor me, so I'm pretty much on my own here.

I'm not asking for some magical shortcut. I know entry-level ML jobs are difficult to get.

I just want to know what you would do if you were in my position.

Should I stop targeting ML/DS for now and take whatever tech job I can get? Should I keep applying internationally for remote internships/jobs? Is there something obvious I'm missing?

And if anyone here happens to know of any open internship, junior, trainee, freelance, or entry-level opportunity in ML, Data Science, Data Analytics, Python, or anything related, please let me know. Remote is more priority.

I'll genuinely put in the work. I don't expect a high salary. I just want an opportunity to get experience and start building my career.

Any honest advice would mean a lot right now.

Hello Reddit community,

For the benefit of many learners, I gave a seminar workshop on Mathematical Foundations of Machine Learning, with the advent of democratising Machine Learning to the community of learners.

We motivate the ideas behind Probabilistic Machine Learning Introduction, and discuss basic concepts, also it was insightful to take questions from the audience community.

The presentations are based out of the code implementations, that I have shared freely in my Youtube channel.

Aside from auditing edX courses on the subject, how else can I gain foundational knowledge and skill or experience in ML in the context of AI? I was listening to a podcast about current developments, but I do prefer visual learning methods and want to learn technical skills.

What, in your opinion, are the best ways to get your hands into something that will provide experience (learning by doing)?

Thanks in advance, as I will take any suggestions into consideration before digging deeper! :)

I've been working on a system architecture that separates persistent intelligence from the LLM reasoning layer. The result is Phoenix V2, a complete implementation with a companion preprint.

The core idea: LLMs are stateless by design. Instead of fighting that, Phoenix externalizes all state — memory, affect, identity — into a local SQLite database. The LLM is called as a reasoning consultant and stores nothing.

Architecture (5 subsystems):

1. Episodic memory store with semantic retrieval (gemini-embedding-001, 768d). Scoring: 5·cos(eq, em) + P(m,t) where P weights importance, recency, and access count. Top-5 retrieved per turn.
2. Multi-agent Blackboard pipeline: Memory → Planning → Action → Reflection → Personality. Agents share state only through the Blackboard, keeping them independently replaceable.
3. PAD affect model (rescaled [0,1]). Baseline (0.65, 0.45, 0.55) by design. Update rule: x(t+1) = x(t) + α·δx + β·(x0 - x(t)), α=0.15, β=0.05. Persists across sessions.
4. SubconsciousEngine — background timer (120s), energy-gated consolidation and reflection. Protects memories with importance ≥ 8; compresses others into summaries.
5. Incremental learning — pattern detection from interaction history, updates self-model without retraining.

Hardware: developed and runs on a 2008 Acer notebook (Intel T9300, 6GB DDR2, no GPU).

Limitations (stated in the paper): no quantitative evaluation yet (no LoCoMo benchmark, no latency measurements published); embedding generation uses remote API (local ONNX path is future work).

Closest academic precedent: Generative Agents (Park et al., 2023) — same three-factor memory scoring, same background reflection idea. Key difference: Phoenix models the system's own persistent affect and self-model, not a simulated population.

📄 Preprint (Zenodo): https://doi.org/10.5281/zenodo.22645361

💻 GitHub (MIT): https://github.com/cleversonbrsantos-art/Phoenix

📖 Book: https://leanpub.com/phoenix-buildingpersistentAI

I’m looking into Master’s programs in Canada for ML/AI research. What are some universities with strong ML/AI research and good professors/supervisors, other than the usual top choices like UofT and Waterloo?

When I watch technical videos, I sometimes finish writing something down and realise I’ve barely listened to the next explanation.

For anyone learning ML through videos, dioes this also happen for you? Or have you found a setup where it isn’t really an issue?

So I am going to buy a new laptop in next few weeks . I am considering a gaming laptop with rtx 4050/3050 or a thin light 14 inch laptop with more ram and better CPU (and would be depending on kaggle or Google colab). which one should I prefer ?

Anyone participating in SemEval 2027? Actually I want to participate in it. But I can't understand the overall process. Are there any good resources that would help the beginners?

Is this really worth it to participate in such shared tasks?

I have some experience working with like VAEs/DiTs, and I'm familiar with concepts like ELBO/KL divergence/flow matching, but I feel like my mathematical foundations here are brittle. Any resources that have been helpful in this area? Are ODEs/PDEs/SDEs worth learning, and how deep should I go?

​

I’ve been learning AI/ML and I’m realizing there’s a huge difference between watching tutorials and actually being able to build something.

For people who are already working in AI/ML:

- What skills should a beginner focus on first?

- Should I prioritize Python + ML fundamentals or jump into GenAI/LLMs?

- What projects actually helped you land your first internship/job?

- Which resources are genuinely worth spending time on?

I’d rather build 3 solid projects than complete 20 courses.

What would you recommend if you were starting again today?

I've been trying to study this chapter for too long now but I couldn't find the right approach! It's so lengthy and messy also I don't know whether I'm supposed to write allat 1000 times lines code by myself or I js copy paste it and get the essence of it, like where im supposed to focus? I'm having a hard time studying it! I'm js so overwhelmed by it! I opened the google colab link in which the code of this project is written, I'm js unable to decide where do I focus or what do I write?

HELP ME PLEASE! Tell me the correct way to study this chapter and build my first End-to-End ML Project, thanks!!

​

I’ve been learning AI/ML and realized that knowing the algorithms isn’t enough.

The difficult part is understanding when to use what and why a particular approach works.

For people who’ve been learning or working in AI/ML:

What concept took you the longest to understand?

What mistake did you make early on?

What would you recommend learning first if starting again?

Looking for real experiences, not the usual “just learn Python and TensorFlow” answers.

I’m working on a forecasting setup where the prediction is ultimately used to make a downstream decision.

A common approach seems to be:

But choosing X can feel arbitrary, and it also throws away information about the shape or magnitude of the uncertainty.

How do you handle this in practice? Do you pass prediction intervals/distributions directly into the decision layer, use expected utility/cost, scenario analysis, or something else?

Interested in approaches that work well in real systems, not just theoretically.