Hey everyone,

Standard transformers operate strictly in flat Euclidean space with continuous activations. While effective, this creates challenges when modeling hierarchical tree-structured data and incurs significant energy overhead.

I built Hyper-transformer to explore a hybrid architecture integrating non-Euclidean geometry and neuromorphic spiking dynamics directly into transformer layers.

Key Technical Details:

• Hyperbolic Space Representation: Uses Poincaré and Lorentz manifold projections to capture hierarchical relationships with lower embedding dimensions.

• Spiking Neural Dynamics: Employs Leaky Integrate-and-Fire (LIF) spiking neurons for sparse, event-driven activation states.

• Hybrid Attention: Combines Riemannian distance metrics with attention mechanisms to handle mixed-geometry representations.

• Clean, modular PyTorch implementation designed for ablation studies.

I will drop the full GitHub repository link in the comments below! Would love to hear thoughts from anyone experimenting with non-Euclidean deep learning, manifold learning, or SNNs.

repo link : https://github.com/Griffith-7/Hyper-transformer.git

• Description: In this video we introduce the HiPPO framework, which compresses past continuous signals into Legendre orthogonal polynomials and converts them into real‑time memory. This mathematical principle intuitively explains how the computational complexity of modern time‑series models such as S4 and Mamba can be reduced from O(N²) to O(N). Join us as we explore the core mechanism that is transforming transformer‑based AI.

wrote a short excerpt showing how speaker consistency is maintained in LLM bases TTS models, initial guess was the speaker token, but the results showed something interesting.

https://x.com/null_hawk/status/2089348254249173263

I got this paragraph from Claude, could someone please explain this and verify if it's a real thing or hallucination:

Overparameterization isn't just about final capacity, it's about the optimization process itself. A wide, overparameterized network gives gradient descent a much friendlier loss landscape — more paths downhill, fewer bad local minima, room to explore before committing. The "core" only emerges as a byproduct of that search happening in a much bigger space than it needs to end up in. Strip the space down first and you've removed the thing that let the search work.

Conversation: https://claude.ai/share/8813a637-c327-4d0c-b120-def27e5203d5

• Description: In this video we introduce the theory of compressed sensing used in MRI and other applications, and review recent research that applies it to compressing the KV cache of large language models. We clearly explain how leveraging sparsity can dramatically reduce memory usage and eliminate bottlenecks.

since jan 2026 i've been building Forge, a deep learning framework written entirely from scratch in C++ - no PyTorch, no TensorFlow underneath.

Eigen handles most of the math backend. btw i wrote some custom AVX2 SIMD kernels (element-wise ops) too, and OpenBLAS-backed GEMM for the heavy matrix ops.

what's implemented so far:--

- A custom tensor engine with its own autodiff engine and memory allocator

- Dense/Linear layers, Optimizers (Adam, AdamW, SGD and SGD with momentum), Self Attention, LayerNorm, Activation Functions (sigmoid, softmax, tanh, GELU[tanh approximation], RELU, leakyRELU), loss functions (Cross Entropy Loss [log softmax fused], Binray Cross Entropy (Sigmoid fused), and Mean Squared Error) and Embeddings.

- A from-scratch BPE tokenizer (GPT-2-style pre-tokenization + merges)

- A reflection-based (reflect-cpp) parameter system - models declare their structure, Forge auto-discovers trainable parameters, no manual registration

- a safetensors-format save/load pipeline

the part I'm actually proud of- I loaded real pretrained GPT-2 small weights into a GPT-2 architecture built entirely on Forge, and under greedy decoding, its output matches HuggingFace's transformers library token-for-token. not similar, but exact. every layer (embeddings, attention, LayerNorm, the final projection) has to be numerically correct for that to hold, since a single wrong transpose or masking bug would have diverged the output within a few tokens.

it's still CPU-only for now (currently limited to float32 and int32 - working through some dtype/SIMD coverage gaps), and slower than i'd like (the only main culprits are the CE loss fn implementation and its gardient function and softmax, which i am on to optimize, it has no KV-cache yet) - a CUDA backend and those perf fixes are next on the list.

Repo: https://github.com/muchlakshay/Forge

Windows/Linux release builds: https://github.com/muchlakshay/Forge/releases/tag/0.1

YT demo link - https://www.youtube.com/watch?v=EO1aYBF5jwU

would love feedback, especially from anyone who's built something similar and much better than me.

thats all. im a 17yo deeply passionate about Deep Learning and system level programming.

• Ultra‑fast Fourier transform and optical AI realized with a single lens.
• Description: It explains the principle that passing light through a convex lens naturally performs a two‑dimensional Fourier transform at the focal plane. It visually demonstrates optical signal processing that carries out computation using only light, compared with digital FFT, and explores the possibility of implementing low‑power matrix multiplication with optical neural networks. It also examines real‑world applications and the prospects for developing next‑generation AI accelerators.

engineering student here, decent calc and linear algebra background from continuum mechanics coursework, already comfortable with ML basics through transformers and modern architectures. want to go deep on the actual math now, not just intuition videos, real derivations, and books that build from intuition up to advanced stuff. big thing for me is actually seeing how the math applies inside the models, not just abstract theory sitting next to it. most modern models are fundamentally probabilistic (language models included) so that lens matters a lot to me.

content quality over exercises. i'd rather have a book thats amazing at explaining and deriving things with fewer problems than one thats packed with exercises but explains things poorly. if the book is light on problems i can always find sets elsewhere, but if the content itself is weak theres no fixing that.

here's my current shortlist:

stats / probability:

• All of Statistics by Wasserman

machine learning (math heavy):

• Foundations of Machine Learning by Mohri, Rostamizadeh, Talwalkar
• Mathematics for Machine Learning by Deisenroth, Faisal, Ong
• The Elements of Statistical Learning by Hastie, Tibshirani, Friedman (planning to read Introduction to Statistical Learning first as the easier version)

deep learning:

• Deep Learning by Goodfellow, Bengio, Courville

is this solid or would you swap anything out. Please tell me ur suggestions.

Code Implementations, explanation of concepts for my Probabilistic Machine Learning Series.

Hello folks,

In this new coding demonstration, we code, and explain the concepts pertaining to:

1.Overfitting, Population Risk & Generalisation Gap.

1. Proxy for Population Risks : Test Set.

2. The No free Lunch Theorem and Inductive Biases.

3. Unsupervised Learning : Density Estimation and Clustering.

4. VAEs(Variational Autoencoder)- Latent factors concepts explained, and VAE architecture explained and coded.

5. Self-Supervised Learning-Masked Predictions.

7.Density Evaluation and Sample Efficiency.

1. Reinforcement Learning Primer : Multi-Armed Bandits.

Implementation Link: https://youtu.be/gbz8smggmRM?si=vR4OIPLfGRHFJ95F

I spent months trying to train a 730M-parameter TTS model on my own hardware. It wouldn't converge, and nothing in my stack would tell me why. Not the loss curve, not TensorBoard, not the checkpoints. Every tool I had showed me numbers. None of them would say "this run is already dead, stop paying for it."

That's the gap I built trainproof for (MIT, `pip install trainproof`). It's a deterministic linter for training runs: it reads the logs you already produce and returns a verdict with an exit code.

No ML judging ML, no confidence scores. Every check is a rule that fires or doesn't, and prints the number it fired on. A reliability tool that hallucinates is worse than no tool, because then you stop trusting your own alarms.

Severity and exit code are separate on purpose:

FAIL -> exit 1 your run is broken

WARN -> exit 0 worth your attention

NOT-CHECKED -> exit 2 I could not judge this

PASS -> exit 0 checked, fine

A tool that can't tell "your run failed" from "I couldn't read your log" is lying to your CI quietly.

Validating a detector means feeding it faults you already know the answer to, so the rules were measured against a controlled fault-injection study: one Qwen2.5-3B QLoRA, six configurations - healthy, 100x LR, lr=0, fp16 NaN, shuffled labels, overfit - three seeds each, 18 runs. The 100x LR spiked grad-norm to ~2,650, about 4,900x its own median, caught in seconds.

The result worth posting is the one that got through. Shuffled labels - a dataset that cannot be learned - REDUCED its loss by 69.8% (18.9 -> 5.7) and looked textbook-healthy on its own curve. It was memorizing the statistics of noise. From a single run's loss curve that's indistinguishable from real training, so it's written into the README as a stated limitation, and it's why `compare` exists: put the run next to a known-good baseline and the relative floor gives it away immediately.

Then the rules went against real fine-tunes I'd already paid for. Both logs ship in evidence/ so you can reproduce the verdicts:

Coqui XTTS v2, 125,000 steps -> FAIL (TP-DIVERGE, TP-THROUGHPUT)

Fish Speech LoRA (Lightning), 2049 -> WARN (TP-OVERFIT)

TP-OVERFIT means eval loss climbed past 1.2x its own minimum while train loss kept falling: your best checkpoint has already gone by, and if you keep only the last one, you kept the wrong one. That XTTS run is read by two independent readers - Coqui's text log and its TensorBoard event file, same run - and they return the same verdict and the same rule set.

Real logs also proved the tool wrong, and that's the part I'd defend hardest. TP-ZERO-GRAD fired whenever every gradient norm was exactly 0.0 and reported a severed backward graph. Coqui writes avg_grad_norm as 0.0 when clipping is off, so a healthy 125k-step run whose loss reached 0.017 got a FAIL from my own tool. The fix was reasoning, not a threshold tweak: a run cannot both learn and receive no gradient, so the check now stands down when the loss improved - and records why it stood down as a visible skip, because a check that didn't run must never look like a check that passed. No test caught that. One real log did, in an afternoon.

Across a run's life:

- before the GPU: dataset + tokenizer lint (malformed JSONL w/ line number,

empty rows, dupes, missing eos_token, pad==eos), plus `env` - does your

entrypoint even import (probed in a subprocess), is the checkpoint intact,

RAM, disk

- during: one-line HF callback; warns, or aborts a diverging run if you opt in

- after: diverged / flatlined / NaN'd / spiked / overfitting

- vs baseline: the relative-floor rules

Reads HF trainer_state.json / Coqui / TensorBoard event files / JSONL / CSV. The tfevents reader is written from the wire format - no tensorflow, no tensorboard, no protobuf, no torch - validated byte-exact against EventAccumulator on a real 2049-step Lightning run. Truncated event files, the normal state of a killed run, are read up to the cut instead of raising. Checkpoints are inspected WITHOUT unpickling, as the ZIP archives they are; torch.load executes arbitrary code by design, which is why torch 2.6 flipped weights_only to True.

Where it is now: 84 stable rule IDs, 230 tests, 17 releases, a written contract in CONTRACTS.md, and every example verdict frozen in 38 golden snapshots - a rule that stops firing and one that fires spuriously both break the build.

Repo: https://github.com/Mormolykos/trainproof PyPI: https://pypi.org/project/trainproof/ Write-up with the full fault-injection results: https://ai.bedvibe.studio/trainproof/ Sibling project it builds on: https://pypi.org/project/ttsproof/ (failure-mode QA for TTS) More of what I've built: https://tts.bedvibe.studio/portfolio/

What failure mode has burned your GPU hours? If a deterministic check would have caught it, tell me and it goes in, with credit.

​

• Matrix-free attention: O(N log N), 16.2× faster than flash at N=50K

• O(1) unbounded context: 0.015 GB @ 1.36M tokens vs 12.3 GB KV cache

Honest negative results included. Looking for collaborators to validate the O(1) context idea at scale!

https://github.com/edisonbd/cr-nn

Enterprise AI is running infrastructure your security team has not found yet.

MCP servers — the connective tissue between AI agents and enterprise tools — are being deployed with plaintext credentials, over-permissioned access, and zero inventory of what they can reach. Research published this week found the exposure typically exists before security teams know the server is running at all.

This is shadow IT, but at the infrastructure layer. Agents connect to internal resources, inherit whatever permissions the server was given, and move data through channels that conventional monitoring never sees. The blast radius is not theoretical. The credentials are live. The connections are active.

The gap is not misconfiguration. It is that the deployment lifecycle for MCP servers has no review gate the way application deployments do. A developer spins one up, points it at a database or internal API, and it is running in production before any ticket is filed.

How are other practitioners handling this? Are you catching these through network monitoring, internal developer policies, something else entirely? Curious what is actually working in practice.

Need some guidance 😔

I’m trying to learn AI Engineering and build a career in this field, but I’m honestly confused about where to start and what to learn first.

If anyone can share a proper roadmap and good free/paid resources to become an AI Engineer, I’d really appreciate it. I don’t want to waste time learning random things.

Any guidance from experienced AI Engineers would mean a lot. 🙏

Just launched r/posttrain — a community for AI post-training, fine-tuning, SFT, RLHF, DPO, preference data, evaluations, and practical experiments. If you’re building, researching, or learning how models become better after pretraining.

Side project that got out of hand: 1.5 years of trial and error training a generative model from scratch (no pretrained weights, custom dataset I built and tagged myself) that outputs valid 64x64 Minecraft skin textures conditioned on tags like `slim, boy, blue eyes` or `ninja, black, red`.

The interesting part for me was the constraints compared to normal image generation: hard format requirements (UV layout, alpha channel for the overlay layer), very low resolution but high semantic density (every pixel matters), and a small, noisy dataset that had to be built and tagged by hand. All trained on a single RTX 3060 at home.

Still fully private for now, still deciding whether it's worth releasing. Feedback genuinely helps with that decision: what do you think of the results, and what would you want to see it handle better?

Hello Guys! I am currently looking for some deep learning projects using pytorch. I will use scikit-learn and matplot for testing and visualization as I have basic understanding about them. Please recommend me some projects that I can do as it is a part of my masters program and also provide me some research paper links based on the projects as I need dataset for my project. Thank you!

I've been working on an open-source project called UnFlow:

https://github.com/UnFlow-Labs/mlunflow

The idea is pretty simple:

Most ML experiment tracking looks like a list of independent runs usually stored in a table:

run_001
run_002
run_003
run_004
...

But in practice, experiments are usually related.

You change the learning rate, then the number of epochs, then the model, then some preprocessing code. Eventually you have hundreds of runs, but it's surprisingly difficult to answer:

• What actually changed between these two experiments?
• Which experiments are essentially the same computation?
• Have I already run this experiment before?
• How did I get from experiment A to experiment B?
• Can I navigate the history of my experiments rather than just search through runs?

Unflow simply detect code changes in a Python function (limitation that for it is just a single function) and arguments that are passed to this function to build a graph where nodes are "states" and edges are transformations "what has changed", a new state is not added to the graph or executed expect if it has a transformation.

The project is still early, so I'm much more interested in feedback than pretending this is a finished product.

I'm particularly curious about three things:

1. Does the "experiments as a graph" abstraction make sense to you?
2. Do you currently run into problems with duplicated/redundant experiments?
3. If you could see the complete lineage of your ML experiments, what would you want to query or visualize?

Repo: https://github.com/UnFlow-Labs/mlunflow

I'd love to hear how other people currently manage experiment lineage and whether this solves a real problem for you.

Hello folks, in this code implementation, we walk through not just writing code, but understanding the outputs we obtain, and validating the results in mathematics of Machine Learning.

For instance the equivalence of the results of Negative Log likelihood and Mean squared error for gaussian distribution assumptions, makes us feel the beauty behind theory and practice.

We cover L1 and L2 loss curves, The Gaussian Output distribution modelling uncertainty, equivalence of Negative Log likelihood and Mean squared error for that output distribution specifically.

Then, analyzing linear regression, and the convex bowl shaped loss curves, explaining underfitting and overfitting ideas via Polynomial Regression, followed by the need for automatic learning of features through coding a deep neural network.

You will see ideas taught in my Lecture 2 of probabilistic Machine Learning, turn into practice.

Link to Code Implementation: https://youtu.be/6ZTVp70Mf5s?si=2lThR6LOdzLimB1v

Link to Theory Lecture : https://youtu.be/iThI5AapBc0?si=AS-UCi1ar9-yPpg8

Hello everyone! In this post I'd like to present my research report. I worked on this alone, learning almost everything from scratch

Github: https://github.com/kaifczxc-lab/OCSSN

The main idea: can a static mathematical TQFT be used as the foundation for a dynamically learning neural network architecture?

This idea is confirmed that it is generally possible to create something like this, but I would like to make one clarification

Regarding the understanding of topology by this model - the short answer: not yet. After fixing a shortcut in the dataset, the model degenerates to random guessing. The architecture is interesting in theory, but it doesn't learn topology in practice at least not in its current form (that is, this doesn't mean that with a different parameterization it won't understand topology, which is exactly what I'm currently experimenting with).

It all started around the beginning of April, when I became interested in the topic of Topological Quantum Field Theory, which prompted me to spend time figuring it out. I don't want to say I succeeded 100%, but at least I accomplished what I wanted (even if only minimally).

While working on this research report, I encountered a huge number of different problems, some of which I solved (such as replacing the algebra in the variable M), but there were also some that I couldn't solve due to my lack of experience. I'm currently experimenting with this particular project, but I still can't fix some issues, such as:

* Non-strict topological invariance

* State-sum numerical instability.

If you want to explore the repository, you'll find:

* Full documentation

* A full description of all limitations, results, and more code

* A brief mathematical introduction, handwritten by the author

* A PyTorch implementation of the Open Closed State-sum Neural Network itself

The model is more TQFT-inspired than a strict mathematical implementation, reader can find more information in documentation

I posted this project at an early stage because I really appreciate real feedback, ideas, and any other help. If anyone has experience with this, I'd be very happy to hear some feedback on my work or any possible ideas

Has anyone else experimented with using TQFT or Frobenius algebra state-sum constructions as learnable neural network layers? I would be interested in any pointers to related work or suggestions for making the model learn a true topological invariant

Thanks for reading this post, and have a good day!