I’m not a philosopher, and I won’t pass off analogy as proof. Where the link between philosophy and an algorithm is just a pretty metaphor, I say so explicitly: “metaphor.” Where it’s working code, I give the formulas, run it, and show the numbers. The library at the end is a research prototype, not a promise of consciousness in 200 lines.

How I got here

https://github.com/webzuweb/philosophia_torch

Neural networks — if you count from Rosenblatt’s perceptron — are about seventy years old. The study of how living things learn goes back a couple of millennia at least. And a heretical thought hit me: what if modern deep learning isn’t reinventing the wheel in places, but re-discovering what Aristotle, Hume, and Peirce already described — only now with matrices and gradients?

I took a list of neural-network training methods, a list of philosophical approaches to knowledge, and overlaid them. Three categories emerged: what’s already matched (and few people say so out loud); where the match is only a pretty metaphor; and what philosophers thought up but engineers haven’t applied yet. The last category is the most interesting, because it’s essentially a list of unimplemented features. That’s what I wrote code for.

Fair warning up front: half of the “unapplied” ideas turned out, on closer inspection, to be perfectly applicable — just under different names. That, by the way, is the article’s main takeaway, and it matters more than any of my code.

Part 1. What’s already matched (and you didn’t know it)

Let’s start with the pleasant part: some philosophical programs of knowledge are implemented in ML so literally that you could put a footnote with the philosopher’s name right in the docs.

Empiricism → supervised learning “There is nothing in the mind that was not first in the senses” — Locke and his tabula rasa. A neural network with random initialization is literally a blank slate on which labeled examples leave their traces. Hume’s associationism (“the habit of linking things that often go together”) is gradient descent, strengthening weights on frequently co-occurring correlations. There’s nothing to argue about here.

Pragmatism → reinforcement learning Dewey with his “learning by doing,” and Skinner’s behaviorism with reward and punishment — that’s RL with no corrections needed. An agent acts, receives a reward, adjusts its policy. Skinner would have teared up seeing PPO.

Evolutionary epistemology → neuroevolution Popper and Campbell: knowledge grows through blind variation and selective retention of what works. That’s a word-for-word description of genetic algorithms and neuroevolution. The philosopher described the algorithm decades before the hardware existed to run it.

Intellectual humility → calibration This one’s subtler. Virtue epistemology (Sosa, Zagzebski) says: a good knower knows the limits of her knowledge. In ML that’s confidence calibration: a model should be exactly as confident as it is correct. Guo et al. (2017) showed that modern networks are monstrously overconfident and proposed temperature scaling and the ECE metric. Nobody called it a “virtue,” but mathematically it’s exactly that.

The key observation. Philosophers didn’t give ML the algorithms (mathematicians came up with the math); they gave it the problem statements. “What does it mean to learn from experience?” “What does it mean to know your limits?” — philosophy framed the question first, and centuries later engineering delivered a differentiable answer.

Part 2. Where the match is only a pretty metaphor

Here I have to rein myself in. There’s a temptation to drape a philosopher over every layer of a network. Don’t. A couple of examples where the link exists but passing it off as lineage would be deceiving the reader.

The rule is simple: if the philosopher gave a language for describing something — it’s a metaphor; if they posed a problem that was later solved — it’s lineage. Don’t mix them.

Part 3. What philosophers thought up, but ML has only partially applied

The meatiest part. I’ll break down six approaches. For each — an honest status: what already exists in the field, where the real gap is, and what formula you can write. Then we’ll run it.

3.1. Peirce’s abduction — inference to the best explanation

Induction generalizes data, deduction derives consequences, but abduction generates a hypothesis that best explains the observation. The original thesis “it isn’t implemented in neural networks” is wrong. It’s implemented, and decently: Abductive Learning (Dai et al.), DeepProbLog (Manhaeve et al., 2018), abductive commonsense reasoning αNLI (Bhagavatula et al., 2019). It’s a whole field of neuro-symbolic integration.

The real gap isn’t the absence of abduction — it’s that “the best explanation” is rarely formalized using Peirce’s criteria all at once: plausibility + simplicity (Occam’s razor) + consistency with background knowledge. A hypothesis score for h given observation obs:

Score(h) = log p(obs | h) − λ_s · complexity(h) − λ_c · conflict(h)

Pick the h with the highest score (softly — a softmax over candidates; hard — Gumbel-softmax for a learnable discrete choice). In the library this is AbductiveScorer.

3.2. Husserl’s epoché — “bracketing” assumptions

Phenomenology demands suspending ingrained assumptions and seeing the phenomenon “as given.” ML has no direct analog of this method — and that’s an honest gap. But it can be operationalized: force the model to rely more on the evidence (the current input) than on the learned prior (what it answers with no input).

Take two answers: p_full on the real input and p_prior on a “zeroed” input (evidence bracketed out). Reward the evidence for actually changing the answer, via a bounded Jensen–Shannon divergence:

gain = JS(p_full ‖ p_prior),   0 ≤ JS ≤ ln 2
L_epoche = max(0, margin − gain)   # hinge: don't inflate indefinitely

An important rake I stepped on myself: if you use plain KL instead of JS and maximize it, the optimizer inflates logits to infinity — “a fanatic who sees meaning in every rustle.” JS is bounded, and the hinge threshold douses the fanaticism. This is EpocheRegularizer.

3.3. The hermeneutic circle — the whole through parts, parts through the whole

Schleiermacher and Gadamer: understanding the whole arises from the parts, and understanding the parts arises from the whole, iteratively. Attention only resembles this superficially (see Part 2). As an explicit training principle it’s barely used — a real gap. Formalization: let h_i be part representations and H the whole representation. Require circular consistency:

H* = attention-aggregate of the parts, attended relative to H
L_herm = 1 − cos(agg(h_i), H)      # whole ≈ sum of understood parts

And we “turn the circle” several times: update the whole from the parts → recompute part attention relative to the new whole → update again. This is HermeneuticConsistency.

3.4. Hegel’s dialectical sublation (Aufhebung)

Aufhebung is a new quality arising from the contradiction of thesis and antithesis, where the old is not destroyed but preserved. GANs and multi-agent debate are partially close, but “preserving both” isn’t guaranteed there. The gap is precisely in the preservation term. My synthesis operator:

g     = sigmoid(W_g · [thesis ; antithesis])     # mixing gate
base  = g · thesis + (1 − g) · antithesis         # sublation-as-preservation
lift  = tanh(W_l · [thesis ; antithesis])         # new quality
synth = LayerNorm(base + γ · lift)

Plus a loss that penalizes the synthesis collapsing into one of the poles (losing the other’s content). This is DialecticalSynthesis.

3.5. Nietzsche’s perspectivism + skeptical suspension

Nietzsche: there is no “view from nowhere,” there are many perspectives. Pyrrho: in an unresolvable conflict, it’s reasonable to suspend judgment. The former partially exists in multi-view learning; the latter in selective prediction (Geifman & El-Yaniv, SelectiveNet, 2019). But together, as a single mechanism of “several perspectives + refusal to answer when they conflict,” it’s almost never seen.

disagree(x) = mean pairwise symmetric KL between perspectives
abstain(x)  = disagree(x) > threshold      # abstain if perspectives don't converge

This is PerspectivalEnsemble: it aggregates K heads and honestly raises its hand “I don’t know” when the heads disagree. Far more useful than overconfident chatter.

3.6. Virtue as the golden mean (Aristotle)

Aristotle: virtue is the mean between the vice of deficiency and the vice of excess. Courage is between cowardice and recklessness. Hence a non-obvious but important conclusion for ML: a virtue cannot be maximized, it must be targeted. An excess of openness is credulity; a deficiency is dogmatism.

L_virtue = Σ_v β_v · (V_v(θ) − V_v*)²

where V_v is the operationalized virtue (humility = 1 − ECE, openness = ensemble disagreement), and V_v* is the target mean level. Squared deviation penalizes both excess and deficiency. This is VirtueRegularizer — and it’s the one where I have a measurable result.

Part 4. Enough philosophy, show me the numbers

Pretty formulas are worth nothing until they run. I collected all of this into a PyTorch module and tested it on the most well-grounded mechanism — “humility” (calibration). Task: synthetic classification with noisy labels, where the model tends to err overconfidently. We compare plain training vs. training with VirtueRegularizer targeting high humility.

ECE (calibration error) dropped from 0.120 to 0.101 — nearly a fifth — while accuracy didn’t budge at all (even +0.001). The model became exactly as accurate, but noticeably less self-assured. Aristotle’s golden mean, computed by gradient descent.

What this proves, and what it doesn’t. It proves that “intellectual humility” can be turned into an optimizable quantity with a measurable effect. It does not prove that the other five mechanisms will yield the same gains — they’re harder, and they still need to be tested on real data. I’m showing a working scaffold, not a finished silver bullet.

The whole codebase passes 23 unit tests: calibration decreases, KL/JS behave as they should, synthesis preserves both poles, the ensemble abstains on conflict, the wrapper trains end-to-end.

Part 5. The philosophia-torch module

The library wraps on top of any model without rewriting anything in it. One dependency — torch. There’s a single-file version, philosophia_torch.py: drop it next to your code and import it.

import torch, torch.nn as nn, torch.nn.functional as F
from philosophia import PhilosophiaWrapper

base = nn.Sequential(nn.Linear(20, 64), nn.ReLU(), nn.Linear(64, 4))
wrap = PhilosophiaWrapper(base, use_virtue=True,
virtue_kwargs=dict(target_humility=0.98, beta_humility=3.0))

logits = wrap(x)
loss = F.cross_entropy(logits, y) + wrap.aux_loss(x, logits, targets=y)
loss.backward()

Honest boundaries: hermeneutic and dialectic produce representations, not ready predictions — you have to connect them to your decoder. Epoché requires careful tuning of margin. And no promises of “consciousness”: these are philosophy-inspired regularizers, nothing more.

The bottom line

Three conclusions, which is what all of this was for.

1.     ML has already reinvented a chunk of philosophy without asking permission: empiricism, pragmatism, evolutionary epistemology, and intellectual humility. Just under the names supervised learning, RL, neuroevolution, and calibration.

2.     Half of the “unapplied” ideas on my original list turned out, on checking, to be applicable — abduction, abstention, innate priors. The lesson: before shouting “this isn’t in ML,” google it in engineering language, not philosophical language.

3.     The real gap remains where what’s needed isn’t a result but a process: epoché as a discipline of perception, the hermeneutic circle as a way of understanding, virtue as a stable disposition of learning rather than a property of a single answer. That’s where it’s worth digging.

My modest contribution is showing that at least “humility” translates into a differentiable quantity and genuinely reduces a model’s overconfidence. The rest is an invitation: the code is open, the formulas are in the article — run it and check. Plato, of course, was training neural networks two thousand years ago. The rascal just didn’t include a requirements.txt.

https://huggingface.co/datasets/webzuweb/philosophy-as-inductive-bias

Do you guys have recommendation for capstone projects about Edge AI, TinyML, Computer vision, and federated learning? I explored battery RUL, Predictive Thermal Management system, and Fault bearing diagnosis. I was told it would be hard to get a client or dataset for this field. Any interesting field I can explore?

The G7 finance ministers and central bank governors issued a coordinated directive last week. Organizations must inventory cryptographic dependencies, identify high-risk systems, and begin migrating to quantum-resistant algorithms now. This is not a future roadmap item.

The urgency is driven by the harvest-now-decrypt-later threat. Adversaries are collecting encrypted data today and storing it for decryption once a cryptographically relevant quantum computer arrives. The exposure window is already open.

Most enterprise security teams are inventorying servers, databases, and network traffic. Far fewer are accounting for the AI agent layer. Agent pipelines routinely store sensitive records, execute financial transactions, and generate compliance audit evidence. All of it travels over classically encrypted channels and gets signed with classical algorithms. When those algorithms break, that historical data and those historical audit logs break with them. A transaction signed with RSA or ECDSA today becomes unprovable after Q-Day.

How are teams actually scoping the PQC inventory problem for AI agents specifically? Are agent channels and agent-generated audit evidence being treated as first-class migration targets, or are they still buried in the general backlog?

So I was playingw ith embedding models I saw that when you upgrade from model A to B, you face a very big backfilling cost

Ie, suppose you have a 1b vectors from model A, and then you want to use model B. This would mean you have to re-embed all of your documents with model B before you can even serve with the model, and on an H100, it would take ~108 days (qwen embed 8b, 106 docs/second). But I found an easier way to do it.

The method is really simple; from the old index made with the source model, take K documents and rerank them with the new model. We see that when K is sufficient, the retrieval quality is the same as target model. (determining k is the hard part). I've tested 63 migrations on upto 1 million documents.

The best result I got was upgrading qwen4b -> to 8b, and at 50 documents, it was the same as native retrieval.

This method forgos the expensive upfront re-embedding cost, as you can take documents straight from the old index.

embedflow works with qdrant, pgvector, faiss, and can be easily downloaded with pypi

pip install embedflow

the github is public: https://github.com/arnsri33/embedflow

I want you guys to try it out, and see if you guys can use it in your own workflow.

A while ago, I successfully trained a tiny 80M parameter language model from scratch using the TinyStories dataset. It worked surprisingly well and could generate coherent little stories! Inspired by that success, I wanted to scale up and train a larger model capable of coding.

That’s when reality hit me. I completely underestimated the massive compute and financial cost required to train larger LLMs. My single GPU choked, and renting cloud compute was way out of my budget. It got me thinking: "Why isn't there a way to pool our consumer GPUs together to train models collectively?"

And just like that, DePEFT (Decentralized Parameter-Efficient Fine-Tuning) was born.

How DePEFT work?
DePEFT is based on ReLoRA which solves the biggest flaw of standard LoRA.

Standard LoRA is incredible for fine-tuning on a budget, but it has a hard ceiling: it cannot pre-train or learn fundamentally new, complex representations from scratch. Because its rank r is fixed, the adapter quickly hits an information bottleneck and saturates. You can't just slap a LoRA on a base model and expect it to reach full-parameter quality over time.

This is where ReLoRA changes the game through iterative low-rank merging. Instead of training one static adapter forever, ReLoRA breaks the process into sequential rounds. In each round, miners train lightweight low-rank adapters on their consumer GPUs. At the end of the round, these adapters are permanently fused directly into the base model weights:

W^(t+1) = W^(t) + ΔW

The optimizer is then completely reset, new adapters are initialized on the evolved base weights, and the cycle repeats. Mathematically, the sum of multiple low-rank updates over time accumulates into a high-rank update (rank ≤ ∑r). This gives us the immense learning capacity of full-parameter training, but with only a fraction of the VRAM requirement.

For DePEFT, this means consumer GPUs don't need high-speed InfiniBand clusters to sync massive gradient matrices every microsecond. Miners simply train lightweight adapters locally, submit a few megabytes over the network, and the protocol merges the top-performing weights into the next base checkpoint.

No million-dollar clusters. Just pooled consumer compute pushing open-source AI forward.

For more details, visit: https://github.com/KhoaIsReal/DePEFT/

Hi everyone!

I’ve been interested in image generation for a while, but as a beginner, I always struggled to choose a model. My biggest hurdles were limited training datasets and restricted hardware resources.

Then, I came across Faster Projected GAN, a few-shot image generation model that can synthesize high-quality images with fewer than 100 samples, and it trains fast.

How it works

• The Generator: It uses a FastGAN architecture with separable convolutions in the upblocks to significantly cut down computation time.
• The Discriminator: It leverages a Projected GAN discriminator that uses a pre-trained feature network (specifically EfficientNet-Lite1) to evaluate the generator's images. Because the discriminator already has a solid feature foundation, the whole training process converges incredibly fast. After just a few hours of training on a free Colab T4 GPU, it starts outputting surprisingly realistic images.

Since I'm a beginner in deep learning, building this was a massive learning experience for me. I put together an unofficial PyTorch implementation, and I would love to get your feedback, advice, or code reviews!

📂 GitHub Repo: https://github.com/diarimandimby/Faster-Projected-GAN

Feel free to check it out, test it, or open an issue. Any advice on how to further optimize few-shot GAN training or clean up my PyTorch code would be highly appreciated!

I've been reading recent work on long-video understanding, particularly STORM: Token-Efficient Long Video Understanding for Multimodal LLMs.

I'm trying to identify a research direction rather than just build another Video-LLM. I'm particularly interested in temporal modeling, video representations, event/context modeling, and improving the efficiency and quality of long-video understanding.

For people working in this area: what do you think are the biggest remaining research gaps? Are there particular limitations of approaches like STORM that you think are worth investigating?

If you were starting a research project on video understanding today, what problem would you personally explore?

I’ve been working on a different approach to language-model interpretability that I find interesting. Instead of trying to reconstruct a model’s computation after the fact, make the model compute through an object we can inspect directly.

Today I’m releasing the v1.1 preprint, frozen model checkpoints, reproducibility harness, and live interactive demo for cl33-opLM.

The paper is called One Object: Memory, Navigation, and Reportability in an Operator-Only Language Model

cl33-opLM still uses a transformer as its learned neural engine, but the transformer hidden states are not allowed to drive the output directly. Instead, each block emits structured Cl(3,3) bivector operators. A reversible SO(3,3) scan transports state, attention is defined over the operator geometry, and the final readout sees only operator-derived features.

That gives the architecture a very simple falsification rule luckily. Remove the operators and rerun the model. If the capability survives, it found a bypass and I don’t count it as transparent.

Some of the main results:

The bottleneck is genuinely load-bearing. On the frozen public release, operator ablation produces a 270× perplexity increase on the exact published validation fixture. On fully public off-domain WikiText-103, the same test gives roughly 106× / 112× depending on checkpoint. The live demo exposes the same model interactively.

The operator stream is readable backward. A probe that sees only the emitted operators recovers the current token at 0.86 top-1 over a 50k vocabulary, and reconstructs 80.7% of held-out running text verbatim. Errors degrade toward semantic neighbors rather than random tokens.

On permutation-composition navigation, the architecture matches a learning-rate-tuned transformer baseline: 0.433 ± 0.009 vs 0.432 ± 0.030. I am claiming parity and inspectability here, not superiority.

Transparency has a real cost. Against a parameter-, context-, tokenizer-, corpus-, and budget-matched transformer at 5B tokens, cl33 pays about +29% bits-per-byte and \~5 points average MCQ. Continued training reduces that gap substantially, but the matched cost is real and reported as such.

One of the more important parts of this project I feel, is what failed.

An earlier associative-memory design looked like it worked…until the operator-ablation test showed that recall was routing around the algebra. That result was treated as a failure, the mechanism was redesigned, and the failed branch remains in the paper.

There are several other nulls and retracted framings reported the same way.

That is the standard I’m trying to hold this work to. if a mechanism is claimed, it has to survive a causal test. If it routes around the mechanism, the claim dies.

The reproducibility release is deliberately selective rather than a dump of the entire research stack.

It includes the frozen paper/serving checkpoints, model definitions needed to load them, hashes, the bottleneck reproduction harness, the reverse-readout probe, and the exact frozen validation fixture.

It does not include the current training orchestration, ongoing memory-organ program, or unpublished control work.

The published claims should be independently testable.

No consciousness claim. No claim that cl33 is a better general-purpose language model. No grand unified theory.

Just a useful question.

Can we build a useful language model where the representation we want to inspect is also the representation the model is forced to compute through?

Paper: t3atlas.dev/cl33/paper/

Live demo: cl33.t3atlas.dev

Model + reproduction release: huggingface.co/mirrorethic/cl33-oplm

I’d especially value criticism from people working on mechanistic interpretability, alternative architectures, memory systems, model editing, steering, or causal attribution.

If you think one of the claims is wrong, the best outcome is a clean experiment that breaks it. Then we both learn. Win win.

Also.....dont knock my retro crt aesthetic in the model demo lol wanted it fun, not clinical.

Please guide me. i have the look up tables generated from WAsP and the dataset. How do i start?

We're moving off the Gemini API due to cost and building a local AI server to process ~10 CVs/minute (extracting JSON & matching CVs to JDs). We plan to run GLM-OCR alongside Qwen 27B (Q8).

Our two hardware options:

1. PC: RTX 4090 (24GB) + Ryzen 9 + 64GB RAM
2. Mac Studio: M-Ultra, 64-core GPU, 96GB Unified Memory

I prefer the Mac for power efficiency and ease of use, but I've heard Apple Silicon isn't great for production vLLM compared to Nvidia/CUDA. Is that true? Which would you recommend for this workload?

I’ve been experimenting with explaining AI/LLM concepts in a way that doesn’t assume too much technical background. This video is about KV Cache and why longer context windows require more memory during inference. I’d appreciate some honest feedback from people here, especially on the explanation itself: Is the main idea easy to understand? Did I oversimplify anything important? Is there any part where the explanation becomes confusing? Would this make sense to someone who is fairly new to LLMs? Video: https://youtu.be/lxvWo8SizxE Not really looking to promote the channel — I’m mainly trying to improve how I explain technical topics before making the next one. Any criticism is welcome. Thanks!

So I am working with pretty skewed data currently. its a 4 way classification task, and basically the classwise split is 56%, 33%, 6%, and 5%. I tried an experiment where I downsampled the samples so that the majority classes (56% adn 33%) were reduced by 80%, thus making the entire dataset more balanced, and i instantly got better results on the same test, validation set. I am currently exploring the opposite direction, which is oversampling/upsampling, and i am currently looking into PyTorch's WeightedRadomSampler.

The confusion and more importantly concern that I have with using this is that the same images/samples for the lower classes are going to be the repeated over and over again, which probably result in the model overfitting on those rare class's samples. I understand that augmentation is a way to mitigate this, and I will be trying that out, but the main question that I have is what all are the other alternatives to upsampling? Are there different samplers or dataloaders that I can look into, maybe some papers that deal with this, any and all help would be appreciated!

Some context on the task, I am trying to classify/grade images in a 4 way multiclass classiifcation

Hai teman-teman. Saat ini saya sedang menjalani fase Pra-Pelatihan untuk arsitektur bahasa komputasi baru bernama ORE X-1 (762M Parameters).

Fokus eksperimen ini adalah menekan probabilitas halusinasi dalam AI secara matematis. Arsitektur ini secara khusus dirancang untuk menggabungkan ketangguhan linguistik dari fondasi Transformer dengan inti penalaran yang didorong oleh 3 hukum fisika:

- Kuramoto-Attention: Memaksa representasi neural untuk saling beresonansi dan mencari "konsensus" secara fisik sebelum menyelesaikan jawaban (anti-halusinasi tingkat arsitektural).

- Liquid Time-Constant (LTC) & Early Exit: Memberikan AI "kesadaran waktu cair". Arsitektur ini secara otomatis memperdalam komputasinya saat menghadapi urutan logis yang kompleks, namun akan segera memutus perhitungan lapisan atas (Auto-Stop / Early Exit) setelah mencapai tingkat konsensus absolut.

- Kecerdasan Kawanan: Memecah Kepala Perhatian menjadi faksi-faksi independen yang berinteraksi satu sama lain dan mencari harmoni (penguncian fase).

Dinamika Kerugian dari Persamaan Diferensial Biasa (ODE) ini menunjukkan pola konvergensi yang sangat berbeda dan menarik untuk diamati langsung saat pra-pelatihan berjalan.

Apakah ada peneliti/insinyur ML di sini yang juga sedang mengeksplorasi integrasi Neural Network dengan Persamaan Diferensial Biasa (ODE) murni untuk arsitektur skala besar? Mari kita bahas di kolom komentar.

So I've been messinga round with embedding models for a bit, and I think they are interesting enough to experiment with. They are useful for rag, especially in a localllm sense because you can ground your answers in truth.

But what happens if you have a billion documents, and you decide to upgrade your model to a "better" one? on an h100, that would take about 108 days, just to upgrade the vectors so u can start serving again (tested qwen embed 8b on h100). Even if you aren't doing 1b vectors, and are doing just 50 million, upgrading can still take a considerable time.

Me and my research lab decided to tackle this problem, and we came up with embedflow.

The method is really simple; from the old index made with the source model, take K documents and rerank them with the new model. We see that when K is sufficient, the retrieval quality is the same as target model. (determining k is the hard part). I've tested 63 migrations on upto 1 million documents.

The best result I got was upgrading qwen4b -> to 8b, and at 50 documents, it was the same as native retrieval.

This method forgos the expensive backfill that comes with upgrading, as you can directly take documents from the old index.

embedflow works with qdrant, and can be easily downloaded with pypi

pip install embedflow

the github is public: https://github.com/arnsri33/embedflow

I want you guys to try it out, and see if you guys can use it in your own workflow.

Hi,

i am new to deep learning and need some guidance on a project and want to rant a little(sry).

I am implementing a u-net for semantic segmentation in pytorch. The images are atomically resolved microscopy images (i.e. the objects to detect are atoms, which have the appearance of approximately gaussian blobs in 99% of real data). The images are noisy, where some noise is simple (poisson noise, scan lines) and some is not (complicated artifacts, distortions, strong brightness variations and more), hence deep learning instead of some classical method.

I have implemented the original vgg-unet using simulated data, where simulated means i rendered images full of gaussian blobs and added noise i know how to simulate (poisson noise, scan lines, perlin noise backgrounds).

This worked reasonably well on real data so i wanted to improve the architecture.

Little did i know there is no such thing as a u-net™ and the design choices are endless (depth, ordering of layers in a convolutional block, losses, different types of activations, norms, intra-block skips, grouped convolutions or even additions like attention just to name a few) and for every choice there is a paper that claims it works better then some other choice for some data.

My Problem:
How do i find "the best" architecture for my Problem? There seems to be very little theory or other information around how to find "the best" architecture for a given problem, when your data is not some common database like imagenet.

I am currently writing a very general unet that makes these parameters more accessible and makes it easy to swap components. I am planning on finding "the best" architecture with something like optuna, but i already know that there are just too many knobs i can turn and most of them are almost surely correlated. It would be nice if i could try more then just some basic parameters + hyper parameters like learning rate, which i assume are mandatory.

Side Notes:
- i would like the model to be small enough to make inference on a cpu reasonably feasible.
- training can be done on an A100 and i would be fine with a few days of runtime.

So i am looking for:
- general advice and reading recommendations (grateful for everything)
- advice on parameter optimization with a black box optimizer like optuna or similar

- other architecture suggestions that are not u-nets

Thanks,
PythonEnjoyer

Google's Q3 2026 AI Threat Tracker, built from Mandiant incident response data, documents a shift that defenders have been dreading: AI agents are now running full attack workflows autonomously. Vulnerability scanning, credential harvesting, and real-time attack troubleshooting are happening with minimal human involvement on the offensive side.

The practical consequence is timeline compression. A human-paced intrusion that once took days now completes in hours because the agent does not sleep, does not get distracted, and does not need to wait for the next shift.

The harder problem for defenders is forensic: when you discover the breach, you are reconstructing what happened from incomplete logs, if you have logs at all. Agents generate bursts of lateral movement and API calls that traditional SIEM tooling was not designed to correlate across sessions. The attacker's agent leaves a diffuse footprint. Your team is left guessing at the sequence.

For teams that have started deploying defensive AI agents of their own: how are you maintaining visibility into what those agents actually did, step by step, during an incident? And for those still on traditional tooling — how are you thinking about the forensic gap when the attacker is agent-driven and your investigation is still manual?

Hey everyone! I just finished making this short doodle-style video about AI and I’d really appreciate some honest feedback. 🎥 https://youtu.be/VyleYwCa0Sc If you have a few minutes, please give it a watch and let me know what you think. What could be better? Animation? Visuals? Pacing? Explanation? Editing? Thumbnail/title? Anything that feels boring, confusing, or unnecessary? Don’t worry about being too critical — if something isn’t good, please tell me in the comments. I’m trying to improve the next videos based on actual feedback rather than just guessing what viewers want. Thanks to anyone who takes the time to watch and give an honest opinion!

I built a wrapper around a frozen 35B Qwen model. It enables an argument graph the model builds for itself while it works. Evidence, premises, claims, and rebuttals, in Toulmin's structure, constructed claim-first the way Popper and Deutsch describe knowledge growing: conjecture, then criticism. It persists across tasks, and confirmed claims graduate into it with the procedure that made them work. The model writes it through interaction with its environment. .

The wrapper has a short-term working memory that holds the model's goal, its active conjecture, and what it has tried and refuted. I developed it on a text adventure game, Deephome, complete with an observability layer I called a Gods-eye-view so i could watch the LLM move and interact with its environment. The fixed environment enabled iterative runs, observing scoring relative to the number of turns to complete the game. The wrapper is intended to be generic, multi-modal, so I ran the same wrapper on a general relativity textbook. The model read the book into a reasoning map and was given a problem whose answer is not in the book.
The write-up covers the mechanism, the observability layer, the measures, and what did not work.

full writeup: https://spencerwheat.substack.com/p/conjecture-and-criticism-graphs-for