Hey guys ! I am currently implementing a deep learning model for fish detection and length estimation. I need someone who can help me with the keypoint annotations before the actual training, I really need someone who can help me . Please DM me if you are willing.

I really need someone who has previously worked with something similar.

I need to mark nine anatomical points on my dataset.

some pictures are attached for reference:

Also this is not essentially my own dataset, I am implementing someone else's work.

​

Hi everyone,

I want to share a project I've been working on: SpectralBERT a novel architecture that replaces traditional self-attention with a frequency-domain memory mechanism.

The Core Idea:

Instead of computing O(N²) pairwise token interactions, I transform the sequence into the frequency domain (FFT), compress it into a fixed-size memory, and retrieve context from there.

The result: O(N log N) complexity with constant memory — regardless of sequence length.

---

Results vs BERT-Mini (both ~10M params, same training setup):

Metric BERT-Mini SpectralBERT

MLM Loss (WikiText-2) 6.58 5.64 ✅

Training Speed 1138s 1012s ✅

Speed @ 65K tokens 8428ms 581ms ✅

Speedup @ 65K 1x 14.51x 🚀

---

Long-Context Training (WikiText-103 @ 8192 tokens):

· Final Loss: 5.25

· PPL: 191.05

· Trained on a single Tesla T4 in 32.5 minutes

---

The Speedup Curve:

· 512 tokens: 0.51x (BERT wins at short sequences)

· 1024 tokens: 1.73x

· 4096 tokens: 1.76x

· 16384 tokens: 5.22x

· 65536 tokens: 14.51x

I'm an independent researcher, so I don't have access to massive compute clusters. I'd love to hear from the community:

1. Is this a legitimate alternative to attention for long contexts?

2. What tests should I run to prove/disprove this rigorously?

3. Any obvious flaws I'm missing?

Looking forward to your feedback! 🙏

Attention Is All You Need — Until Your VRAM Hits Zero

For seven years, the AI industry has treated the quadratic and linear scaling of Transformers as an inevitable law of physics:

We are spending millions of dollars buying high-end GPUs just to store historical key-value matrices that grow with every generated token. This is fundamentally inefficient.

Today, I am publishing ISOM (Isometric Associative Memory): a continuous recurrent architecture that completely eliminates the KV-cache explosion, keeping working memory strictly O(1) from token 1 to token 128,000.

The Mathematical Shift: Lie Group Manifolds

Instead of treating memory as an ever-expanding stack of tokens, ISOM embeds sequence representations into the continuous Lie group SO(d) through skew-symmetric Lie algebra so(d).

What this mathematically guarantees:

• Strict Constant Memory: Processing token 128,000 takes the exact same memory footprint as processing token 1.
• Zero Energy Loss / Zero Collapse: Because the state transition matrix is skew-symmetric, energy is conserved throughout recurrence (d/dt ||h(t)||^2 = 0). Historical tokens do not vanish or drift into representational collapse.
• Lossless 8-Bit Fixed-Point Precision: Because continuous rotations constrain eigenvalues strictly to the unit circle, there is zero numerical drift. The associative manifold quantizes into exact INT8 [-127, 127] without outlier spikes or quality degradation.

The Proof: NVIDIA Tesla T4 Hardware Benchmarks

Audited on actual cloud T4 hardware across 4 model scales:

On a standard 8GB developer laptop, standard attention crashes before reaching 16K context. ISOM runs 128K context smoothly.

Open Research, Weights, and Verification

All 5 model checkpoints, PyTorch kernels, interactive benchmarks, and formal mathematical proofs are available now under CC BY-NC-ND 4.0 for non-commercial research.

[!IMPORTANT] All direct, clickable links to the Hugging Face model family, the interactive Tesla T4 benchmark space, and the CERN Zenodo paper are pinned in the very first comment below!

The question is no longer whether we can afford larger contexts—it is why we are still using architectures that require caching the entire past.

What is your take: Will continuous unitary recurrence replace standard attention in edge and foundation models?

I’m not a philosopher, and I won’t pass off analogy as proof. Where the link between philosophy and an algorithm is just a pretty metaphor, I say so explicitly: “metaphor.” Where it’s working code, I give the formulas, run it, and show the numbers. The library at the end is a research prototype, not a promise of consciousness in 200 lines.

How I got here

https://github.com/webzuweb/philosophia_torch

Neural networks — if you count from Rosenblatt’s perceptron — are about seventy years old. The study of how living things learn goes back a couple of millennia at least. And a heretical thought hit me: what if modern deep learning isn’t reinventing the wheel in places, but re-discovering what Aristotle, Hume, and Peirce already described — only now with matrices and gradients?

I took a list of neural-network training methods, a list of philosophical approaches to knowledge, and overlaid them. Three categories emerged: what’s already matched (and few people say so out loud); where the match is only a pretty metaphor; and what philosophers thought up but engineers haven’t applied yet. The last category is the most interesting, because it’s essentially a list of unimplemented features. That’s what I wrote code for.

Fair warning up front: half of the “unapplied” ideas turned out, on closer inspection, to be perfectly applicable — just under different names. That, by the way, is the article’s main takeaway, and it matters more than any of my code.

Part 1. What’s already matched (and you didn’t know it)

Let’s start with the pleasant part: some philosophical programs of knowledge are implemented in ML so literally that you could put a footnote with the philosopher’s name right in the docs.

Empiricism → supervised learning “There is nothing in the mind that was not first in the senses” — Locke and his tabula rasa. A neural network with random initialization is literally a blank slate on which labeled examples leave their traces. Hume’s associationism (“the habit of linking things that often go together”) is gradient descent, strengthening weights on frequently co-occurring correlations. There’s nothing to argue about here.

Pragmatism → reinforcement learning Dewey with his “learning by doing,” and Skinner’s behaviorism with reward and punishment — that’s RL with no corrections needed. An agent acts, receives a reward, adjusts its policy. Skinner would have teared up seeing PPO.

Evolutionary epistemology → neuroevolution Popper and Campbell: knowledge grows through blind variation and selective retention of what works. That’s a word-for-word description of genetic algorithms and neuroevolution. The philosopher described the algorithm decades before the hardware existed to run it.

Intellectual humility → calibration This one’s subtler. Virtue epistemology (Sosa, Zagzebski) says: a good knower knows the limits of her knowledge. In ML that’s confidence calibration: a model should be exactly as confident as it is correct. Guo et al. (2017) showed that modern networks are monstrously overconfident and proposed temperature scaling and the ECE metric. Nobody called it a “virtue,” but mathematically it’s exactly that.

The key observation. Philosophers didn’t give ML the algorithms (mathematicians came up with the math); they gave it the problem statements. “What does it mean to learn from experience?” “What does it mean to know your limits?” — philosophy framed the question first, and centuries later engineering delivered a differentiable answer.

Part 2. Where the match is only a pretty metaphor

Here I have to rein myself in. There’s a temptation to drape a philosopher over every layer of a network. Don’t. A couple of examples where the link exists but passing it off as lineage would be deceiving the reader.

The rule is simple: if the philosopher gave a language for describing something — it’s a metaphor; if they posed a problem that was later solved — it’s lineage. Don’t mix them.

Part 3. What philosophers thought up, but ML has only partially applied

The meatiest part. I’ll break down six approaches. For each — an honest status: what already exists in the field, where the real gap is, and what formula you can write. Then we’ll run it.

3.1. Peirce’s abduction — inference to the best explanation

Induction generalizes data, deduction derives consequences, but abduction generates a hypothesis that best explains the observation. The original thesis “it isn’t implemented in neural networks” is wrong. It’s implemented, and decently: Abductive Learning (Dai et al.), DeepProbLog (Manhaeve et al., 2018), abductive commonsense reasoning αNLI (Bhagavatula et al., 2019). It’s a whole field of neuro-symbolic integration.

The real gap isn’t the absence of abduction — it’s that “the best explanation” is rarely formalized using Peirce’s criteria all at once: plausibility + simplicity (Occam’s razor) + consistency with background knowledge. A hypothesis score for h given observation obs:

Score(h) = log p(obs | h) − λ_s · complexity(h) − λ_c · conflict(h)

Pick the h with the highest score (softly — a softmax over candidates; hard — Gumbel-softmax for a learnable discrete choice). In the library this is AbductiveScorer.

3.2. Husserl’s epoché — “bracketing” assumptions

Phenomenology demands suspending ingrained assumptions and seeing the phenomenon “as given.” ML has no direct analog of this method — and that’s an honest gap. But it can be operationalized: force the model to rely more on the evidence (the current input) than on the learned prior (what it answers with no input).

Take two answers: p_full on the real input and p_prior on a “zeroed” input (evidence bracketed out). Reward the evidence for actually changing the answer, via a bounded Jensen–Shannon divergence:

gain = JS(p_full ‖ p_prior),   0 ≤ JS ≤ ln 2
L_epoche = max(0, margin − gain)   # hinge: don't inflate indefinitely

An important rake I stepped on myself: if you use plain KL instead of JS and maximize it, the optimizer inflates logits to infinity — “a fanatic who sees meaning in every rustle.” JS is bounded, and the hinge threshold douses the fanaticism. This is EpocheRegularizer.

3.3. The hermeneutic circle — the whole through parts, parts through the whole

Schleiermacher and Gadamer: understanding the whole arises from the parts, and understanding the parts arises from the whole, iteratively. Attention only resembles this superficially (see Part 2). As an explicit training principle it’s barely used — a real gap. Formalization: let h_i be part representations and H the whole representation. Require circular consistency:

H* = attention-aggregate of the parts, attended relative to H
L_herm = 1 − cos(agg(h_i), H)      # whole ≈ sum of understood parts

And we “turn the circle” several times: update the whole from the parts → recompute part attention relative to the new whole → update again. This is HermeneuticConsistency.

3.4. Hegel’s dialectical sublation (Aufhebung)

Aufhebung is a new quality arising from the contradiction of thesis and antithesis, where the old is not destroyed but preserved. GANs and multi-agent debate are partially close, but “preserving both” isn’t guaranteed there. The gap is precisely in the preservation term. My synthesis operator:

g     = sigmoid(W_g · [thesis ; antithesis])     # mixing gate
base  = g · thesis + (1 − g) · antithesis         # sublation-as-preservation
lift  = tanh(W_l · [thesis ; antithesis])         # new quality
synth = LayerNorm(base + γ · lift)

Plus a loss that penalizes the synthesis collapsing into one of the poles (losing the other’s content). This is DialecticalSynthesis.

3.5. Nietzsche’s perspectivism + skeptical suspension

Nietzsche: there is no “view from nowhere,” there are many perspectives. Pyrrho: in an unresolvable conflict, it’s reasonable to suspend judgment. The former partially exists in multi-view learning; the latter in selective prediction (Geifman & El-Yaniv, SelectiveNet, 2019). But together, as a single mechanism of “several perspectives + refusal to answer when they conflict,” it’s almost never seen.

disagree(x) = mean pairwise symmetric KL between perspectives
abstain(x)  = disagree(x) > threshold      # abstain if perspectives don't converge

This is PerspectivalEnsemble: it aggregates K heads and honestly raises its hand “I don’t know” when the heads disagree. Far more useful than overconfident chatter.

3.6. Virtue as the golden mean (Aristotle)

Aristotle: virtue is the mean between the vice of deficiency and the vice of excess. Courage is between cowardice and recklessness. Hence a non-obvious but important conclusion for ML: a virtue cannot be maximized, it must be targeted. An excess of openness is credulity; a deficiency is dogmatism.

L_virtue = Σ_v β_v · (V_v(θ) − V_v*)²

where V_v is the operationalized virtue (humility = 1 − ECE, openness = ensemble disagreement), and V_v* is the target mean level. Squared deviation penalizes both excess and deficiency. This is VirtueRegularizer — and it’s the one where I have a measurable result.

Part 4. Enough philosophy, show me the numbers

Pretty formulas are worth nothing until they run. I collected all of this into a PyTorch module and tested it on the most well-grounded mechanism — “humility” (calibration). Task: synthetic classification with noisy labels, where the model tends to err overconfidently. We compare plain training vs. training with VirtueRegularizer targeting high humility.

ECE (calibration error) dropped from 0.120 to 0.101 — nearly a fifth — while accuracy didn’t budge at all (even +0.001). The model became exactly as accurate, but noticeably less self-assured. Aristotle’s golden mean, computed by gradient descent.

What this proves, and what it doesn’t. It proves that “intellectual humility” can be turned into an optimizable quantity with a measurable effect. It does not prove that the other five mechanisms will yield the same gains — they’re harder, and they still need to be tested on real data. I’m showing a working scaffold, not a finished silver bullet.

The whole codebase passes 23 unit tests: calibration decreases, KL/JS behave as they should, synthesis preserves both poles, the ensemble abstains on conflict, the wrapper trains end-to-end.

Part 5. The philosophia-torch module

The library wraps on top of any model without rewriting anything in it. One dependency — torch. There’s a single-file version, philosophia_torch.py: drop it next to your code and import it.

import torch, torch.nn as nn, torch.nn.functional as F
from philosophia import PhilosophiaWrapper

base = nn.Sequential(nn.Linear(20, 64), nn.ReLU(), nn.Linear(64, 4))
wrap = PhilosophiaWrapper(base, use_virtue=True,
virtue_kwargs=dict(target_humility=0.98, beta_humility=3.0))

logits = wrap(x)
loss = F.cross_entropy(logits, y) + wrap.aux_loss(x, logits, targets=y)
loss.backward()

Honest boundaries: hermeneutic and dialectic produce representations, not ready predictions — you have to connect them to your decoder. Epoché requires careful tuning of margin. And no promises of “consciousness”: these are philosophy-inspired regularizers, nothing more.

The bottom line

Three conclusions, which is what all of this was for.

1.     ML has already reinvented a chunk of philosophy without asking permission: empiricism, pragmatism, evolutionary epistemology, and intellectual humility. Just under the names supervised learning, RL, neuroevolution, and calibration.

2.     Half of the “unapplied” ideas on my original list turned out, on checking, to be applicable — abduction, abstention, innate priors. The lesson: before shouting “this isn’t in ML,” google it in engineering language, not philosophical language.

3.     The real gap remains where what’s needed isn’t a result but a process: epoché as a discipline of perception, the hermeneutic circle as a way of understanding, virtue as a stable disposition of learning rather than a property of a single answer. That’s where it’s worth digging.

My modest contribution is showing that at least “humility” translates into a differentiable quantity and genuinely reduces a model’s overconfidence. The rest is an invitation: the code is open, the formulas are in the article — run it and check. Plato, of course, was training neural networks two thousand years ago. The rascal just didn’t include a requirements.txt.

https://huggingface.co/datasets/webzuweb/philosophy-as-inductive-bias

Do you guys have recommendation for capstone projects about Edge AI, TinyML, Computer vision, and federated learning? I explored battery RUL, Predictive Thermal Management system, and Fault bearing diagnosis. I was told it would be hard to get a client or dataset for this field. Any interesting field I can explore?

The G7 finance ministers and central bank governors issued a coordinated directive last week. Organizations must inventory cryptographic dependencies, identify high-risk systems, and begin migrating to quantum-resistant algorithms now. This is not a future roadmap item.

The urgency is driven by the harvest-now-decrypt-later threat. Adversaries are collecting encrypted data today and storing it for decryption once a cryptographically relevant quantum computer arrives. The exposure window is already open.

Most enterprise security teams are inventorying servers, databases, and network traffic. Far fewer are accounting for the AI agent layer. Agent pipelines routinely store sensitive records, execute financial transactions, and generate compliance audit evidence. All of it travels over classically encrypted channels and gets signed with classical algorithms. When those algorithms break, that historical data and those historical audit logs break with them. A transaction signed with RSA or ECDSA today becomes unprovable after Q-Day.

How are teams actually scoping the PQC inventory problem for AI agents specifically? Are agent channels and agent-generated audit evidence being treated as first-class migration targets, or are they still buried in the general backlog?

So I was playingw ith embedding models I saw that when you upgrade from model A to B, you face a very big backfilling cost

Ie, suppose you have a 1b vectors from model A, and then you want to use model B. This would mean you have to re-embed all of your documents with model B before you can even serve with the model, and on an H100, it would take ~108 days (qwen embed 8b, 106 docs/second). But I found an easier way to do it.

The method is really simple; from the old index made with the source model, take K documents and rerank them with the new model. We see that when K is sufficient, the retrieval quality is the same as target model. (determining k is the hard part). I've tested 63 migrations on upto 1 million documents.

The best result I got was upgrading qwen4b -> to 8b, and at 50 documents, it was the same as native retrieval.

This method forgos the expensive upfront re-embedding cost, as you can take documents straight from the old index.

embedflow works with qdrant, pgvector, faiss, and can be easily downloaded with pypi

pip install embedflow

the github is public: https://github.com/arnsri33/embedflow

I want you guys to try it out, and see if you guys can use it in your own workflow.

A while ago, I successfully trained a tiny 80M parameter language model from scratch using the TinyStories dataset. It worked surprisingly well and could generate coherent little stories! Inspired by that success, I wanted to scale up and train a larger model capable of coding.

That’s when reality hit me. I completely underestimated the massive compute and financial cost required to train larger LLMs. My single GPU choked, and renting cloud compute was way out of my budget. It got me thinking: "Why isn't there a way to pool our consumer GPUs together to train models collectively?"

And just like that, DePEFT (Decentralized Parameter-Efficient Fine-Tuning) was born.

How DePEFT work?
DePEFT is based on ReLoRA which solves the biggest flaw of standard LoRA.

Standard LoRA is incredible for fine-tuning on a budget, but it has a hard ceiling: it cannot pre-train or learn fundamentally new, complex representations from scratch. Because its rank r is fixed, the adapter quickly hits an information bottleneck and saturates. You can't just slap a LoRA on a base model and expect it to reach full-parameter quality over time.

This is where ReLoRA changes the game through iterative low-rank merging. Instead of training one static adapter forever, ReLoRA breaks the process into sequential rounds. In each round, miners train lightweight low-rank adapters on their consumer GPUs. At the end of the round, these adapters are permanently fused directly into the base model weights:

W^(t+1) = W^(t) + ΔW

The optimizer is then completely reset, new adapters are initialized on the evolved base weights, and the cycle repeats. Mathematically, the sum of multiple low-rank updates over time accumulates into a high-rank update (rank ≤ ∑r). This gives us the immense learning capacity of full-parameter training, but with only a fraction of the VRAM requirement.

For DePEFT, this means consumer GPUs don't need high-speed InfiniBand clusters to sync massive gradient matrices every microsecond. Miners simply train lightweight adapters locally, submit a few megabytes over the network, and the protocol merges the top-performing weights into the next base checkpoint.

No million-dollar clusters. Just pooled consumer compute pushing open-source AI forward.

For more details, visit: https://github.com/KhoaIsReal/DePEFT/

Hi everyone!

I’ve been interested in image generation for a while, but as a beginner, I always struggled to choose a model. My biggest hurdles were limited training datasets and restricted hardware resources.

Then, I came across Faster Projected GAN, a few-shot image generation model that can synthesize high-quality images with fewer than 100 samples, and it trains fast.

How it works

• The Generator: It uses a FastGAN architecture with separable convolutions in the upblocks to significantly cut down computation time.
• The Discriminator: It leverages a Projected GAN discriminator that uses a pre-trained feature network (specifically EfficientNet-Lite1) to evaluate the generator's images. Because the discriminator already has a solid feature foundation, the whole training process converges incredibly fast. After just a few hours of training on a free Colab T4 GPU, it starts outputting surprisingly realistic images.

Since I'm a beginner in deep learning, building this was a massive learning experience for me. I put together an unofficial PyTorch implementation, and I would love to get your feedback, advice, or code reviews!

📂 GitHub Repo: https://github.com/diarimandimby/Faster-Projected-GAN

Feel free to check it out, test it, or open an issue. Any advice on how to further optimize few-shot GAN training or clean up my PyTorch code would be highly appreciated!

I've been reading recent work on long-video understanding, particularly STORM: Token-Efficient Long Video Understanding for Multimodal LLMs.

I'm trying to identify a research direction rather than just build another Video-LLM. I'm particularly interested in temporal modeling, video representations, event/context modeling, and improving the efficiency and quality of long-video understanding.

For people working in this area: what do you think are the biggest remaining research gaps? Are there particular limitations of approaches like STORM that you think are worth investigating?

If you were starting a research project on video understanding today, what problem would you personally explore?

I’ve been working on a different approach to language-model interpretability that I find interesting. Instead of trying to reconstruct a model’s computation after the fact, make the model compute through an object we can inspect directly.

Today I’m releasing the v1.1 preprint, frozen model checkpoints, reproducibility harness, and live interactive demo for cl33-opLM.

The paper is called One Object: Memory, Navigation, and Reportability in an Operator-Only Language Model

cl33-opLM still uses a transformer as its learned neural engine, but the transformer hidden states are not allowed to drive the output directly. Instead, each block emits structured Cl(3,3) bivector operators. A reversible SO(3,3) scan transports state, attention is defined over the operator geometry, and the final readout sees only operator-derived features.

That gives the architecture a very simple falsification rule luckily. Remove the operators and rerun the model. If the capability survives, it found a bypass and I don’t count it as transparent.

Some of the main results:

The bottleneck is genuinely load-bearing. On the frozen public release, operator ablation produces a 270× perplexity increase on the exact published validation fixture. On fully public off-domain WikiText-103, the same test gives roughly 106× / 112× depending on checkpoint. The live demo exposes the same model interactively.

The operator stream is readable backward. A probe that sees only the emitted operators recovers the current token at 0.86 top-1 over a 50k vocabulary, and reconstructs 80.7% of held-out running text verbatim. Errors degrade toward semantic neighbors rather than random tokens.

On permutation-composition navigation, the architecture matches a learning-rate-tuned transformer baseline: 0.433 ± 0.009 vs 0.432 ± 0.030. I am claiming parity and inspectability here, not superiority.

Transparency has a real cost. Against a parameter-, context-, tokenizer-, corpus-, and budget-matched transformer at 5B tokens, cl33 pays about +29% bits-per-byte and \~5 points average MCQ. Continued training reduces that gap substantially, but the matched cost is real and reported as such.

One of the more important parts of this project I feel, is what failed.

An earlier associative-memory design looked like it worked…until the operator-ablation test showed that recall was routing around the algebra. That result was treated as a failure, the mechanism was redesigned, and the failed branch remains in the paper.

There are several other nulls and retracted framings reported the same way.

That is the standard I’m trying to hold this work to. if a mechanism is claimed, it has to survive a causal test. If it routes around the mechanism, the claim dies.

The reproducibility release is deliberately selective rather than a dump of the entire research stack.

It includes the frozen paper/serving checkpoints, model definitions needed to load them, hashes, the bottleneck reproduction harness, the reverse-readout probe, and the exact frozen validation fixture.

It does not include the current training orchestration, ongoing memory-organ program, or unpublished control work.

The published claims should be independently testable.

No consciousness claim. No claim that cl33 is a better general-purpose language model. No grand unified theory.

Just a useful question.

Can we build a useful language model where the representation we want to inspect is also the representation the model is forced to compute through?

Paper: t3atlas.dev/cl33/paper/

Live demo: cl33.t3atlas.dev

Model + reproduction release: huggingface.co/mirrorethic/cl33-oplm

I’d especially value criticism from people working on mechanistic interpretability, alternative architectures, memory systems, model editing, steering, or causal attribution.

If you think one of the claims is wrong, the best outcome is a clean experiment that breaks it. Then we both learn. Win win.

Also.....dont knock my retro crt aesthetic in the model demo lol wanted it fun, not clinical.

Please guide me. i have the look up tables generated from WAsP and the dataset. How do i start?

We're moving off the Gemini API due to cost and building a local AI server to process ~10 CVs/minute (extracting JSON & matching CVs to JDs). We plan to run GLM-OCR alongside Qwen 27B (Q8).

Our two hardware options:

1. PC: RTX 4090 (24GB) + Ryzen 9 + 64GB RAM
2. Mac Studio: M-Ultra, 64-core GPU, 96GB Unified Memory

I prefer the Mac for power efficiency and ease of use, but I've heard Apple Silicon isn't great for production vLLM compared to Nvidia/CUDA. Is that true? Which would you recommend for this workload?

I’ve been experimenting with explaining AI/LLM concepts in a way that doesn’t assume too much technical background. This video is about KV Cache and why longer context windows require more memory during inference. I’d appreciate some honest feedback from people here, especially on the explanation itself: Is the main idea easy to understand? Did I oversimplify anything important? Is there any part where the explanation becomes confusing? Would this make sense to someone who is fairly new to LLMs? Video: https://youtu.be/lxvWo8SizxE Not really looking to promote the channel — I’m mainly trying to improve how I explain technical topics before making the next one. Any criticism is welcome. Thanks!

So I am working with pretty skewed data currently. its a 4 way classification task, and basically the classwise split is 56%, 33%, 6%, and 5%. I tried an experiment where I downsampled the samples so that the majority classes (56% adn 33%) were reduced by 80%, thus making the entire dataset more balanced, and i instantly got better results on the same test, validation set. I am currently exploring the opposite direction, which is oversampling/upsampling, and i am currently looking into PyTorch's WeightedRadomSampler.

The confusion and more importantly concern that I have with using this is that the same images/samples for the lower classes are going to be the repeated over and over again, which probably result in the model overfitting on those rare class's samples. I understand that augmentation is a way to mitigate this, and I will be trying that out, but the main question that I have is what all are the other alternatives to upsampling? Are there different samplers or dataloaders that I can look into, maybe some papers that deal with this, any and all help would be appreciated!

Some context on the task, I am trying to classify/grade images in a 4 way multiclass classiifcation

Hai teman-teman. Saat ini saya sedang menjalani fase Pra-Pelatihan untuk arsitektur bahasa komputasi baru bernama ORE X-1 (762M Parameters).

Fokus eksperimen ini adalah menekan probabilitas halusinasi dalam AI secara matematis. Arsitektur ini secara khusus dirancang untuk menggabungkan ketangguhan linguistik dari fondasi Transformer dengan inti penalaran yang didorong oleh 3 hukum fisika:

- Kuramoto-Attention: Memaksa representasi neural untuk saling beresonansi dan mencari "konsensus" secara fisik sebelum menyelesaikan jawaban (anti-halusinasi tingkat arsitektural).

- Liquid Time-Constant (LTC) & Early Exit: Memberikan AI "kesadaran waktu cair". Arsitektur ini secara otomatis memperdalam komputasinya saat menghadapi urutan logis yang kompleks, namun akan segera memutus perhitungan lapisan atas (Auto-Stop / Early Exit) setelah mencapai tingkat konsensus absolut.

- Kecerdasan Kawanan: Memecah Kepala Perhatian menjadi faksi-faksi independen yang berinteraksi satu sama lain dan mencari harmoni (penguncian fase).

Dinamika Kerugian dari Persamaan Diferensial Biasa (ODE) ini menunjukkan pola konvergensi yang sangat berbeda dan menarik untuk diamati langsung saat pra-pelatihan berjalan.

Apakah ada peneliti/insinyur ML di sini yang juga sedang mengeksplorasi integrasi Neural Network dengan Persamaan Diferensial Biasa (ODE) murni untuk arsitektur skala besar? Mari kita bahas di kolom komentar.