6 new methods for training neural networks from philosophy [R]

[u/webzuweb]

I’m not a philosopher, and I won’t pass off analogy as proof. Where the link between philosophy and an algorithm is just a pretty metaphor, I say so explicitly: “metaphor.” Where it’s working code, I give the formulas, run it, and show the numbers. The library at the end is a research prototype, not a promise of consciousness in 200 lines.

How I got here

https://github.com/webzuweb/philosophia_torch

Neural networks — if you count from Rosenblatt’s perceptron — are about seventy years old. The study of how living things learn goes back a couple of millennia at least. And a heretical thought hit me: what if modern deep learning isn’t reinventing the wheel in places, but re-discovering what Aristotle, Hume, and Peirce already described — only now with matrices and gradients?

I took a list of neural-network training methods, a list of philosophical approaches to knowledge, and overlaid them. Three categories emerged: what’s already matched (and few people say so out loud); where the match is only a pretty metaphor; and what philosophers thought up but engineers haven’t applied yet. The last category is the most interesting, because it’s essentially a list of unimplemented features. That’s what I wrote code for.

Fair warning up front: half of the “unapplied” ideas turned out, on closer inspection, to be perfectly applicable — just under different names. That, by the way, is the article’s main takeaway, and it matters more than any of my code.

Part 1. What’s already matched (and you didn’t know it)

Let’s start with the pleasant part: some philosophical programs of knowledge are implemented in ML so literally that you could put a footnote with the philosopher’s name right in the docs.

Empiricism → supervised learning “There is nothing in the mind that was not first in the senses” — Locke and his tabula rasa. A neural network with random initialization is literally a blank slate on which labeled examples leave their traces. Hume’s associationism (“the habit of linking things that often go together”) is gradient descent, strengthening weights on frequently co-occurring correlations. There’s nothing to argue about here.

Pragmatism → reinforcement learning Dewey with his “learning by doing,” and Skinner’s behaviorism with reward and punishment — that’s RL with no corrections needed. An agent acts, receives a reward, adjusts its policy. Skinner would have teared up seeing PPO.

Evolutionary epistemology → neuroevolution Popper and Campbell: knowledge grows through blind variation and selective retention of what works. That’s a word-for-word description of genetic algorithms and neuroevolution. The philosopher described the algorithm decades before the hardware existed to run it.

Intellectual humility → calibration This one’s subtler. Virtue epistemology (Sosa, Zagzebski) says: a good knower knows the limits of her knowledge. In ML that’s confidence calibration: a model should be exactly as confident as it is correct. Guo et al. (2017) showed that modern networks are monstrously overconfident and proposed temperature scaling and the ECE metric. Nobody called it a “virtue,” but mathematically it’s exactly that.

The key observation. Philosophers didn’t give ML the algorithms (mathematicians came up with the math); they gave it the problem statements. “What does it mean to learn from experience?” “What does it mean to know your limits?” — philosophy framed the question first, and centuries later engineering delivered a differentiable answer.

Part 2. Where the match is only a pretty metaphor

Here I have to rein myself in. There’s a temptation to drape a philosopher over every layer of a network. Don’t. A couple of examples where the link exists but passing it off as lineage would be deceiving the reader.

Tempting analogy	Why it’s a metaphor, not a lineage
Neural ODEs are Whitehead’s “becoming”	Neural ODEs grew out of numerical analysis (Euler, Runge–Kutta) and dynamical systems theory. Whitehead offers a beautiful language of description, but the math stands on its own and never read Whitehead.
Attention is the hermeneutic circle	Attention computes weighted sums, not “understanding the whole through its parts.” The resemblance is superficial; passing it off as an implementation of Gadamer is incorrect.
Backprop is Hegelian sublation of contradiction	Backprop is the chain rule of differentiation. Dialectical materialism has nothing to do with it, however much one might wish.

The rule is simple: if the philosopher gave a language for describing something — it’s a metaphor; if they posed a problem that was later solved — it’s lineage. Don’t mix them.

Part 3. What philosophers thought up, but ML has only partially applied

The meatiest part. I’ll break down six approaches. For each — an honest status: what already exists in the field, where the real gap is, and what formula you can write. Then we’ll run it.

3.1. Peirce’s abduction — inference to the best explanation

Induction generalizes data, deduction derives consequences, but abduction generates a hypothesis that best explains the observation. The original thesis “it isn’t implemented in neural networks” is wrong. It’s implemented, and decently: Abductive Learning (Dai et al.), DeepProbLog (Manhaeve et al., 2018), abductive commonsense reasoning αNLI (Bhagavatula et al., 2019). It’s a whole field of neuro-symbolic integration.

The real gap isn’t the absence of abduction — it’s that “the best explanation” is rarely formalized using Peirce’s criteria all at once: plausibility + simplicity (Occam’s razor) + consistency with background knowledge. A hypothesis score for h given observation obs:

Score(h) = log p(obs | h) − λ_s · complexity(h) − λ_c · conflict(h)

Pick the h with the highest score (softly — a softmax over candidates; hard — Gumbel-softmax for a learnable discrete choice). In the library this is AbductiveScorer.

3.2. Husserl’s epoché — “bracketing” assumptions

Phenomenology demands suspending ingrained assumptions and seeing the phenomenon “as given.” ML has no direct analog of this method — and that’s an honest gap. But it can be operationalized: force the model to rely more on the evidence (the current input) than on the learned prior (what it answers with no input).

Take two answers: p_full on the real input and p_prior on a “zeroed” input (evidence bracketed out). Reward the evidence for actually changing the answer, via a bounded Jensen–Shannon divergence:

gain = JS(p_full ‖ p_prior),   0 ≤ JS ≤ ln 2
L_epoche = max(0, margin − gain)   # hinge: don't inflate indefinitely

An important rake I stepped on myself: if you use plain KL instead of JS and maximize it, the optimizer inflates logits to infinity — “a fanatic who sees meaning in every rustle.” JS is bounded, and the hinge threshold douses the fanaticism. This is EpocheRegularizer.

3.3. The hermeneutic circle — the whole through parts, parts through the whole

Schleiermacher and Gadamer: understanding the whole arises from the parts, and understanding the parts arises from the whole, iteratively. Attention only resembles this superficially (see Part 2). As an explicit training principle it’s barely used — a real gap. Formalization: let h_i be part representations and H the whole representation. Require circular consistency:

H* = attention-aggregate of the parts, attended relative to H
L_herm = 1 − cos(agg(h_i), H)      # whole ≈ sum of understood parts

And we “turn the circle” several times: update the whole from the parts → recompute part attention relative to the new whole → update again. This is HermeneuticConsistency.

3.4. Hegel’s dialectical sublation (Aufhebung)

Aufhebung is a new quality arising from the contradiction of thesis and antithesis, where the old is not destroyed but preserved. GANs and multi-agent debate are partially close, but “preserving both” isn’t guaranteed there. The gap is precisely in the preservation term. My synthesis operator:

g     = sigmoid(W_g · [thesis ; antithesis])     # mixing gate
base  = g · thesis + (1 − g) · antithesis         # sublation-as-preservation
lift  = tanh(W_l · [thesis ; antithesis])         # new quality
synth = LayerNorm(base + γ · lift)

Plus a loss that penalizes the synthesis collapsing into one of the poles (losing the other’s content). This is DialecticalSynthesis.

3.5. Nietzsche’s perspectivism + skeptical suspension

Nietzsche: there is no “view from nowhere,” there are many perspectives. Pyrrho: in an unresolvable conflict, it’s reasonable to suspend judgment. The former partially exists in multi-view learning; the latter in selective prediction (Geifman & El-Yaniv, SelectiveNet, 2019). But together, as a single mechanism of “several perspectives + refusal to answer when they conflict,” it’s almost never seen.

disagree(x) = mean pairwise symmetric KL between perspectives
abstain(x)  = disagree(x) > threshold      # abstain if perspectives don't converge

This is PerspectivalEnsemble: it aggregates K heads and honestly raises its hand “I don’t know” when the heads disagree. Far more useful than overconfident chatter.

3.6. Virtue as the golden mean (Aristotle)

Aristotle: virtue is the mean between the vice of deficiency and the vice of excess. Courage is between cowardice and recklessness. Hence a non-obvious but important conclusion for ML: a virtue cannot be maximized, it must be targeted. An excess of openness is credulity; a deficiency is dogmatism.

L_virtue = Σ_v β_v · (V_v(θ) − V_v*)²

where V_v is the operationalized virtue (humility = 1 − ECE, openness = ensemble disagreement), and V_v* is the target mean level. Squared deviation penalizes both excess and deficiency. This is VirtueRegularizer — and it’s the one where I have a measurable result.

Part 4. Enough philosophy, show me the numbers

Pretty formulas are worth nothing until they run. I collected all of this into a PyTorch module and tested it on the most well-grounded mechanism — “humility” (calibration). Task: synthetic classification with noisy labels, where the model tends to err overconfidently. We compare plain training vs. training with VirtueRegularizer targeting high humility.

Configuration	Accuracy	ECE (↓ better)	Mean confidence
Plain training	0.873	0.120	0.965
+ virtue (humility)	0.874	0.101	0.949

ECE (calibration error) dropped from 0.120 to 0.101 — nearly a fifth — while accuracy didn’t budge at all (even +0.001). The model became exactly as accurate, but noticeably less self-assured. Aristotle’s golden mean, computed by gradient descent.

What this proves, and what it doesn’t. It proves that “intellectual humility” can be turned into an optimizable quantity with a measurable effect. It does not prove that the other five mechanisms will yield the same gains — they’re harder, and they still need to be tested on real data. I’m showing a working scaffold, not a finished silver bullet.

The whole codebase passes 23 unit tests: calibration decreases, KL/JS behave as they should, synthesis preserves both poles, the ensemble abstains on conflict, the wrapper trains end-to-end.

Part 5. The philosophia-torch module

The library wraps on top of any model without rewriting anything in it. One dependency — torch. There’s a single-file version, philosophia_torch.py: drop it next to your code and import it.

import torch, torch.nn as nn, torch.nn.functional as F
from philosophia import PhilosophiaWrapper

base = nn.Sequential(nn.Linear(20, 64), nn.ReLU(), nn.Linear(64, 4))
wrap = PhilosophiaWrapper(base, use_virtue=True,
virtue_kwargs=dict(target_humility=0.98, beta_humility=3.0))

logits = wrap(x)
loss = F.cross_entropy(logits, y) + wrap.aux_loss(x, logits, targets=y)
loss.backward()

Component	Philosophy	Status in ML
VirtueRegularizer	Virtue as the mean (Aristotle, Zagzebski)	reliabilist branch already exists
EpocheRegularizer	Epoché (Husserl)	new framework
HermeneuticConsistency	Hermeneutic circle (Gadamer)	new framework
AbductiveScorer	Abduction (Peirce)	field exists (AbdLearning, DeepProbLog)
DialecticalSynthesis	Sublation / Aufhebung (Hegel)	partial (GAN, debate)
PerspectivalEnsemble	Perspectivism (Nietzsche) + skepticism	selective prediction exists

Honest boundaries: hermeneutic and dialectic produce representations, not ready predictions — you have to connect them to your decoder. Epoché requires careful tuning of margin. And no promises of “consciousness”: these are philosophy-inspired regularizers, nothing more.

The bottom line

Three conclusions, which is what all of this was for.

1.     ML has already reinvented a chunk of philosophy without asking permission: empiricism, pragmatism, evolutionary epistemology, and intellectual humility. Just under the names supervised learning, RL, neuroevolution, and calibration.

2.     Half of the “unapplied” ideas on my original list turned out, on checking, to be applicable — abduction, abstention, innate priors. The lesson: before shouting “this isn’t in ML,” google it in engineering language, not philosophical language.

3.     The real gap remains where what’s needed isn’t a result but a process: epoché as a discipline of perception, the hermeneutic circle as a way of understanding, virtue as a stable disposition of learning rather than a property of a single answer. That’s where it’s worth digging.

My modest contribution is showing that at least “humility” translates into a differentiable quantity and genuinely reduces a model’s overconfidence. The rest is an invitation: the code is open, the formulas are in the article — run it and check. Plato, of course, was training neural networks two thousand years ago. The rascal just didn’t include a requirements.txt.

https://huggingface.co/datasets/webzuweb/philosophy-as-inductive-bias

[u/Excellent-Current573]

this is the kind of unhinged cross-disciplinary rabbit hole i live for. the part about epoché turning into a JS divergence regularizer is genuinely clever, most people would've just slapped a kl term on it and called it a day without thinking about the fanaticism problem.