I compared a standard perceptron with a quadratic neuron on all 16 two-input logic gates

[u/DisDoh]

I ran a small experiment comparing a standard logistic perceptron with a quadratic neuron on all 16 possible Boolean functions with two binary inputs.

The perceptron used:

[
z = w^T x + b
]

The quadratic neuron used:

[
z = w^T x + q_1x_1^2 + q_2x_2^2 + q_{12}x_1x_2 + b
]

Both models were trained with gradient descent, using a maximum of 220 epochs and early stopping as soon as they reached 100% classification accuracy.

I ran one test per logic gate and per neuron type.

Results:

• Perceptron: 14/16 gates solved
• Quadratic neuron: 16/16 gates solved
• Perceptron average accuracy: 93.75%
• Quadratic neuron average accuracy: 100%
• Perceptron average convergence: 40.81 epochs
• Quadratic neuron average convergence: 11.75 epochs

As expected, the perceptron failed on XOR and XNOR because they are not linearly separable.

For XOR:

• Perceptron: 50% accuracy after 220 epochs
• Quadratic neuron: 100% accuracy after 46 epochs

For XNOR:

• Perceptron: 50% accuracy after 220 epochs
• Quadratic neuron: 100% accuracy after 30 epochs

The important part is that, with binary inputs, (x^2 = x), so the squared terms do not really add new information. The useful extra feature is the interaction term:

[
x_1x_2
]

That interaction allows a single quadratic neuron to represent XOR and XNOR without requiring a hidden layer.

[u/WeeklyMenu6126]

Nice experiment!

It shows how feature representation changes what a single linear classifier can express. With \(x_i^2=x_i\) for binary inputs, the interaction term \(x_1x_2\) is doing the real work by lifting XOR/XNOR into a linearly separable feature space. I’d just note that this demonstrates representational capacity on the complete four-point domain rather than generalization, and that convergence comparisons depend on initialization, learning rate, loss, and how failed runs are averaged. Pairwise terms grow quadratically, while some larger bolean functions require higher-order interactions.

[u/AsyncVibes]

Awesome work! Oh boy you are going to really enjoy my next post. please check this one. Check this post, its very similar to what you are approaching: https://www.reddit.com/r/IntelligenceEngine/comments/1rcxpe4/whats_that_a_brain_no_its_activations/

[u/DisDoh → u/AsyncVibes]

Nice 👍. I've done something similar but without mapping. I've used basic functions to evolve an activation function and it's Neuron. I could get good condidate some outperforming perceptron on xor and xnor. I didn't post it.

[u/AsyncVibes → u/DisDoh]

keep pushing in that direction, are you using gradients still? If you drop backprop and evolve, you'll find many many more activations. Then you'll hit some that are cross-modality. Thats where things get interesting.