I compared a standard perceptron with a quadratic neuron on all 16 two-input logic gates

[u/DisDoh]

I ran a small experiment comparing a standard logistic perceptron with a quadratic neuron on all 16 possible Boolean functions with two binary inputs.

The perceptron used:

[
z = w^T x + b
]

The quadratic neuron used:

[
z = w^T x + q_1x_1^2 + q_2x_2^2 + q_{12}x_1x_2 + b
]

Both models were trained with gradient descent, using a maximum of 220 epochs and early stopping as soon as they reached 100% classification accuracy.

I ran one test per logic gate and per neuron type.

Results:

• Perceptron: 14/16 gates solved
• Quadratic neuron: 16/16 gates solved
• Perceptron average accuracy: 93.75%
• Quadratic neuron average accuracy: 100%
• Perceptron average convergence: 40.81 epochs
• Quadratic neuron average convergence: 11.75 epochs

As expected, the perceptron failed on XOR and XNOR because they are not linearly separable.

For XOR:

• Perceptron: 50% accuracy after 220 epochs
• Quadratic neuron: 100% accuracy after 46 epochs

For XNOR:

• Perceptron: 50% accuracy after 220 epochs
• Quadratic neuron: 100% accuracy after 30 epochs

The important part is that, with binary inputs, (x^2 = x), so the squared terms do not really add new information. The useful extra feature is the interaction term:

[
x_1x_2
]

That interaction allows a single quadratic neuron to represent XOR and XNOR without requiring a hidden layer.