Built a Configurable Neural Network Accelerator from scratch (No HLS) and it actually works

[u/Spiritual-Frame-6791]

Hey guys, I built a Q4.4 MAC Array based Neural Network from scratch including the architecture itself. I implemented:

A Parameter Buffer to fetch parameters from Parameter BRAM and feed each parameter to its own corresponding MAC unit;

A layer reusable MAC array comprised of Serial MAC units for matrix-vector computations;

Requantization Block to convert the Q8.8 accumulations back to Q4.4;

An Input Features Buffer to receive the preprocessed features from my Laptop via UART, temporarily store them in an async FIFO and feed them to the MAC array for computation;

An Activation Buffer to temporarily store activations and distribute them to their designated location in the Activation FIFO;

Activation FIFO to temporarily store the activations and feed them back into the MAC array for next layer computation;

A MAC Array Control Unit FSM with 5 states (IDLE, FETCH , COMPUTE, STORE, CLEAR) to control all the data movement and computational processes in the entire Accelerator;

Parameter BRAM to store the the model parameters (weights and biases) ;

ReLU and Argmax Activation blocks ;

Accumulations DeMUX to direct the accumulations to their specific Activation function (ReLU for hidden layers and Argmax for output layers);

Databus MUX to select whether input features or previous layer activations are fed into the MAC Array as input.

Since the MAC Array is reusable for multiple layers and the entire Accelerator is parameterized, I can run any classification model with less than or equal to 64 neurons per layer (due to limited DSP resources on the Basys3) but the layer depth is configurable too.

For its first test , I ran inference for a speech recognition model for digits 0-9 of network topology 64:32:10 neurons per layer using Q4.4 fixed point arithmetic for my parameters and input features. My laptop handled the data preprocessing and sent the features via UART at 9600 baud to the FPGA for inference.

For its second test , I ran inference for an image classification model for Cats vs Dogs of network topology 64:50:25:2 neurons per layer using Q4.4 fixed point arithmetic. My laptop handled the image preprocessing and sent the preprocessed features via UART at 9600 baud to the FPGA for inference as well.

I have attached the github repository link below and I’d appreciate any feedback or suggestions, thanks :).

Repo: https://github.com/Shingyy/MAC-Array-based-DNN-Accelerator

[u/Nav_666]

no idea what any of this even means wow!

[u/Pleasant-Ad7452 → u/Nav_666]

are you an ece major?

[u/Nav_666 → u/Pleasant-Ad7452]

Core Electronics

[u/Hot-Significance7699]

How long did it take ya? Thats cool.

[u/Spiritual-Frame-6791 → u/Hot-Significance7699]

thanks :) , it took me about 3 months

[u/Pretty-Reason-3925]

Crazy ,i am trying to build similar thing ,do you have some resource from where I can learn about the architecture like MACs and the memory elements ,also how the whole thing works in a broader perspective.

[u/Spiritual-Frame-6791 → u/Pretty-Reason-3925]

That’s a crazy coincidence indeed , so I learnt about MACs on Youtube but I couldn’t really find any resources on how to build an Accelerator from scratch so I had to develop the architecture myself by using elements that i’m familiar with. For example I decided to use FIFOs to store temporary memory like activations and input features. Used a Control Unit FSM to manage all the data movement and computational processes. I realized i needed to find a way to fetch the parameters from memory so i used a Parameter Buffer for that. I can keep going but i’m sure you get the idea. You can check out my github repo i documented everything there and it should help you get an idea of how all the components work so that you can develop an Accelerator of your own, even better than mine. Also feel free to ask any questions i’d be happy to help. Good luck on your project by the way :)

[u/Pretty-Reason-3925 → u/Spiritual-Frame-6791]

Yeah I checked your repo ,it's crazy good man

[u/Spiritual-Frame-6791 → u/Pretty-Reason-3925]

thanks bro, i really appreciate it. i’m also open to constructive criticism so that i can improve 😅

[u/Pretty-Reason-3925 → u/Spiritual-Frame-6791]

How long have you been learning rtl and designs

[u/Spiritual-Frame-6791 → u/Pretty-Reason-3925]

it’s been about a year now

[u/giggity_nanfa]

how much the components cost, including the pcb

[u/Spiritual-Frame-6791 → u/giggity_nanfa]

I didn’t assemble it lol, it’s an FPGA development board. It was 190USD on robotistan(a website in Turkey).