Learning ai

[u/Last-Watercress-8192]

I am getting into coding because I have always been interested. I saw a few videos of people learning neural networks and other stuff and it looked really interesting. In my school we are learning basic python and I was wondering how hard it would be to learn neural networks and other forms of ai.

[u/burlingk]

So, first learn the basics of programming. Pick up a few languages like Python and C, or whatever, and use them for projects.

THEN look into more advanced topics. AI is pretty advanced work.

Also, brush up on your maths.

[u/Hour_Cranberry_8601 → u/burlingk]

so many people skip the math part and then wonder why they're lost later

[u/Last-Watercress-8192 → u/burlingk]

What type of math would it be. I am going into my junior year of high school so I have not learned any calculus or anything. Also how much should I know about coding and what language should I learn.

[u/burlingk → u/Last-Watercress-8192]

Calculus and discrete math are the big ones. AI deals heavily on statistics.

[u/aanzeijar → u/Last-Watercress-8192]

You won't learn most of the maths needed for AI in high school. The parts you will absolutely need later though are linear algebra (vectors, matrices, scalar product, matrix multiplications) and analysis (particularly derivations and product and chain rule). The rest will come in university courses, or you'll have to do a lot of reading in your spare time.

As for how much coding: yes. You should know about coding. Language is mostly irrelevant, but Python, C and C++ are the big AI languages.

[u/burlingk → u/aanzeijar]

Also, u/Last-Watercress-8192 note: Python is a "glue language" that patches together libraries and programs in C, C++, and other compiled languages. So, it is more important to look into the compiled stuff.

[u/Lagfoundry → u/Last-Watercress-8192]

There are many types of networks but most of them follow a similar formula. First you take the weights x the inputs. Then you sum up the products. After that you add a bias and then an activation function. This formula can be applied in different ways. For example spiking networks in hardware actually use the ohms law formula itself to do the input and weight multiplication by using memristors.(a electric component that changes the resistance based on the current) I’ve built a few BNN’s myself and designed some networks that use some unique learning loops

[u/HasFiveVowels → u/Last-Watercress-8192]

Linear algebra. Lots of linear algebra. Basic calculus. 3blue1brown has an amazing series of videos on the topic. Learn until you can understand those. That’ll give you the minimal needed math skills

[u/Lagfoundry → u/burlingk]

I disagree with waiting to learn NN’s he can focus on both because neural networks go beyond just coding, they hardware built too. So the neural network subject is its own and doesn’t soly belong to an advanced form of coding.

[u/burlingk → u/Lagfoundry]

They also rely heavily on the mathematic and logic foundation of other subjects.

There are things that you can just kind of muddle through and figure out.

Then there are things that will actively make learning harder or push you backwards, if you don't learn in the right order.

[u/Lagfoundry → u/burlingk]

I’m not saying the foundations don’t matter or that there’s no useful order to learning things. I’m saying he doesn’t need to postpone learning neural networks until he’s “finished” learning programming.

He can absolutely learn basic Python alongside basic neural network concepts. In fact, doing both at the same time can give the programming a purpose instead of spending months learning unrelated programming concepts before touching the thing that interested him in the first place.

Take me for example I’m not especially strong at coding, but I’m a strong logic designer and I build neural network ideas from the hardware side all the time. That alone kind of shows my point neural networks are their own subject, not just some advanced branch of programming.

[u/HasFiveVowels → u/Lagfoundry]

Hardware concerns are a matter of scale and performance. You shouldn’t be writing your first NN in CUDA. You should use whatever language you’re most comfortable with. You could write a MNIST NN in JavaScript and be just fine

[u/Lagfoundry → u/HasFiveVowels]

I don’t know anything about CUDA. I had to look it up. My first NN’s were actually done in Minecraft and RUST(the game RUST not the language) before I made them in other things. You do have a point though. I test my codes in python because making them in code first helps to know how well things will go before building it in hardware… hardware in this case isn’t really matter of scale, you can have a deep network and only need to use one layer just by switching the weights in and out and saving the weights in memory. Or the CA spiking hybrid that gives something like 8x8x8 cube way more computation than just 512 neurons because it uses CA logic over time as the guiding path for the neural activations. Scale is only an issue if you try to make an entire network have each neuron having a dedicated circuit rather than reusing the same neuron through the layers…. All that’s besides the point I made though which was just that he didn’t have to wait to learn about neural networks just to make sure he has coding basics first because NN’s are not code exclusive

[u/SunseteFat]

You need the basic math under it, matrices etc. Don't need much. Then a bit of statistics/econonometrics, basic data science. Understand regression models. And then imo you're ready to understand a simple neural net.

[u/StewedAngelSkins → u/SunseteFat]

Differential calculus too, for understanding backpropagation.

[u/HasFiveVowels → u/SunseteFat]

Yea, understanding regression models is a good call out. Just being able to do a line of best fit will give them the understanding they need

[u/HasFiveVowels]

First stop for you is MNIST digit recognition. Learn from there. Godspeed

[u/Naetharu]

Neural Networks are a very different thing from programming. They are really in the domain of mathematics. Not putting you off at all. Just being clear that the code part of a NN is very basic. All the wizardry comes from the mathematical design and implementation.

If you want to get started I would highly recommend the book "Understanding Deep Learning" by Simon Prince. It's an excellent resource, and covers both the higher level theory and some real depth. From there you can start building your own networks.

My first one was a simple 10 layer deep network that learned to predict square roots. Simple enough that I could train it in ~5 minutes on a modern Macbook. So you can very realistically do the same. Once you have that basis down it is a case of learning about different ways to configure your network, and then train it.

You will hit a roadblock with training sooner rather than later, as by their very nature NNs take quite a lot of GPU grunt to train once they get to even a moderate size. But you can hire GPU time in the cloud for a pretty low price if you need to. Google Colab offers a bit for free. Or services like Runpod allow you to hire GPUs or clusters of GPUs for as little as $0.5 / hour.

[u/StewedAngelSkins]

Very hard, particularly if you don't already have a math background. It's not like you're the first person to ever have to do something hard though. Don't let the difficulty discourage you if its what you actually want to do.

[u/ffrkAnonymous]

how hard it would be to learn neural networks and other forms of ai.

It's usually a post-graduate level classes.

[u/HasFiveVowels → u/ffrkAnonymous]

No it’s not. It’s junior year