Share your repls and programming experiences

← Back to all posts
Machine learning neural network
finlay44111 (39)

A machine learning neural network i have been working on

Commentshotnewtop
mewant_taco (67)

Dude, very cool!
What is it doing exactly tho?

finlay44111 (39)

@mewant_taco the data it is training on is the first 1000 images in the MNIST dataset, it is then being tested in the next 100 images after that. the images are hand written digits. and it is learning to recognise the digits.

LiamDonohue (286)

would you mind if I recreated this in C#?

finlay44111 (39)

@LiamDonohue sure, i would recomend you watch 3blue1brown's series, https://www.youtube.com/watch?v=aircAruvnKk&list=PLZHQObOWTQDNU6R1_67000Dx_ZCJB-3pi which shows you the basics and bits of the underlying mathematics surounding it.

finlay44111 (39)

@LiamDonohue also i'd like to add the multithreading doesn't seem to work, or well it does, but it runs about 10 time slower than a single thread when use it on my laptop with 8 threads.

LiamDonohue (286)

this is cool

the brain

generationXcode (255)

Hi, I couldn't go through all the code but did you just train the NN using a genetic algorithm?wow

finlay44111 (39)

@generationXcode i used backpropagation, it works by basically finding the derivative of the network at a certain point.

generationXcode (255)

@finlay44111 ohh backprop... I thought u used genetic algorithms

Andi_Chin (221)

nice machine learning big data science neural network evolution program!

ChezCoder (1504)

I really wanted to make one of these in python but i kinda just gave up because of the complexity

finlay44111 (39)

@ChezCoder i would recommend 3blue1brown's series on it, if you have decent knowledge of high school/college math then you can probably understand and implement the machine learning algorithm used. also python is not a good language, since it is very inefficient.

3blue1brown's video: https://www.youtube.com/watch?v=aircAruvnKk&list=PLZHQObOWTQDNU6R1_67000Dx_ZCJB-3pi

generationXcode (255)

@finlay44111 # python isn't good for ML???
what??!!! no! NO!NONONO! Maybe C is faster, but thats because its compiled... Python is better for machine learning because its data handling capacity is great, its faster and easier to write ML programs because of libraries that have been built for it and its simple to use. It can work on any platform easily. I would go on to say that python is the best for ML. Its also gaining lots of popularity so more tools will be coming up as well for ML. Also there will be more help online if you're facing issues on anything regarding ML with python. Yes, eventually you will have to load your model onto C but dont make the mistake of making the model and experimenting with it on C...
so my advice here is Use C only in production, but build your models in python
But anyways I must admit your project is awesome

Also if you want performance, use Julia programming language

finlay44111 (39)

@generationXcode ahh k, so if you want to use libraries (that are written in c or c++) then python is for you. But if you want to do it all from scratch like I did then you need to be using c or c++, you can't write all of the logic in python, it's just too slow.

generationXcode (255)

@finlay44111 not really... The running of the program might be slower than C, but that doesn't mean it isn't fast... I mean for self drive level ML programs you need the speed of C... Writing a neural net from scratch in python is much easier than in C. What do you mean python loic is slow? And I think you may be referring Tensorflow library when you say its written in C, but in reality there are many more machine learning libraries in python that aren't available in C. What I'm saying is that its better to build the model in python and run it in C , if you really want lots of speed...
What do you mean by logic? the math or the AND, OR kind of statements?

Its also suggested by the experts in this field to use python... I mean why doesn't kaggle have a section to write C kernels? Obviously because python is the best language to develop your models in. In C if you haven't properly made your model structure and you want to make some changes to it so that (for example) it stops overfitting on the data, you will have to recompile all your code and wait till it changes to machine code. I mean its much easier to do all of that in python, right? But I agree that for production you should load your model on to C for greater speed. But if you want to develop a model just like that and don't want to reduce its loss then I don't really know...

And again python isn't that slow... My first CNN trained in like a minute and had a really nice accuracy(and it was in python).

finlay44111 (39)

@generationXcode it is suggested by the experts to use python, since libraries like tensorflow are available (which is written in c++ and CUDA), if you want to write all of the logic from yourself using basic mathematical functions, in order to actually gain an understanding of what is happening, then c++ is the only way to get the performance required to train these programs.

If you only want to use the APIs these libraries give you and claim that you have build a neural network, sure, go code it all in python and import tensorflow. but if you want to see how it works, in order to be able to expand on it, you need to use a lower level, more efficient language.

I had a look at your repl, https://repl.it/@generationXcode/negative, and it seems like you are just importing tensorflow, a c++ library to do all of the logic, and you are just using the API. this is probably why it didn't take too long to train.

generationXcode (255)

@finlay44111 I don't just use API to build neural nets and even with them, the neural nets formed do exactly what yours does, there is no difference in the math.
And how many times do I have to say that python isn't that slow, its quite fast.Maybe its not as fast as C but that doesn't mean its slow.
Its also very wise to write a NN from scratch in python because the language is much more simple. The math needed can be easily expressed unlike in C.
And I don't really do ML in repl.it. I have notebooks in which I do them(they are TPU accelerated in the cloud)

And please dont forget to read this point - you should develop your models in python and run them in C If you want to know why please read through all of my comments here.
Also, you built your model from scratch, yes, but you also used a library for vectors.
And another thing - I've seen the python tensorflow library and even contributed to it - its written in python.

generationXcode (255)

@finlay44111 This will take forever and ever to be honest, why dont we ask an expert in this field like francois chollet, jeremy howard, rachel thomas?

generationXcode (255)

@finlay44111 I meant the keras functional api within tensorflow.Its written in python. If you don't beleive me, check the github repo for tensorflow for python.
I beleive TF 2.0 is mainly keras now.

finlay44111 (39)

@generationXcode yes that is exactly right the API is written in python. all of the logic for the ai and machine language is in c++.

Thecrowbar1234 (146)

pretty cool. might try to make this in python.

finlay44111 (39)

@Thecrowbar1234 i wouldn't recommend python, this requires a lot of computing power and python is not efficient enough to run it well.

generationXcode (255)

@finlay44111 training the MNIST dataset doesnt require that much computing power though...

finlay44111 (39)

@generationXcode nope, but with the limited power that you get in repl, you can train it pretty fast if you use an efficient language.

generationXcode (255)

@finlay44111 But you have to wait till its compiled...

finlay44111 (39)

@generationXcode which is exacly the reason its faster than python, it takes about a second to compile, and then it is already in machine instructions. with python it is interpreted on the go, so it has to expend extra cpu cycles in translating from the python into the machine language.

which means that with c++ you only go over each line once to generate machine instructions, but with python you will have to interpret each individual instruction however many times that instruction is run.

generationXcode (255)

@finlay44111 Ok I agree with you on this one, you're right - machine code runs faster