Python basics. How to create artificial intelligence? How to create artificial intelligence in python
This week you could read an extremely motivating case from a GeekBrains student who studied the profession, where he talked about one of his goals that led to the profession - the desire to learn the principle of work and learn how to create game bots yourself.
Indeed, it was the desire to create perfect artificial intelligence, whether it be a game model or a mobile program, that prompted many of us to the path of a programmer. The problem is that, behind tons of educational material and the harsh reality of customers, this very desire has been replaced by a simple desire for self-development. For those who have not yet begun to fulfill their childhood dreams, here is a short guide to creating a real artificial intelligence.
Stage 1. Disappointment
When we talk about creating at least simple bots, the eyes are filled with sparkle, and hundreds of ideas flash in his head about what he should be able to do. However, when it comes to implementation, it turns out that mathematics is the key to unraveling the actual behavior. Yes, artificial intelligence is much more difficult than writing application programs - knowledge of software design alone will not be enough for you.
Mathematics is the scientific springboard on which your further programming will be built. Without knowledge and understanding of this theory, all ideas will quickly break up about interaction with a person, because an artificial intelligence is really nothing more than a set of formulas.
Stage 2. Acceptance
When the arrogance is a little knocked down by student literature, you can start practice. It's not worth rushing to LISP or others yet - first you should get comfortable with the principles of AI design. Python is perfect for both quick learning and further development - this is the language most often used for scientific purposes, for it you will find many libraries that will make your work easier.
Stage 3. Development
Now we turn directly to the theory of AI. They can be roughly divided into 3 categories:
- Weak AI - bots that we see in computer games, or simple helpers like Siri. They either perform highly specialized tasks or are an insignificant complex of them, and any unpredictability of interaction baffles them.
- Strong AIs are machines whose intelligence is comparable to the human brain. Today, there are no real representatives of this class, but computers like Watson are very close to achieving this goal.
- Perfect AI is the future, a machine brain that will surpass our capabilities. It is about the dangers of such developments that Stephen Hawking, Elon Musk and the Terminator film franchise warn.
Naturally, you should start with the simplest bots. To do this, remember the good old tic-tac-toe game when using the 3x3 field and try to figure out the basic algorithms for yourself: the probability of winning with error-free actions, the most successful places on the field for placing a piece, the need to reduce the game to a draw, and so on.
Dozens of games and analyzing your own actions, you can probably highlight all the important aspects and rewrite them into machine code. If not, keep thinking, and this link is here just in case.
By the way, if you still took up the Python language, then you can create a fairly simple bot by referring to this detailed manual. For other languages, such as C ++ or Java, you won't have any trouble finding step-by-step materials either. Feeling that there is nothing supernatural behind the creation of AI, you can safely close the browser and start personal experiments.
Stage 4. Excitement
Now that things have gotten off the ground, you probably want to create something more serious. A number of the following resources will help you with this:
As you understand even from the names, these are APIs that will allow you to create some semblance of serious AI without wasting time.
Stage 5. Work
Now, when you already quite clearly understand how to create AI and what to use at the same time, it's time to take your knowledge to a new level. Firstly, this will require a study of the discipline, which is called "Machine Learning". Secondly, you need to learn how to work with the appropriate libraries of the selected programming language. For the Python we are considering, these are Scikit-learn, NLTK, SciPy, PyBrain, and Numpy. Thirdly, development is indispensable. And most importantly, you will now be able to read AI literature with full understanding of the matter:
- Artificial Intelligence for Games, Ian Millington;
- Game Programming Patterns, Robert Nystorm;
- AI Algorithms, Data Structures, and Idioms in Prolog, Lisp, and Java, George Luger, William Stbalfield;
- Computational Cognitive Neuroscience, Randall O'Reilly, Yuko Munakata;
- Artificial Intelligence: A Modern Approach, Stuart Russell, Peter Norvig.
And yes, all or almost all of the literature on this topic is presented in a foreign language, so if you want to create AI professionally, you need to improve your English to a technical level. However, this is relevant for any area of \u200b\u200bprogramming, isn't it?
Neural networks are created and taught mainly in Python. Therefore, it is very important to have a basic understanding of how to write programs in it. In this article, I will briefly and clearly describe the basic concepts of this language: variables, functions, classes and modules.
The material is intended for people who are not familiar with programming languages.
First you need to install Python. Then you need to put in a comfortable environment for writing Python programs. The portal is dedicated to these two steps.
If everything is installed and configured, you can start.
Variables
Variable - a key concept in any programming language (and not only in them). The easiest way to think of a variable is a box with a label. This box contains something (number, matrix, object, ...) that is valuable to us.
For example, we want to create a variable x that should store the value 10. In Python, the code for creating this variable would look like this:
On the left we declarea variable named x. It's like putting a name tag on the box. Next comes the equal sign and the number 10. The equal sign plays an unusual role here. It does not mean that "x is 10". Equality in this case puts the number 10 in the box. More correctly, we assign variable x number 10.
Now, in the code below, we can access this variable, and also perform various actions with it.
You can simply display the value of this variable on the screen:
X \u003d 10 print (x)
Print (x) is a function call. We will consider them further. What is important now is that this function prints to the console what is between the brackets. We have x between the brackets. Earlier we set x to 10. It is 10 that is printed to the console if you run the program above.
With variables that store numbers, you can perform various simple actions: add, subtract, multiply, divide and raise to a power.
X \u003d 2 y \u003d 3 # Addition z \u003d x + y print (z) # 5 # Difference z \u003d x - y print (z) # -1 # Product z \u003d x * y print (z) # 6 # Division z \u003d x / y print (z) # 0.66666 ... # Exponentiation z \u003d x ** y print (z) # 8
In the above code, we first create two variables containing 2 and 3. Then we create a variable z that stores the result of the operation with x and y and prints the results to the console. This example clearly shows that a variable can change its value during program execution. So, our variable z changes its value as much as 5 times.
Functions
Sometimes it becomes necessary to perform the same actions many times. For example, in our project, you often need to display 5 lines of text.
"This is a very important text!"
"This text can not be read"
"The error in the top line was made on purpose"
"Hello and goodbye"
"The end"
Our code will look like this:
X \u003d 10 y \u003d x + 8 - 2 print ("This is a very important text!") Print ("This text cannot be read") print ("The error in the top line was made on purpose") print ("Hello, bye") print ("End") z \u003d x + y print ("This is a very important text!") Print ("This text cannot be read") print ("The error in the top line was made on purpose") print ("Hello, bye") print ("End") test \u003d z print ("This is a very important text!") Print ("This text cannot be read") print ("The error in the top line was made on purpose") print ("Hello, bye") print (" The end")
It all looks very redundant and inconvenient. Besides, there was a mistake in the second line. It can be fixed, but it will have to be fixed in three places at once. And if in our project these five lines are called 1000 times? And all in different places and files?
Functions can be created in programming languages \u200b\u200bespecially for cases where you need to frequently execute the same commands.
Function - a separate block of code that can be called by name.
The function is specified using the def keyword. This is followed by the name of the function, then parentheses and a colon. Next, indented, you need to list the actions that will be performed when the function is called.
Def print_5_lines (): print ("This is a very important text!") Print ("This text cannot be read") print ("The error in the top line was made on purpose") print ("Hello and bye") print ("End")
We have now defined the print_5_lines () function. Now, if in our project we once again need to insert five lines, then we simply call our function. It will automatically perform all actions.
# Define the function def print_5_lines (): print ("This is a very important text!") Print ("This text cannot be read") print ("The error in the top line was made on purpose") print ("Hello, bye") print (" End ") # Our project code x \u003d 10 y \u003d x + 8 - 2 print_5_lines () z \u003d x + y print_5_lines () test \u003d z print_5_lines ()
Convenient, isn't it? We've seriously improved the readability of the code. In addition, functions are also good in that if you want to change some of the actions, then it is enough to correct the function itself. This change will work in all places where your function is called. That is, we can correct the error in the second line of the output text ("nilja"\u003e "not allowed") in the function body. The correct version will be automatically called in all places of our project.
Functions with parameters
It's convenient to just repeat a few actions. But that is not all. Sometimes we want to pass some kind of variable to our function. Thus, the function can receive data and use it during the execution of commands.
The variables we pass to the function are called arguments.
Let's write a simple function that adds two numbers given to it and returns the result.
Def sum (a, b): result \u003d a + b return result
The first line looks almost the same as regular functions. But there are now two variables between the brackets. it options functions. Our function has two parameters (i.e. it takes two variables).
Parameters can be used inside a function like regular variables. On the second line, we create the result variable, which is the sum of the parameters a and b. On the third line, we return the value of the result variable.
Now, in the further code, we can write something like:
New \u003d sum (2, 3) print (new)
We call the sum function and pass it two arguments in turn: 2 and 3. 2 becomes the value of a, and 3 becomes the value of b. Our function returns a value (the sum of 2 and 3), and we use that to create a new variable, new.
Remember. In the code above, the numbers 2 and 3 are the arguments to the sum function. And in the function sum itself, the variables a and b are parameters. In other words, the variables we pass to the function when it is called are called arguments. But inside the function, these passed variables are called parameters. In fact, these are two names for the same thing, but they should not be confused.
Let's take another example. Let's create a function square (a) that takes a single number and squares it:
Def square (a): return a * a
Our function consists of just one line. It immediately returns the result of multiplying the parameter a by a.
I think you have already guessed that we also output data to the console using a function. This function is called print () and it prints the argument passed to it to the console: number, string, variable.
Arrays
If a variable can be thought of as a box that stores something (not necessarily a number), then arrays can be thought of as bookshelves. They contain several variables at once. Here's an example of an array of three numbers and one string:
Array \u003d
Here's an example, when a variable does not contain a number, but to some other object. In this case, our variable contains an array. Each element of the array is numbered. Let's try to display some element of the array:
Array \u003d print (array)
You will see 89 in the console. But why 89 and not 1? The thing is that in Python, as in many other programming languages, array numbering starts at 0. Therefore, array gives us secondthe element of the array, not the first. To call the first one, you had to write array.
Array size
Sometimes it is very useful to get the number of elements in an array. You can use the len () function for this. It will count the number of elements itself and return their number.
Array \u003d print (len (array))
The console will display the number 4.
Conditions and cycles
By default, any programs simply execute all commands in a row from top to bottom. But there are situations when we need to check some condition, and depending on whether it is true or not, perform different actions.
In addition, it is often necessary to repeat almost the same sequence of commands many times.
In the first situation, conditions help, and in the second, cycles.
Conditions
Conditions are needed in order to perform two different sets of actions, depending on whether the statement being tested is true or false.
In Python, conditions can be written using the if: ... else: ... construct. Suppose we have some variable x \u003d 10. If x is less than 10, then we want to divide x by 2. If x is greater than or equal to 10, then we want to create another variable new, which is equal to the sum of x and the number 100. This is how the code will look like:
X \u003d 10 if (x< 10): x = x / 2 print(x) else: new = x + 100 print(new)
After creating the variable x, we start writing our condition.
It all starts with the if keyword (translated from English "if"). In parentheses, we indicate the expression to be tested. In this case, we check if our variable x is really less than 10. If it is really less than 10, then we divide it by 2 and print the result to the console.
Then comes the else keyword, followed by a block of actions that will be executed if the expression in parentheses after the if is false.
If it is greater than or equal to 10, then we create a new variable new, which is equal to x + 100 and also output it to the console.
Cycles
Loops are needed for repeated repetition of actions. Suppose we want to display a table of squares of the first 10 natural numbers. It can be done like this.
Print ("Square 1 equals" + str (1 ** 2)) print ("Square 2 equals" + str (2 ** 2)) print ("Square 3 equals" + str (3 ** 2)) print ( "Square 4 is equal to" + str (4 ** 2)) print ("Square 5 is equal to" + str (5 ** 2)) print ("Square 6 is equal to" + str (6 ** 2)) print ("Square 7 equals "+ str (7 ** 2)) print (" Square 8 equals "+ str (8 ** 2)) print (" Square 9 equals "+ str (9 ** 2)) print (" Square 10 equals "+ str (10 ** 2))
Don't be surprised by the fact that we are adding strings. "Start of line" + "end" in Python means simply concatenation of strings: "start of line end". Similarly, above, we add the string "The square of x is equal" and the result of raising the number to the 2nd power, converted using the function str (x ** 2).
The code above looks very redundant. What if we need to print the squares of the first 100 numbers? We'll be tortured to output ...
It is for such cases that cycles exist. There are 2 types of loops in Python: while and for. Let's deal with them in turn.
The while loop repeats the required commands as long as the condition remains true.
X \u003d 1 while x<= 100: print("Квадрат числа " + str(x) + " равен " + str(x**2)) x = x + 1
First we create a variable and assign it the number 1. Then we create a while loop and check if our x is less than (or equal to) 100. If less (or equal) then we perform two actions:
- Draw the square x
- Increase x by 1
After the second command, the program returns to the condition. If the condition is true again, then we perform these two actions again. And so on until x becomes equal to 101. Then the condition will return false and the loop will no longer be executed.
The for loop is designed to iterate over arrays. Let's write the same example with the squares of the first hundred natural numbers, but through the for loop.
For x in range (1,101): print ("Square of number" + str (x) + "is" + str (x ** 2))
Let's analyze the first line. We use the for keyword to create a loop. Next, we indicate that we want to repeat certain actions for all x in the range from 1 to 100. The range (1,101) function creates an array of 100 numbers, starting at 1 and ending at 100.
Here's another example of iterating over an array using a for loop:
For i in: print (i * 2)
The code above displays 4 numbers: 2, 20, 200 and 2000. Here you can clearly see how it takes each element of the array and performs a set of actions. Then it takes the next item and repeats the same set of actions. And so on until the elements in the array run out.
Classes and objects
In real life, we do not operate on variables or functions, but on objects. Pen, machine, man, cat, dog, plane - objects. Now let's start examining the cat in detail.
It has some parameters. These include coat color, eye color, and her nickname. But that is not all. In addition to parameters, the cat can perform various actions: purr, hiss and scratch.
We have just described schematically all cats in general. Similar description of properties and actions some object (for example, a cat) in Python and is called a class. A class is simply a collection of variables and functions that describe an object.
It is important to understand the difference between a class and an object. Class - schemathat describes the object. The object is her material embodiment... Cat class - a description of its properties and actions. The object of the cat is the real cat itself. There can be many different real cats - many cat objects. But there is only one cat class. The picture below is a good demonstration:
Classes
To create a class (schema of our cat), you need to write the class keyword and then specify the name of this class:
Class Cat:
Next, we need to list the actions of this class (actions of the cat). Actions, as you might have guessed, are functions defined within a class. It is customary to call such functions inside a class methods.
Method - a function defined inside a class.
Verbally, we have already described the cat's methods above: purr, hiss, scratch. Now let's do it in Python.
# Cat class class Cat: # Purr def purr (self): print ("Murrr!") # Hiss def hiss (self): print ("Shhhh!") # Scrabble def scrabble (self): print ("Scrabble ! ")
It's that simple! We took and defined three regular functions, but only inside the class.
In order to deal with the incomprehensible parameter self, let's add another method to our cat. This method will call all three already created methods at once.
# Cat class class Cat: # Purr def purr (self): print ("Murrr!") # Hiss def hiss (self): print ("Shhhh!") # Scrabble def scrabble (self): print ("Scrabble ! ") # Put all together def all_in_one (self): self.purr () self.hiss () self.scrabble ()
As you can see, the self parameter, which is required for any method, allows us to access the methods and variables of the class itself! Without this argument, we would not be able to perform such actions.
Let's now set the properties of our cat (fur color, eye color, nickname). How to do it? In absolutely any class, you can define the __init __ () function. This function is always called when we create a real object of our class.
In the __init __ () method highlighted above, we set the variables of our cat. How do we do it? First, we pass 3 arguments to this method, which are responsible for the coat color, eye color and nickname. Then, we use the self parameter in order to immediately set the 3 attributes described above to our cat when creating an object.
What does this line mean?
Self.wool_color \u003d wool_color
On the left side, we create an attribute for our cat named wool_color, and then we assign this attribute to the value that is contained in the wool_color parameter that we passed to the __init __ () function. As you can see, the line above is no different from the usual creation of a variable. Only the self prefix indicates that this variable belongs to the Cat class.
Attribute- a variable that belongs to a class.
So, we have created a finished cat class. Here is its code:
# Cat class class Cat: # Actions to be performed when creating a "Cat" object def __init __ (self, wool_color, eyes_color, name): self.wool_color \u003d wool_color self.eyes_color \u003d eyes_color self.name \u003d name # Purr def purr ( self): print ("Murrr!") # Hiss def hiss (self): print ("Shhhh!") # Scrabble def scrabble (self): print ("Scrabble!") # Together def all_in_one (self) : self.purr () self.hiss () self.scrabble ()
Objects
We have created a cat outline. Now let's create a real cat object using this scheme:
My_cat \u003d Cat ("black", "green", "Zosia")
In the line above, we create the variable my_cat and then assign the Cat object to it. It all looks like a call to some Cat (...) function. In fact, it is. With this entry, we call the __init __ () method of the Cat class. The __init __ () function in our class takes 4 arguments: the self class object itself, which does not need to be specified, as well as 3 more different arguments, which then become the attributes of our cat.
So with the line above we have created a real cat object. Our cat has the following attributes: black fur, green eyes and the nickname Zosia. Let's print these attributes to the console:
Print (my_cat.wool_color) print (my_cat.eyes_color) print (my_cat.name)
That is, we can refer to the attributes of an object by writing down the name of the object, putting a period and specifying the name of the desired attribute.
The cat's attributes can be changed. For example, let's change the name of our cat:
My_cat.name \u003d "Nyusha"
Now, if you display the cat's name in the console again, you will see Nyusha instead of Zosia.
Let me remind you that our cat's class allows her to perform some actions. If we stroke our Zosia / Nyusha, she will begin to purr:
My_cat.purr ()
Executing this command will output the text "Murrr!" To the console. As you can see, accessing an object's methods is as easy as accessing its attributes.
Modules
Any file with a .py extension is a module. Even the one in which you practice this article. What are they needed for? For comfort. A lot of people create files with useful functions and classes. Other programmers connect these third-party modules and can use all the functions and classes defined in them, thereby simplifying their work.
For example, you don't need to waste time writing your own matrix functions. It is enough to connect the numpy module and use its functions and classes.
To date, other Python programmers have written over 110,000 different modules. The numpy module mentioned above allows you to quickly and conveniently work with matrices and multidimensional arrays. The math module provides many methods for working with numbers: sines, cosines, converting degrees to radians, and more and more ...
Installing the module
Python is installed along with a standard set of modules. This set includes a very large number of modules that allow you to work with mathematics, web queries, read and write files and perform other necessary actions.
If you want to use a module that is not included in the standard kit, then you need to install it. To install the module, open the command line (Win + R, then enter “cmd” in the appeared field) and enter the command into it:
Pip install [module_name]
The installation process of the module will begin. When it finishes, you can safely use the installed module in your program.
Connecting and using the module
The third-party module is very easy to connect. You just need to write one short line of code:
Import [module_name]
For example, to import a module that allows you to work with mathematical functions, you need to write the following:
Import math
How do I refer to a module function? You need to write the name of the module, then put a full stop and write the name of the function / class. For example, factorial 10 is found like this:
Math.factorial (10)
That is, we turned to the factorial (a) function, which is defined inside the math module. This is convenient, because we do not need to waste time and manually create a function that calculates the factorial of a number. You can connect the module and immediately perform the required action.
This time, I decided to study neural networks. I was able to get basic skills in this matter over the summer and fall of 2015. By basic skills, I mean I can build a simple neural network myself from scratch. You can find examples in my repositories on GitHub. In this article, I will provide some clarification and share resources that you may find useful to explore.
Step 1. Neurons and the feedforward method
So what is a "neural network"? Let's wait with that and deal with one neuron first.
A neuron is like a function: it takes several values \u200b\u200bas input and returns one.
The circle below represents an artificial neuron. It gets 5 and returns 1. Input is the sum of the three synapses connected to the neuron (three arrows on the left).
On the left side of the image we see 2 input values \u200b\u200b(green) and an offset (highlighted in brown).
The input data can be numerical representations of two different properties. For example, when creating a spam filter, they could mean the presence of more than one word in CAPITAL LETTERS and the presence of the word "viagra".
The input values \u200b\u200bare multiplied by their so-called "weights", 7 and 3 (highlighted in blue).
Now we add the resulting values \u200b\u200bwith the offset and get a number, in our case 5 (highlighted in red). This is the input of our artificial neuron.
Then the neuron performs some kind of calculation and outputs an output value. We got 1 because the rounded sigmoid value at point 5 is 1 (more on this function later).
If it were a spam filter, the fact of output 1 would mean that the text was marked as spam by the neuron.
Neural network illustration from Wikipedia.
If you combine these neurons, you get a straight-forward neural network - the process goes from input to output, through neurons connected by synapses, as in the picture on the left.
Step 2. Sigmoid
After you've watched the Welch Labs tutorials, it's a good idea to check out the fourth week of Coursera's Machine Learning Neural Networks course to help you understand how they work. The course is very deep in math and is based on Octave, and I prefer Python. Because of this, I skipped the exercises and learned all the necessary knowledge from the video.
The sigmoid simply maps your value (along the horizontal axis) to a range from 0 to 1.
The first priority for me was to study the sigmoid, as it figured in many aspects of neural networks. I already knew something about her from the third week of the above course, so I reviewed the video from there.
But videos alone won't get you very far. For a complete understanding, I decided to code it myself. So I started writing an implementation of a logistic regression algorithm (which uses a sigmoid).
It took a whole day and the result was hardly satisfactory. But it doesn't matter, because I figured out how everything works. You can see the code.
You don't have to do it yourself, as it requires special knowledge - the main thing is that you understand how the sigmoid works.
Step 3. Backpropagation method
It is not so difficult to understand how a neural network works from input to output. It is much more difficult to understand how a neural network is trained on datasets. The principle I used is called
James Loy, Georgia Tech. A beginner's guide, after which you can create your own neural network in Python.
Motivation:based on my personal experience in learning deep learning, I decided to create a neural network from scratch without a complex learning library such as. I believe that understanding the internal structure of a neural network is important for a beginner Data Scientist.
This article contains what I have learned and hopefully it will be useful for you too! Other helpful related articles:
What is a neural network?
Most articles on neural networks draw parallels with the brain when describing them. It's easier for me to describe neural networks as a mathematical function that maps a given input to a desired result without going into details.
Neural networks are composed of the following components:
- input layer, x
- arbitrary amount hidden layers
- output layer, ŷ
- set scales and displacements between each layer W and b
- choice activation functions for each hidden layer σ ; in this work we will use the activation function of Sigmoid
The diagram below shows the architecture of a two-layer neural network (note that the input layer is usually excluded when counting the number of layers in the neural network).
Creating a Neural Network class in Python looks simple:
Neural network training
Exit ŷ a simple two-layer neural network:
In the above equation, the weights W and bias b are the only variables that affect the output ŷ.
Naturally, the correct values \u200b\u200bfor the weights and biases determine the accuracy of the predictions. The process of fine-tuning weights and biases from input data is known as neural network training.
Each iteration of the training process consists of the following steps
- calculating the predicted output ŷ called forward propagation
- updating weights and biases called backpropagation
The sequential graph below illustrates the process:
Direct distribution
As we saw in the graph above, forward propagation is just an easy computation, and for a basic 2-layer neural network, the output of the neural network is given by:
Let's add feed forward to our Python code to do this. Note that for simplicity, we have assumed that the offsets are 0.
However, we need a way to assess the "goodness" of our forecasts, that is, how far our forecasts are). Loss function just allows us to do it.
Loss function
There are many loss functions available, and the nature of our problem should dictate our choice of loss function. In this work we will use sum of squares of errors as a loss function.
The sum of squared errors is the average of the difference between each predicted value and the actual value.
The goal of training is to find a set of weights and biases that minimizes the loss function.
Back propagation
Now that we have measured our forecast error (loss), we need to find a way propagating the error back and update our weights and biases.
To find out the appropriate amount to correct for the weights and biases, we need to know the derivative of the loss function with respect to the weights and biases.
Recall from the analysis that the derivative of the function is the slope of the function.
If we have a derivative, then we can simply update the weights and biases by increasing / decreasing them (see diagram above). It is called gradient descent.
However, we cannot directly calculate the derivative of the loss function with respect to the weights and biases, since the equation of the loss function does not contain weights and biases. Therefore, we need a chain rule to help us calculate.
Fuh! It was cumbersome, but it allowed us to get what we need - the derivative (slope) of the loss function with respect to the weights. We can now adjust the weights accordingly.
Let's add the backpropagation function to our Python code:
Checking the operation of the neural network
Now that we have our complete Python code for performing forward and backpropagation, let's take a look at our neural network by example and see how it works.
The perfect set of weights Our neural network needs to learn the ideal set of weights to represent this function.
Let's train a neural network for 1500 iterations and see what happens. Looking at the iteration loss graph below, we can clearly see that the loss monotonically decreases to a minimum. This is consistent with the gradient descent algorithm we talked about earlier.
Let's look at the final prediction (output) from the neural network after 1500 iterations.
We did it!Our forward and backward propagation algorithm has shown the neural network to work successfully, and the predictions converge on true values.
Note that there is little difference between the predictions and the actual values. This is desirable because it prevents overfitting and allows the neural network to better generalize invisible data.
Final reflections
I learned a lot in the process of writing my own neural network from scratch. While deep learning libraries such as TensorFlow and Keras allow deep networks to be built without fully understanding the inner workings of a neural network, I find it helpful for aspiring Data Scientists to gain a deeper understanding of them.
I have invested a lot of my personal time in this work, and I hope you find it useful!
