{"id":208602,"date":"2024-07-02T12:05:28","date_gmt":"2024-07-02T16:05:28","guid":{"rendered":"https:\/\/ibkrcampus.com\/?p=208602"},"modified":"2024-07-02T12:07:12","modified_gmt":"2024-07-02T16:07:12","slug":"forward-propagation-in-neural-networks-components-and-applications-part-ii","status":"publish","type":"post","link":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/forward-propagation-in-neural-networks-components-and-applications-part-ii\/","title":{"rendered":"Forward Propagation In Neural Networks: Components and Applications &#8211; Part II"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\"><em>Learn about the forward propagation algorithm with <a href=\"\/campus\/ibkr-quant-news\/forward-propagation-in-neural-networks-components-and-applications-part-i\/\">Part I<\/a>.<\/em><\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"forward-propagation-vs-backward-propagation-in-neural-network\">Forward propagation vs backward propagation in neural network<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Below is the table for a clear difference between forward and backward propagation in the neural network.<\/p>\n\n\n\n<figure class=\"wp-block-table\"><table><tbody><tr><td><strong>Aspect<\/strong><\/td><td><strong>Forward Propagation<\/strong><\/td><td><strong>Backward Propagation<\/strong><\/td><\/tr><tr><td><strong>Purpose<\/strong><\/td><td>Compute the output of the neural network given inputs<\/td><td>Adjust the weights of the network to minimise error<\/td><\/tr><tr><td><strong>Direction<\/strong><\/td><td>Forward from input to output<\/td><td>Backwards, from output to input<\/td><\/tr><tr><td><strong>Calculation<\/strong><\/td><td>Computes the output using current weights and biases<\/td><td>Updates weights and biases using calculated gradients<\/td><\/tr><tr><td><strong>Information flow<\/strong><\/td><td>Input data -&gt; Output data<\/td><td>Error signal -&gt; Gradient updates<\/td><\/tr><tr><td><strong>Steps<\/strong><\/td><td>1. Input data is fed into the network.2. Data is processed through hidden layers.3. Output is generated.<\/td><td>1. Error is calculated using a loss function.2. Gradients of the loss function are calculated.3. Weights and biases are updated using gradients.<\/td><\/tr><tr><td><strong>Used in<\/strong><\/td><td>Prediction and inference<\/td><td><br>Training the neural network<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">Next, let us see the forward propagation in different types of neural networks.<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\" \/>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"forward-propagation-in-different-types-of-neural-networks\">Forward propagation in different types of neural networks<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Forward propagation is a key process in various types of neural networks, each with its own architecture and specific steps involved in moving input data through the network to produce an output.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Forward propagation is a fundamental process in various types of neural networks, including:<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img decoding=\"async\" width=\"720\" height=\"755\" data-src=\"\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/Forward-propagation-in-different-types-of-NN-quantinsti.png\" alt=\"\" class=\"wp-image-208604 lazyload\" data-srcset=\"https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/Forward-propagation-in-different-types-of-NN-quantinsti.png 720w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/Forward-propagation-in-different-types-of-NN-quantinsti-700x734.png 700w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/Forward-propagation-in-different-types-of-NN-quantinsti-300x315.png 300w\" data-sizes=\"(max-width: 720px) 100vw, 720px\" src=\"data:image\/svg+xml;base64,PHN2ZyB3aWR0aD0iMSIgaGVpZ2h0PSIxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPjwvc3ZnPg==\" style=\"--smush-placeholder-width: 720px; aspect-ratio: 720\/755;\" \/><\/figure>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Feedforward Neural Networks (FNN):<\/strong>&nbsp;In FNNs, also known as Multi-layer Perceptrons (MLPs), forward propagation involves passing the input data through the network&#8217;s layers from the input layer to the output layer without any feedback loop.<\/li>\n\n\n\n<li><strong>Convolutional Neural Networks (CNN):&nbsp;<\/strong>In CNNs, forward propagation involves passing the input data through convolutional layers, pooling layers, and fully connected layers. Convolutional layers apply convolution operations to the input data, extracting features. Pooling layers reduce the spatial dimensions of the data. Fully connected layers perform the final classification.<\/li>\n\n\n\n<li><strong>Recurrent Neural Networks (RNN):<\/strong>&nbsp;In RNNs, forward propagation involves passing the input sequence through the network&#8217;s layers. RNNs have recurrent connections, allowing information to persist. Each step in the sequence feeds the output of the previous step back into the network.<\/li>\n\n\n\n<li><strong>Long Short-Term Memory Networks (LSTM):<\/strong>&nbsp;LSTM networks are a type of RNN designed to address the vanishing gradient problem. Forward propagation in LSTMs involves passing input sequences through gates that control the flow of information. These gates include input, forget, and output gates, which regulate the flow of information in and out of the cell.<\/li>\n\n\n\n<li><strong>Autoencoder Networks:&nbsp;<\/strong>In autoencoder networks, forward propagation involves encoding the input data into a lower-dimensional representation and then decoding it back to the original input space.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Moving forward, let us discuss the components of forward propagation.<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\" \/>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"components-of-forward-propagation\">Components of forward propagation<\/h2>\n\n\n\n<figure class=\"wp-block-image size-full\"><img decoding=\"async\" width=\"720\" height=\"271\" data-src=\"\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/Components-of-forward-propagation-quantinsti.png\" alt=\"\" class=\"wp-image-208605 lazyload\" data-srcset=\"https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/Components-of-forward-propagation-quantinsti.png 720w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/Components-of-forward-propagation-quantinsti-700x263.png 700w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/Components-of-forward-propagation-quantinsti-300x113.png 300w\" data-sizes=\"(max-width: 720px) 100vw, 720px\" src=\"data:image\/svg+xml;base64,PHN2ZyB3aWR0aD0iMSIgaGVpZ2h0PSIxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPjwvc3ZnPg==\" style=\"--smush-placeholder-width: 720px; aspect-ratio: 720\/271;\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">In the above diagram, we see a neural network consisting of three layers. The first and the third layer are straightforward, input and output layers. But what is this middle layer and why is it called the hidden layer?<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Now, in our example, we had just one equation, thus we have only one neuron in each layer.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Nevertheless, the hidden layer consists of two functions:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Pre-activation function: T<\/strong>he weighted sum of the inputs is calculated in this function.<\/li>\n\n\n\n<li><strong>Activation function:<\/strong>&nbsp;Here, based on the weighted sum, an activation function is applied to make the network non-linear and make it learn as the computation progresses. The activation function uses bias to make it non-linear.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Going forward, we must check out the applications of forward propagation to learn about the same in detail.<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\" \/>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"applications-of-forward-propagation\">Applications of forward propagation<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">In this example, we will be using a 3-layer network (with 2 input units, 2 hidden layer units, and 2 output units). The network and parameters (or weights) can be represented as follows.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img decoding=\"async\" width=\"720\" height=\"271\" data-src=\"\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/Components-of-forward-propagation-1-quantinsti.png\" alt=\"\" class=\"wp-image-208607 lazyload\" data-srcset=\"https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/Components-of-forward-propagation-1-quantinsti.png 720w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/Components-of-forward-propagation-1-quantinsti-700x263.png 700w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/Components-of-forward-propagation-1-quantinsti-300x113.png 300w\" data-sizes=\"(max-width: 720px) 100vw, 720px\" src=\"data:image\/svg+xml;base64,PHN2ZyB3aWR0aD0iMSIgaGVpZ2h0PSIxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPjwvc3ZnPg==\" style=\"--smush-placeholder-width: 720px; aspect-ratio: 720\/271;\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">Let us say that we want to train this neural network to predict whether the market will go up or down. For this, we assign two classes Class 0 and Class 1.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Here, Class 0 indicates the data point where the market closes down, and conversely, Class 1 indicates that the market closes up. To make this prediction, a train data(X) consisting of two features x1, and x2. Here x1 represents the correlation between the close prices and the 10-day simple moving average (SMA) of close prices, and x2 refers to the difference between the close price and the 10-day SMA.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">In the example below, the data point belongs to class 1. The mathematical representation of the input data is as follows:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><em>X = [x1, x2] = [0.85,.25] y= [1]<\/em><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Example with two data points:<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img decoding=\"async\" width=\"832\" height=\"154\" data-src=\"\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/forward-propagation-formula-data-points.png\" alt=\"\" class=\"wp-image-208609 lazyload\" data-srcset=\"https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/forward-propagation-formula-data-points.png 832w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/forward-propagation-formula-data-points-700x130.png 700w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/forward-propagation-formula-data-points-300x56.png 300w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/forward-propagation-formula-data-points-768x142.png 768w\" data-sizes=\"(max-width: 832px) 100vw, 832px\" src=\"data:image\/svg+xml;base64,PHN2ZyB3aWR0aD0iMSIgaGVpZ2h0PSIxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPjwvc3ZnPg==\" style=\"--smush-placeholder-width: 832px; aspect-ratio: 832\/154;\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">The output of the model is categorical or a discrete number. We need to convert this output data into a matrix form. This enables the model to predict the probability of a data point belonging to different classes. When we make this matrix conversion, the columns represent the classes to which that example belongs, and the rows represent each of the input examples.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img decoding=\"async\" width=\"850\" height=\"85\" data-src=\"\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/forward-propagation-formula-output.png\" alt=\"\" class=\"wp-image-208610 lazyload\" data-srcset=\"https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/forward-propagation-formula-output.png 850w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/forward-propagation-formula-output-700x70.png 700w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/forward-propagation-formula-output-300x30.png 300w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/forward-propagation-formula-output-768x77.png 768w\" data-sizes=\"(max-width: 850px) 100vw, 850px\" src=\"data:image\/svg+xml;base64,PHN2ZyB3aWR0aD0iMSIgaGVpZ2h0PSIxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPjwvc3ZnPg==\" style=\"--smush-placeholder-width: 850px; aspect-ratio: 850\/85;\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">In the matrix y, the first column represents class 0 and second column represents class 1. Since our example belongs to Class 1, we have 1 in the second column and zero in the first.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img decoding=\"async\" width=\"836\" height=\"192\" data-src=\"\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/forward-propagation-formula-class.png\" alt=\"\" class=\"wp-image-208611 lazyload\" data-srcset=\"https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/forward-propagation-formula-class.png 836w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/forward-propagation-formula-class-700x161.png 700w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/forward-propagation-formula-class-300x69.png 300w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/forward-propagation-formula-class-768x176.png 768w\" data-sizes=\"(max-width: 836px) 100vw, 836px\" src=\"data:image\/svg+xml;base64,PHN2ZyB3aWR0aD0iMSIgaGVpZ2h0PSIxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPjwvc3ZnPg==\" style=\"--smush-placeholder-width: 836px; aspect-ratio: 836\/192;\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">This process of converting discrete\/categorical classes to logical vectors\/matrices is called One-Hot Encoding. It&#8217;s sort of like converting the decimal system (1,2,3,4&#8230;.9) to binary (0,1,01,10,11). We use one-hot encoding as the neural network cannot operate on label data directly. They require all input variables and output variables to be numeric.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">In neural network learning, apart from the input variable, we add a bias term to every layer other than the output layer. This bias term is a constant, mostly initialised to 1. The bias enables moving the activation threshold along the x-axis.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img decoding=\"async\" width=\"720\" height=\"271\" data-src=\"\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/Bias-forward-propagation-bias.png\" alt=\"\" class=\"wp-image-208612 lazyload\" data-srcset=\"https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/Bias-forward-propagation-bias.png 720w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/Bias-forward-propagation-bias-700x263.png 700w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/Bias-forward-propagation-bias-300x113.png 300w\" data-sizes=\"(max-width: 720px) 100vw, 720px\" src=\"data:image\/svg+xml;base64,PHN2ZyB3aWR0aD0iMSIgaGVpZ2h0PSIxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPjwvc3ZnPg==\" style=\"--smush-placeholder-width: 720px; aspect-ratio: 720\/271;\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">When the bias is negative the movement is made to the right side, and when the bias is positive the movement is made to the left side. So a biassed neuron should be capable of learning even such input vectors that an unbiased neuron is not able to learn. In the dataset X, to introduce this bias we add a new column denoted by ones, as shown below.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img decoding=\"async\" width=\"848\" height=\"54\" data-src=\"\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/Bias-forward-propagation-bias-formula.png\" alt=\"\" class=\"wp-image-208613 lazyload\" data-srcset=\"https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/Bias-forward-propagation-bias-formula.png 848w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/Bias-forward-propagation-bias-formula-700x45.png 700w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/Bias-forward-propagation-bias-formula-300x19.png 300w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/Bias-forward-propagation-bias-formula-768x49.png 768w\" data-sizes=\"(max-width: 848px) 100vw, 848px\" src=\"data:image\/svg+xml;base64,PHN2ZyB3aWR0aD0iMSIgaGVpZ2h0PSIxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPjwvc3ZnPg==\" style=\"--smush-placeholder-width: 848px; aspect-ratio: 848\/54;\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">Let us randomly initialise the weights or parameters for each of the neurons in the first layer. As you can see in the diagram we have a line connecting each of the cells in the first layer to the two neurons in the second layer. This gives us a total of 6 weights to be initialized, 3 for each neuron in the hidden layer. We represent these weights as shown below.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img decoding=\"async\" width=\"813\" height=\"85\" data-src=\"\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/Bias-forward-propagation-bias-formula-theta.png\" alt=\"\" class=\"wp-image-208615 lazyload\" data-srcset=\"https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/Bias-forward-propagation-bias-formula-theta.png 813w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/Bias-forward-propagation-bias-formula-theta-700x73.png 700w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/Bias-forward-propagation-bias-formula-theta-300x31.png 300w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/Bias-forward-propagation-bias-formula-theta-768x80.png 768w\" data-sizes=\"(max-width: 813px) 100vw, 813px\" src=\"data:image\/svg+xml;base64,PHN2ZyB3aWR0aD0iMSIgaGVpZ2h0PSIxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPjwvc3ZnPg==\" style=\"--smush-placeholder-width: 813px; aspect-ratio: 813\/85;\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">Here, Theta<sub>1<\/sub>&nbsp;is the weights matrix corresponding to the first layer.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img decoding=\"async\" width=\"407\" height=\"651\" data-src=\"\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/Weights-matrix-quantinsti.png\" alt=\"\" class=\"wp-image-208616 lazyload\" data-srcset=\"https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/Weights-matrix-quantinsti.png 407w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/Weights-matrix-quantinsti-300x480.png 300w\" data-sizes=\"(max-width: 407px) 100vw, 407px\" src=\"data:image\/svg+xml;base64,PHN2ZyB3aWR0aD0iMSIgaGVpZ2h0PSIxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPjwvc3ZnPg==\" style=\"--smush-placeholder-width: 407px; aspect-ratio: 407\/651;\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">The first row in the above representation shows the weights corresponding to the first neuron in the second layer, and the second row represents the weights corresponding to the second neuron in the second layer. Now, let\u2019s do the first step of the forward propagation, by multiplying the input value for each example by their corresponding weights which are mathematically shown below.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Theta<sub>1<\/sub>&nbsp;* X<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Before we go ahead and multiply, we must remember that when you do matrix multiplications, each element of the product, X*\u03b8, is the dot product sum of the row in the first matrix X with each of the columns of the second matrix \u03b8.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">When we multiply the two matrices, X and \u03b8, we are expected to multiply the weights with the corresponding input example values. This means we need to transpose the matrix of example input data, X so that the matrix will multiply each weight with the corresponding input correctly.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img decoding=\"async\" width=\"848\" height=\"166\" data-src=\"\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/forward-propagation-formula-z-theta.png\" alt=\"\" class=\"wp-image-208617 lazyload\" data-srcset=\"https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/forward-propagation-formula-z-theta.png 848w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/forward-propagation-formula-z-theta-700x137.png 700w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/forward-propagation-formula-z-theta-300x59.png 300w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/forward-propagation-formula-z-theta-768x150.png 768w\" data-sizes=\"(max-width: 848px) 100vw, 848px\" src=\"data:image\/svg+xml;base64,PHN2ZyB3aWR0aD0iMSIgaGVpZ2h0PSIxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPjwvc3ZnPg==\" style=\"--smush-placeholder-width: 848px; aspect-ratio: 848\/166;\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">Here z<sup>2<\/sup>\u00a0is the output after matrix multiplication, and X<sub>t<\/sub>\u00a0is the transpose of X.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The matrix multiplication process:<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img decoding=\"async\" width=\"830\" height=\"188\" data-src=\"\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/forward-propagation-matrix-multiplication.png\" alt=\"\" class=\"wp-image-208619 lazyload\" data-srcset=\"https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/forward-propagation-matrix-multiplication.png 830w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/forward-propagation-matrix-multiplication-700x159.png 700w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/forward-propagation-matrix-multiplication-300x68.png 300w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/forward-propagation-matrix-multiplication-768x174.png 768w\" data-sizes=\"(max-width: 830px) 100vw, 830px\" src=\"data:image\/svg+xml;base64,PHN2ZyB3aWR0aD0iMSIgaGVpZ2h0PSIxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPjwvc3ZnPg==\" style=\"--smush-placeholder-width: 830px; aspect-ratio: 830\/188;\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">Let us say that we have applied a sigmoid activation after the input layer. Then we have to element-wise apply the sigmoid function to the elements in the z\u00b2 matrix above. The sigmoid function is given by the following equation:<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img decoding=\"async\" width=\"824\" height=\"92\" data-src=\"\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/forward-propagation-z-equation.png\" alt=\"\" class=\"wp-image-208634 lazyload\" data-srcset=\"https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/forward-propagation-z-equation.png 824w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/forward-propagation-z-equation-700x78.png 700w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/forward-propagation-z-equation-300x33.png 300w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/forward-propagation-z-equation-768x86.png 768w\" data-sizes=\"(max-width: 824px) 100vw, 824px\" src=\"data:image\/svg+xml;base64,PHN2ZyB3aWR0aD0iMSIgaGVpZ2h0PSIxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPjwvc3ZnPg==\" style=\"--smush-placeholder-width: 824px; aspect-ratio: 824\/92;\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">After the application of the activation function, we are left with a 2&#215;1 matrix as shown below.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img decoding=\"async\" width=\"831\" height=\"83\" data-src=\"\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/forward-propagation-activation.png\" alt=\"\" class=\"wp-image-208635 lazyload\" data-srcset=\"https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/forward-propagation-activation.png 831w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/forward-propagation-activation-700x70.png 700w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/forward-propagation-activation-300x30.png 300w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/forward-propagation-activation-768x77.png 768w\" data-sizes=\"(max-width: 831px) 100vw, 831px\" src=\"data:image\/svg+xml;base64,PHN2ZyB3aWR0aD0iMSIgaGVpZ2h0PSIxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPjwvc3ZnPg==\" style=\"--smush-placeholder-width: 831px; aspect-ratio: 831\/83;\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">Here a<sup>(2)<\/sup>&nbsp;represents the output of the activation layer.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">These outputs of the activation layer act as the inputs for the next or the final layer, which is the output layer. Let us initialize another random weights\/parameters called Theta<sub>2<\/sub>\u00a0for the hidden layer. Each row in Theta<sub>2<\/sub>\u00a0represents the weights corresponding to the two neurons in the output layer.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img decoding=\"async\" width=\"829\" height=\"80\" data-src=\"\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/forward-propagation-theta-2.png\" alt=\"\" class=\"wp-image-208650 lazyload\" data-srcset=\"https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/forward-propagation-theta-2.png 829w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/forward-propagation-theta-2-700x68.png 700w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/forward-propagation-theta-2-300x29.png 300w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/forward-propagation-theta-2-768x74.png 768w\" data-sizes=\"(max-width: 829px) 100vw, 829px\" src=\"data:image\/svg+xml;base64,PHN2ZyB3aWR0aD0iMSIgaGVpZ2h0PSIxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPjwvc3ZnPg==\" style=\"--smush-placeholder-width: 829px; aspect-ratio: 829\/80;\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">After initializing the weights (Theta<sub>2<\/sub>), we will repeat the same process that we followed for the input layer. We will add a bias term for the inputs of the previous layer. The a<sup>(2)<\/sup>&nbsp;matrix looks like this after the addition of bias vectors:<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img decoding=\"async\" width=\"823\" height=\"110\" data-src=\"\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/forward-propagation-bias-vectors.png\" alt=\"\" class=\"wp-image-208638 lazyload\" data-srcset=\"https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/forward-propagation-bias-vectors.png 823w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/forward-propagation-bias-vectors-700x94.png 700w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/forward-propagation-bias-vectors-300x40.png 300w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/forward-propagation-bias-vectors-768x103.png 768w\" data-sizes=\"(max-width: 823px) 100vw, 823px\" src=\"data:image\/svg+xml;base64,PHN2ZyB3aWR0aD0iMSIgaGVpZ2h0PSIxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPjwvc3ZnPg==\" style=\"--smush-placeholder-width: 823px; aspect-ratio: 823\/110;\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">Let us see how the neural network looks like after the addition of the bias unit:<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img decoding=\"async\" width=\"720\" height=\"313\" data-src=\"\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/Bias-unit-in-forward-propagation-quantinsti.png\" alt=\"\" class=\"wp-image-208639 lazyload\" data-srcset=\"https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/Bias-unit-in-forward-propagation-quantinsti.png 720w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/Bias-unit-in-forward-propagation-quantinsti-700x304.png 700w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/Bias-unit-in-forward-propagation-quantinsti-300x130.png 300w\" data-sizes=\"(max-width: 720px) 100vw, 720px\" src=\"data:image\/svg+xml;base64,PHN2ZyB3aWR0aD0iMSIgaGVpZ2h0PSIxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPjwvc3ZnPg==\" style=\"--smush-placeholder-width: 720px; aspect-ratio: 720\/313;\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">Before we run our matrix multiplication to compute the final output z\u00b3, remember that before in z\u00b2 calculation we had to transpose the input data a\u00b9 to make it \u201cline up\u201d correctly for the matrix multiplication to result in the computations we wanted. Here, our matrices are already lined up the way we want, so there is no need to take the transpose of the a<sup>(2)<\/sup>&nbsp;matrix. To understand this clearly, ask yourself this question: \u201cWhich weights are being multiplied with what inputs?\u201d. Now, let us perform the matrix multiplication:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">z<sup>3<\/sup>&nbsp;= Theta<sub>2<\/sub>*a<sup>(2)<\/sup>&nbsp;where z<sup>3<\/sup>&nbsp;is the output matrix before the application of an activation function.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Here for the last layer, we will be multiplying a 2&#215;3 with a 3&#215;1 matrix, resulting in a 2&#215;1 matrix of output hypotheses. The mathematical computation is shown below:<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img decoding=\"async\" width=\"821\" height=\"179\" data-src=\"\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/forward-propagation-math-computation.png\" alt=\"\" class=\"wp-image-208640 lazyload\" data-srcset=\"https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/forward-propagation-math-computation.png 821w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/forward-propagation-math-computation-700x153.png 700w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/forward-propagation-math-computation-300x65.png 300w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/forward-propagation-math-computation-768x167.png 768w\" data-sizes=\"(max-width: 821px) 100vw, 821px\" src=\"data:image\/svg+xml;base64,PHN2ZyB3aWR0aD0iMSIgaGVpZ2h0PSIxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPjwvc3ZnPg==\" style=\"--smush-placeholder-width: 821px; aspect-ratio: 821\/179;\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">After this multiplication, before getting the output in the final layer, we apply an element-wise conversion using the sigmoid function on the z\u00b2 matrix.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">a<sup>3<\/sup>&nbsp;= sigmoid(z<sup>3<\/sup>)<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Where a<sup>3<\/sup>&nbsp;denotes the final output matrix.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img decoding=\"async\" width=\"827\" height=\"78\" data-src=\"\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/forward-propagation-z-matrix.png\" alt=\"\" class=\"wp-image-208641 lazyload\" data-srcset=\"https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/forward-propagation-z-matrix.png 827w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/forward-propagation-z-matrix-700x66.png 700w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/forward-propagation-z-matrix-300x28.png 300w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/06\/forward-propagation-z-matrix-768x72.png 768w\" data-sizes=\"(max-width: 827px) 100vw, 827px\" src=\"data:image\/svg+xml;base64,PHN2ZyB3aWR0aD0iMSIgaGVpZ2h0PSIxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPjwvc3ZnPg==\" style=\"--smush-placeholder-width: 827px; aspect-ratio: 827\/78;\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">The output of a sigmoid function is the probability of the given example belonging to a particular class. In the above representation, the first row represents the probability that the example belonging to Class 0 and the second row represents the probability of Class 1.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">That\u2019s all there is to know about forward propagation in Neural networks. <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Author: Chainika Thakar (Originally written by Varun Divakar and Rekhit Pachanekar)<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><em>Stay tuned for Part III to lean about forward propagation in trading.<\/em><\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><em>Originally posted on&nbsp;<a href=\"https:\/\/blog.quantinsti.com\/forward-propagation-neural-networks\/\">QuantInsti<\/a>.<\/em><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Forward propagation is a key process in various types of neural networks, each with its own architecture and specific steps involved in moving input data through the network to produce an output.<\/p>\n","protected":false},"author":186,"featured_media":53216,"comment_status":"open","ping_status":"closed","sticky":true,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":"","jetpack_post_was_ever_published":false},"categories":[339,343,349,338,341],"tags":[17324,4087,17321,806,17320,4363,17323,852,15593,595,17322],"contributors-categories":[13654],"class_list":["post-208602","post","type-post","status-publish","format-standard","has-post-thumbnail","category-data-science","category-programing-languages","category-python-development","category-ibkr-quant-news","category-quant-development","tag-autoencoder-networks","tag-backpropagation","tag-convolutional-neural-networks-cnn","tag-data-science","tag-feedforward-neural-networks-fnn","tag-forward-propagation","tag-long-short-term-memory-networks-lstm","tag-machine-learning","tag-neural-networks","tag-python","tag-recurrent-neural-networks-rnn","contributors-categories-quantinsti"],"pp_statuses_selecting_workflow":false,"pp_workflow_action":"current","pp_status_selection":"publish","acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v26.9 (Yoast SEO v28.3) - https:\/\/yoast.com\/product\/yoast-seo-premium-wordpress\/ -->\n<title>Forward Propagation In Neural Networks: Components and Applications &#8211; Part II<\/title>\n<meta name=\"description\" content=\"Forward propagation is a key process in various types of neural networks, each with its own architecture and specific steps involved in moving input...\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.interactivebrokers.com\/campus\/wp-json\/wp\/v2\/posts\/208602\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Forward Propagation In Neural Networks: Components and Applications - Part II\" \/>\n<meta property=\"og:description\" content=\"Forward propagation is a key process in various types of neural networks, each with its own architecture and specific steps involved in moving input data through the network to produce an output.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/forward-propagation-in-neural-networks-components-and-applications-part-ii\/\" \/>\n<meta property=\"og:site_name\" content=\"IBKR Campus US\" \/>\n<meta property=\"article:published_time\" content=\"2024-07-02T16:05:28+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2024-07-02T16:07:12+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2020\/07\/code-abstract.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"900\" \/>\n\t<meta property=\"og:image:height\" content=\"550\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Contributor Author\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Contributor Author\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"12 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\n\t    \"@context\": \"https:\\\/\\\/schema.org\",\n\t    \"@graph\": [\n\t        {\n\t            \"@type\": \"NewsArticle\",\n\t            \"@id\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/forward-propagation-in-neural-networks-components-and-applications-part-ii\\\/#article\",\n\t            \"isPartOf\": {\n\t                \"@id\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/forward-propagation-in-neural-networks-components-and-applications-part-ii\\\/\"\n\t            },\n\t            \"author\": {\n\t                \"name\": \"Contributor Author\",\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#\\\/schema\\\/person\\\/e823e46b42ca381080387e794318a485\"\n\t            },\n\t            \"headline\": \"Forward Propagation In Neural Networks: Components and Applications &#8211; Part II\",\n\t            \"datePublished\": \"2024-07-02T16:05:28+00:00\",\n\t            \"dateModified\": \"2024-07-02T16:07:12+00:00\",\n\t            \"mainEntityOfPage\": {\n\t                \"@id\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/forward-propagation-in-neural-networks-components-and-applications-part-ii\\\/\"\n\t            },\n\t            \"wordCount\": 1690,\n\t            \"commentCount\": 0,\n\t            \"publisher\": {\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#organization\"\n\t            },\n\t            \"image\": {\n\t                \"@id\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/forward-propagation-in-neural-networks-components-and-applications-part-ii\\\/#primaryimage\"\n\t            },\n\t            \"thumbnailUrl\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2020\\\/07\\\/code-abstract.jpg\",\n\t            \"keywords\": [\n\t                \"Autoencoder Networks\",\n\t                \"Backpropagation\",\n\t                \"Convolutional Neural Networks (CNN)\",\n\t                \"Data Science\",\n\t                \"Feedforward Neural Networks (FNN)\",\n\t                \"Forward propagation\",\n\t                \"Long Short-Term Memory Networks (LSTM)\",\n\t                \"Machine Learning\",\n\t                \"Neural Networks\",\n\t                \"Python\",\n\t                \"Recurrent Neural Networks (RNN)\"\n\t            ],\n\t            \"articleSection\": [\n\t                \"Data Science\",\n\t                \"Programming Languages\",\n\t                \"Python Development\",\n\t                \"Quant\",\n\t                \"Quant Development\"\n\t            ],\n\t            \"inLanguage\": \"en-US\",\n\t            \"potentialAction\": [\n\t                {\n\t                    \"@type\": \"CommentAction\",\n\t                    \"name\": \"Comment\",\n\t                    \"target\": [\n\t                        \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/forward-propagation-in-neural-networks-components-and-applications-part-ii\\\/#respond\"\n\t                    ]\n\t                }\n\t            ]\n\t        },\n\t        {\n\t            \"@type\": \"WebPage\",\n\t            \"@id\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/forward-propagation-in-neural-networks-components-and-applications-part-ii\\\/\",\n\t            \"url\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/forward-propagation-in-neural-networks-components-and-applications-part-ii\\\/\",\n\t            \"name\": \"Forward Propagation In Neural Networks: Components and Applications - Part II | IBKR Campus US\",\n\t            \"isPartOf\": {\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#website\"\n\t            },\n\t            \"primaryImageOfPage\": {\n\t                \"@id\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/forward-propagation-in-neural-networks-components-and-applications-part-ii\\\/#primaryimage\"\n\t            },\n\t            \"image\": {\n\t                \"@id\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/forward-propagation-in-neural-networks-components-and-applications-part-ii\\\/#primaryimage\"\n\t            },\n\t            \"thumbnailUrl\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2020\\\/07\\\/code-abstract.jpg\",\n\t            \"datePublished\": \"2024-07-02T16:05:28+00:00\",\n\t            \"dateModified\": \"2024-07-02T16:07:12+00:00\",\n\t            \"description\": \"Forward propagation is a key process in various types of neural networks, each with its own architecture and specific steps involved in moving input data through the network to produce an output.\",\n\t            \"inLanguage\": \"en-US\",\n\t            \"potentialAction\": [\n\t                {\n\t                    \"@type\": \"ReadAction\",\n\t                    \"target\": [\n\t                        \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/forward-propagation-in-neural-networks-components-and-applications-part-ii\\\/\"\n\t                    ]\n\t                }\n\t            ]\n\t        },\n\t        {\n\t            \"@type\": \"ImageObject\",\n\t            \"inLanguage\": \"en-US\",\n\t            \"@id\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/forward-propagation-in-neural-networks-components-and-applications-part-ii\\\/#primaryimage\",\n\t            \"url\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2020\\\/07\\\/code-abstract.jpg\",\n\t            \"contentUrl\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2020\\\/07\\\/code-abstract.jpg\",\n\t            \"width\": 900,\n\t            \"height\": 550,\n\t            \"caption\": \"Quant\"\n\t        },\n\t        {\n\t            \"@type\": \"WebSite\",\n\t            \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#website\",\n\t            \"url\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/\",\n\t            \"name\": \"IBKR Campus US\",\n\t            \"description\": \"Financial Education from Interactive Brokers\",\n\t            \"publisher\": {\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#organization\"\n\t            },\n\t            \"potentialAction\": [\n\t                {\n\t                    \"@type\": \"SearchAction\",\n\t                    \"target\": {\n\t                        \"@type\": \"EntryPoint\",\n\t                        \"urlTemplate\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/?s={search_term_string}\"\n\t                    },\n\t                    \"query-input\": {\n\t                        \"@type\": \"PropertyValueSpecification\",\n\t                        \"valueRequired\": true,\n\t                        \"valueName\": \"search_term_string\"\n\t                    }\n\t                }\n\t            ],\n\t            \"inLanguage\": \"en-US\"\n\t        },\n\t        {\n\t            \"@type\": \"Organization\",\n\t            \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#organization\",\n\t            \"name\": \"Interactive Brokers\",\n\t            \"alternateName\": \"IBKR\",\n\t            \"url\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/\",\n\t            \"logo\": {\n\t                \"@type\": \"ImageObject\",\n\t                \"inLanguage\": \"en-US\",\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#\\\/schema\\\/logo\\\/image\\\/\",\n\t                \"url\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2024\\\/05\\\/ibkr-campus-logo.jpg\",\n\t                \"contentUrl\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2024\\\/05\\\/ibkr-campus-logo.jpg\",\n\t                \"width\": 669,\n\t                \"height\": 669,\n\t                \"caption\": \"Interactive Brokers\"\n\t            },\n\t            \"image\": {\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#\\\/schema\\\/logo\\\/image\\\/\"\n\t            },\n\t            \"publishingPrinciples\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/about-ibkr-campus\\\/\",\n\t            \"ethicsPolicy\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/cyber-security-notice\\\/\"\n\t        },\n\t        {\n\t            \"@type\": \"Person\",\n\t            \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#\\\/schema\\\/person\\\/e823e46b42ca381080387e794318a485\",\n\t            \"name\": \"Contributor Author\",\n\t            \"url\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/author\\\/contributor-author\\\/\"\n\t        }\n\t    ]\n\t}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Forward Propagation In Neural Networks: Components and Applications &#8211; Part II","description":"Forward propagation is a key process in various types of neural networks, each with its own architecture and specific steps involved in moving input...","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.interactivebrokers.com\/campus\/wp-json\/wp\/v2\/posts\/208602\/","og_locale":"en_US","og_type":"article","og_title":"Forward Propagation In Neural Networks: Components and Applications - Part II","og_description":"Forward propagation is a key process in various types of neural networks, each with its own architecture and specific steps involved in moving input data through the network to produce an output.","og_url":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/forward-propagation-in-neural-networks-components-and-applications-part-ii\/","og_site_name":"IBKR Campus US","article_published_time":"2024-07-02T16:05:28+00:00","article_modified_time":"2024-07-02T16:07:12+00:00","og_image":[{"width":900,"height":550,"url":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2020\/07\/code-abstract.jpg","type":"image\/jpeg"}],"author":"Contributor Author","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Contributor Author","Est. reading time":"12 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"NewsArticle","@id":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/forward-propagation-in-neural-networks-components-and-applications-part-ii\/#article","isPartOf":{"@id":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/forward-propagation-in-neural-networks-components-and-applications-part-ii\/"},"author":{"name":"Contributor Author","@id":"https:\/\/ibkrcampus.com\/campus\/#\/schema\/person\/e823e46b42ca381080387e794318a485"},"headline":"Forward Propagation In Neural Networks: Components and Applications &#8211; Part II","datePublished":"2024-07-02T16:05:28+00:00","dateModified":"2024-07-02T16:07:12+00:00","mainEntityOfPage":{"@id":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/forward-propagation-in-neural-networks-components-and-applications-part-ii\/"},"wordCount":1690,"commentCount":0,"publisher":{"@id":"https:\/\/ibkrcampus.com\/campus\/#organization"},"image":{"@id":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/forward-propagation-in-neural-networks-components-and-applications-part-ii\/#primaryimage"},"thumbnailUrl":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2020\/07\/code-abstract.jpg","keywords":["Autoencoder Networks","Backpropagation","Convolutional Neural Networks (CNN)","Data Science","Feedforward Neural Networks (FNN)","Forward propagation","Long Short-Term Memory Networks (LSTM)","Machine Learning","Neural Networks","Python","Recurrent Neural Networks (RNN)"],"articleSection":["Data Science","Programming Languages","Python Development","Quant","Quant Development"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/forward-propagation-in-neural-networks-components-and-applications-part-ii\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/forward-propagation-in-neural-networks-components-and-applications-part-ii\/","url":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/forward-propagation-in-neural-networks-components-and-applications-part-ii\/","name":"Forward Propagation In Neural Networks: Components and Applications - Part II | IBKR Campus US","isPartOf":{"@id":"https:\/\/ibkrcampus.com\/campus\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/forward-propagation-in-neural-networks-components-and-applications-part-ii\/#primaryimage"},"image":{"@id":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/forward-propagation-in-neural-networks-components-and-applications-part-ii\/#primaryimage"},"thumbnailUrl":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2020\/07\/code-abstract.jpg","datePublished":"2024-07-02T16:05:28+00:00","dateModified":"2024-07-02T16:07:12+00:00","description":"Forward propagation is a key process in various types of neural networks, each with its own architecture and specific steps involved in moving input data through the network to produce an output.","inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/forward-propagation-in-neural-networks-components-and-applications-part-ii\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/forward-propagation-in-neural-networks-components-and-applications-part-ii\/#primaryimage","url":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2020\/07\/code-abstract.jpg","contentUrl":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2020\/07\/code-abstract.jpg","width":900,"height":550,"caption":"Quant"},{"@type":"WebSite","@id":"https:\/\/ibkrcampus.com\/campus\/#website","url":"https:\/\/ibkrcampus.com\/campus\/","name":"IBKR Campus US","description":"Financial Education from Interactive Brokers","publisher":{"@id":"https:\/\/ibkrcampus.com\/campus\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/ibkrcampus.com\/campus\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/ibkrcampus.com\/campus\/#organization","name":"Interactive Brokers","alternateName":"IBKR","url":"https:\/\/ibkrcampus.com\/campus\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/ibkrcampus.com\/campus\/#\/schema\/logo\/image\/","url":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/05\/ibkr-campus-logo.jpg","contentUrl":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/05\/ibkr-campus-logo.jpg","width":669,"height":669,"caption":"Interactive Brokers"},"image":{"@id":"https:\/\/ibkrcampus.com\/campus\/#\/schema\/logo\/image\/"},"publishingPrinciples":"https:\/\/www.interactivebrokers.com\/campus\/about-ibkr-campus\/","ethicsPolicy":"https:\/\/www.interactivebrokers.com\/campus\/cyber-security-notice\/"},{"@type":"Person","@id":"https:\/\/ibkrcampus.com\/campus\/#\/schema\/person\/e823e46b42ca381080387e794318a485","name":"Contributor Author","url":"https:\/\/www.interactivebrokers.com\/campus\/author\/contributor-author\/"}]}},"jetpack_featured_media_url":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2020\/07\/code-abstract.jpg","_links":{"self":[{"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/posts\/208602","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/users\/186"}],"replies":[{"embeddable":true,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/comments?post=208602"}],"version-history":[{"count":0,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/posts\/208602\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/media\/53216"}],"wp:attachment":[{"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/media?parent=208602"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/categories?post=208602"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/tags?post=208602"},{"taxonomy":"contributors-categories","embeddable":true,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/contributors-categories?post=208602"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}