{"id":43880,"date":"2020-05-01T13:52:04","date_gmt":"2020-05-01T17:52:04","guid":{"rendered":"https:\/\/ibkrcampus.com\/?p=43880"},"modified":"2022-11-21T09:45:28","modified_gmt":"2022-11-21T14:45:28","slug":"neural-network-in-python-part-viii","status":"publish","type":"post","link":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/neural-network-in-python-part-viii\/","title":{"rendered":"Neural Network In Python \u2013 Part VIII"},"content":{"rendered":"\n<p><em>In the&nbsp;<a href=\"\/campus\/ibkr-quant-news\/neural-network-in-python-part-vii\/\">previous installmen<\/a>t, the author discussed Importing the dataset<\/em>.<\/p>\n\n\n\n<p><strong>Preparing the dataset<\/strong><\/p>\n\n\n\n<p style=\"background-color:#fcfcdb;font-size:11px\" class=\"has-background\">\ndataset[&#8216;H-L&#8217;] = dataset[&#8216;High&#8217;] &#8211; dataset[&#8216;Low&#8217;]<br>\ndataset[&#8216;O-C&#8217;] = dataset[&#8216;Close&#8217;] &#8211; dataset[&#8216;Open&#8217;]<br>\ndataset[&#8216;3day MA&#8217;] = dataset[&#8216;Close&#8217;].shift(1).rolling(window = 3).mean()<br>\ndataset[&#8217;10day MA&#8217;] = dataset[&#8216;Close&#8217;].shift(1).rolling(window = 10).mean()<br>\ndataset[&#8217;30day MA&#8217;] = dataset[&#8216;Close&#8217;].shift(1).rolling(window = 30).mean()<br>\ndataset[&#8216;Std_dev&#8217;]= dataset[&#8216;Close&#8217;].rolling(5).std()<br>\ndataset[&#8216;RSI&#8217;] = talib.RSI(dataset[&#8216;Close&#8217;].values, timeperiod = 9)<br>\ndataset[&#8216;Williams %R&#8217;] = talib.WILLR(dataset[&#8216;High&#8217;].values, dataset[&#8216;Low&#8217;].values, dataset[&#8216;Close&#8217;].values, 7)\n<\/p>\n\n\n\n<p>We then prepare the various input features which will be used by the artificial&nbsp;<a href=\"https:\/\/quantra.quantinsti.com\/course\/neural-networks-deep-learning-trading-ernest-chan\" target=\"_blank\" rel=\"noreferrer noopener\">neural network learning<\/a>&nbsp;for making the predictions. We define the following input features:<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li>High minus Low price<\/li><li>Close minus Open price<\/li><li>Three day moving average<\/li><li>Ten day moving average<\/li><li>30 day moving average<\/li><li>Standard deviation for a period of 5 days<\/li><li>Relative Strength Index<\/li><li>Williams %R<\/li><\/ul>\n\n\n\n<p style=\"background-color:#fcfcdb;font-size:11px\" class=\"has-background\">\ndataset[&#8216;Price_Rise&#8217;] = np.where(dataset[&#8216;Close&#8217;].shift(-1) &gt; dataset[&#8216;Close&#8217;], 1, 0)\n<\/p>\n\n\n\n<p>We then define the output value as price rise, which is a binary variable storing 1 when the closing price of tomorrow is greater than the closing price of today.<\/p>\n\n\n\n<p>dataset = dataset.dropna()<\/p>\n\n\n\n<p>Next, we drop all the rows storing NaN values by using the dropna() function.<\/p>\n\n\n\n<p style=\"background-color:#fcfcdb;font-size:11px\" class=\"has-background\">\nX = dataset.iloc[:, 4:-1]<br>\ny = dataset.iloc[:, -1]\n<\/p>\n\n\n\n<p>We then create two data frames storing the input and the output variables. The dataframe \u2018X\u2019 stores the input features, the columns starting from the fifth column (or index 4) of the dataset till the second last column. The last column will be stored in the dataframe y, which is the value we want to predict, i.e. the price rise.<\/p>\n\n\n\n<p><strong>Splitting the dataset<\/strong><\/p>\n\n\n\n<p style=\"background-color:#fcfcdb;font-size:11px\" class=\"has-background\">\nsplit = int(len(dataset)*0.8)<br>\nX_train, X_test, y_train, y_test = X[:split], X[split:], y[:split], y[split:]\n\n<\/p>\n\n\n\n<p>In this part of the code, we will split our input and output variables to create the test and train datasets. This is done by creating a variable called split, which is defined to be the integer value of 0.8 times the length of the dataset.<\/p>\n\n\n\n<p>We then slice the X and y variables into four separate data frames: Xtrain, Xtest, ytrain and ytest. This is an essential part of any&nbsp;<a href=\"https:\/\/blog.quantinsti.com\/introduction-machine-learning-quantiacs\/\">machine learning<\/a>&nbsp;algorithm, the training data is used by the model to arrive at the weights of the model. The test dataset is used to see how the model will perform on new data which would be fed into the model. The test dataset also has the actual value for the output, which helps us in understanding how efficient the model is. We will look at the confusion matrix later in the code, which essentially is a measure of how accurate the predictions made by the model are.<\/p>\n\n\n\n<p><em>In the next installment, the author will demonstrate how Feature Scaling<\/em><\/p>\n\n\n\n<p>Visit&nbsp;<a href=\"https:\/\/www.quantinsti.com\/\">https:\/\/www.quantinsti.com\/<\/a>&nbsp;for ready-to-use functions as applied in trading and data analysis.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Join QuantInsti for a demonstration on how to prepare and split a dataset in Python for how to customizing neural networks for trading<\/p>\n","protected":false},"author":251,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":true,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[339,349,338,350,341,344],"tags":[851,4087,4582,2105,2697,4363,4227,852,3866,1225,1224,595,5902,5482,5483],"contributors-categories":[13654],"class_list":{"0":"post-43880","1":"post","2":"type-post","3":"status-publish","4":"format-standard","6":"category-data-science","7":"category-python-development","8":"category-ibkr-quant-news","9":"category-quant-asia-pacific","10":"category-quant-development","11":"category-quant-regions","12":"tag-algo-trading","13":"tag-backpropagation","14":"tag-dataframe","15":"tag-deep-learning","16":"tag-deep-neural-networks","17":"tag-forward-propagation","18":"tag-gradient-descent","19":"tag-machine-learning","20":"tag-neuron","21":"tag-numpy","22":"tag-pandas","23":"tag-python","24":"tag-rsi-stochastic-gradient-descent","25":"tag-talib","26":"tag-williams-r","27":"contributors-categories-quantinsti"},"pp_statuses_selecting_workflow":false,"pp_workflow_action":"current","pp_status_selection":"publish","acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v26.9 (Yoast SEO v27.5) - https:\/\/yoast.com\/product\/yoast-seo-premium-wordpress\/ -->\n<title>Neural Network In Python \u2013 Part VIII | IBKR Quant<\/title>\n<meta name=\"description\" content=\"Join QuantInsti for a demonstration on how to prepare and split a dataset in Python, and continue customizing neural networks for trading\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.interactivebrokers.com\/campus\/wp-json\/wp\/v2\/posts\/43880\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Neural Network In Python \u2013 Part VIII | IBKR Quant Blog\" \/>\n<meta property=\"og:description\" content=\"Join QuantInsti for a demonstration on how to prepare and split a dataset in Python, and continue customizing neural networks for trading\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/neural-network-in-python-part-viii\/\" \/>\n<meta property=\"og:site_name\" content=\"IBKR Campus US\" \/>\n<meta property=\"article:published_time\" content=\"2020-05-01T17:52:04+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2022-11-21T14:45:28+00:00\" \/>\n<meta name=\"author\" content=\"Devang Singh\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Devang Singh\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\n\t    \"@context\": \"https:\\\/\\\/schema.org\",\n\t    \"@graph\": [\n\t        {\n\t            \"@type\": \"NewsArticle\",\n\t            \"@id\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/neural-network-in-python-part-viii\\\/#article\",\n\t            \"isPartOf\": {\n\t                \"@id\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/neural-network-in-python-part-viii\\\/\"\n\t            },\n\t            \"author\": {\n\t                \"name\": \"Devang Singh\",\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#\\\/schema\\\/person\\\/e3e144de3dc01a5e3d3b01449fb26629\"\n\t            },\n\t            \"headline\": \"Neural Network In Python \u2013 Part VIII\",\n\t            \"datePublished\": \"2020-05-01T17:52:04+00:00\",\n\t            \"dateModified\": \"2022-11-21T14:45:28+00:00\",\n\t            \"mainEntityOfPage\": {\n\t                \"@id\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/neural-network-in-python-part-viii\\\/\"\n\t            },\n\t            \"wordCount\": 487,\n\t            \"publisher\": {\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#organization\"\n\t            },\n\t            \"keywords\": [\n\t                \"Algo Trading\",\n\t                \"Backpropagation\",\n\t                \"Dataframe\",\n\t                \"Deep Learning\",\n\t                \"Deep Neural Networks\",\n\t                \"Forward propagation\",\n\t                \"Gradient Descent\",\n\t                \"Machine Learning\",\n\t                \"Neuron\",\n\t                \"NumPy\",\n\t                \"Pandas\",\n\t                \"Python\",\n\t                \"rsi Stochastic Gradient Descent\",\n\t                \"Talib\",\n\t                \"Williams %R\"\n\t            ],\n\t            \"articleSection\": [\n\t                \"Data Science\",\n\t                \"Python Development\",\n\t                \"Quant\",\n\t                \"Quant Asia Pacific\",\n\t                \"Quant Development\",\n\t                \"Quant Regions\"\n\t            ],\n\t            \"inLanguage\": \"en-US\"\n\t        },\n\t        {\n\t            \"@type\": \"WebPage\",\n\t            \"@id\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/neural-network-in-python-part-viii\\\/\",\n\t            \"url\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/neural-network-in-python-part-viii\\\/\",\n\t            \"name\": \"Neural Network In Python \u2013 Part VIII | IBKR Quant Blog\",\n\t            \"isPartOf\": {\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#website\"\n\t            },\n\t            \"datePublished\": \"2020-05-01T17:52:04+00:00\",\n\t            \"dateModified\": \"2022-11-21T14:45:28+00:00\",\n\t            \"description\": \"Join QuantInsti for a demonstration on how to prepare and split a dataset in Python, and continue customizing neural networks for trading\",\n\t            \"inLanguage\": \"en-US\",\n\t            \"potentialAction\": [\n\t                {\n\t                    \"@type\": \"ReadAction\",\n\t                    \"target\": [\n\t                        \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/neural-network-in-python-part-viii\\\/\"\n\t                    ]\n\t                }\n\t            ]\n\t        },\n\t        {\n\t            \"@type\": \"WebSite\",\n\t            \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#website\",\n\t            \"url\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/\",\n\t            \"name\": \"IBKR Campus US\",\n\t            \"description\": \"Financial Education from Interactive Brokers\",\n\t            \"publisher\": {\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#organization\"\n\t            },\n\t            \"potentialAction\": [\n\t                {\n\t                    \"@type\": \"SearchAction\",\n\t                    \"target\": {\n\t                        \"@type\": \"EntryPoint\",\n\t                        \"urlTemplate\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/?s={search_term_string}\"\n\t                    },\n\t                    \"query-input\": {\n\t                        \"@type\": \"PropertyValueSpecification\",\n\t                        \"valueRequired\": true,\n\t                        \"valueName\": \"search_term_string\"\n\t                    }\n\t                }\n\t            ],\n\t            \"inLanguage\": \"en-US\"\n\t        },\n\t        {\n\t            \"@type\": \"Organization\",\n\t            \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#organization\",\n\t            \"name\": \"Interactive Brokers\",\n\t            \"alternateName\": \"IBKR\",\n\t            \"url\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/\",\n\t            \"logo\": {\n\t                \"@type\": \"ImageObject\",\n\t                \"inLanguage\": \"en-US\",\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#\\\/schema\\\/logo\\\/image\\\/\",\n\t                \"url\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2024\\\/05\\\/ibkr-campus-logo.jpg\",\n\t                \"contentUrl\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2024\\\/05\\\/ibkr-campus-logo.jpg\",\n\t                \"width\": 669,\n\t                \"height\": 669,\n\t                \"caption\": \"Interactive Brokers\"\n\t            },\n\t            \"image\": {\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#\\\/schema\\\/logo\\\/image\\\/\"\n\t            },\n\t            \"publishingPrinciples\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/about-ibkr-campus\\\/\",\n\t            \"ethicsPolicy\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/cyber-security-notice\\\/\"\n\t        },\n\t        {\n\t            \"@type\": \"Person\",\n\t            \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#\\\/schema\\\/person\\\/e3e144de3dc01a5e3d3b01449fb26629\",\n\t            \"name\": \"Devang Singh\",\n\t            \"url\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/author\\\/devangsingh\\\/\"\n\t        }\n\t    ]\n\t}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Neural Network In Python \u2013 Part VIII | IBKR Quant","description":"Join QuantInsti for a demonstration on how to prepare and split a dataset in Python, and continue customizing neural networks for trading","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.interactivebrokers.com\/campus\/wp-json\/wp\/v2\/posts\/43880\/","og_locale":"en_US","og_type":"article","og_title":"Neural Network In Python \u2013 Part VIII | IBKR Quant Blog","og_description":"Join QuantInsti for a demonstration on how to prepare and split a dataset in Python, and continue customizing neural networks for trading","og_url":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/neural-network-in-python-part-viii\/","og_site_name":"IBKR Campus US","article_published_time":"2020-05-01T17:52:04+00:00","article_modified_time":"2022-11-21T14:45:28+00:00","author":"Devang Singh","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Devang Singh","Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"NewsArticle","@id":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/neural-network-in-python-part-viii\/#article","isPartOf":{"@id":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/neural-network-in-python-part-viii\/"},"author":{"name":"Devang Singh","@id":"https:\/\/ibkrcampus.com\/campus\/#\/schema\/person\/e3e144de3dc01a5e3d3b01449fb26629"},"headline":"Neural Network In Python \u2013 Part VIII","datePublished":"2020-05-01T17:52:04+00:00","dateModified":"2022-11-21T14:45:28+00:00","mainEntityOfPage":{"@id":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/neural-network-in-python-part-viii\/"},"wordCount":487,"publisher":{"@id":"https:\/\/ibkrcampus.com\/campus\/#organization"},"keywords":["Algo Trading","Backpropagation","Dataframe","Deep Learning","Deep Neural Networks","Forward propagation","Gradient Descent","Machine Learning","Neuron","NumPy","Pandas","Python","rsi Stochastic Gradient Descent","Talib","Williams %R"],"articleSection":["Data Science","Python Development","Quant","Quant Asia Pacific","Quant Development","Quant Regions"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/neural-network-in-python-part-viii\/","url":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/neural-network-in-python-part-viii\/","name":"Neural Network In Python \u2013 Part VIII | IBKR Quant Blog","isPartOf":{"@id":"https:\/\/ibkrcampus.com\/campus\/#website"},"datePublished":"2020-05-01T17:52:04+00:00","dateModified":"2022-11-21T14:45:28+00:00","description":"Join QuantInsti for a demonstration on how to prepare and split a dataset in Python, and continue customizing neural networks for trading","inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/neural-network-in-python-part-viii\/"]}]},{"@type":"WebSite","@id":"https:\/\/ibkrcampus.com\/campus\/#website","url":"https:\/\/ibkrcampus.com\/campus\/","name":"IBKR Campus US","description":"Financial Education from Interactive Brokers","publisher":{"@id":"https:\/\/ibkrcampus.com\/campus\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/ibkrcampus.com\/campus\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/ibkrcampus.com\/campus\/#organization","name":"Interactive Brokers","alternateName":"IBKR","url":"https:\/\/ibkrcampus.com\/campus\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/ibkrcampus.com\/campus\/#\/schema\/logo\/image\/","url":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/05\/ibkr-campus-logo.jpg","contentUrl":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/05\/ibkr-campus-logo.jpg","width":669,"height":669,"caption":"Interactive Brokers"},"image":{"@id":"https:\/\/ibkrcampus.com\/campus\/#\/schema\/logo\/image\/"},"publishingPrinciples":"https:\/\/www.interactivebrokers.com\/campus\/about-ibkr-campus\/","ethicsPolicy":"https:\/\/www.interactivebrokers.com\/campus\/cyber-security-notice\/"},{"@type":"Person","@id":"https:\/\/ibkrcampus.com\/campus\/#\/schema\/person\/e3e144de3dc01a5e3d3b01449fb26629","name":"Devang Singh","url":"https:\/\/www.interactivebrokers.com\/campus\/author\/devangsingh\/"}]}},"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/posts\/43880","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/users\/251"}],"replies":[{"embeddable":true,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/comments?post=43880"}],"version-history":[{"count":0,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/posts\/43880\/revisions"}],"wp:attachment":[{"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/media?parent=43880"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/categories?post=43880"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/tags?post=43880"},{"taxonomy":"contributors-categories","embeddable":true,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/contributors-categories?post=43880"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}