{"id":76745,"date":"2021-02-19T10:25:44","date_gmt":"2021-02-19T15:25:44","guid":{"rendered":"https:\/\/ibkrcampus.com\/?p=76745"},"modified":"2022-11-21T09:47:08","modified_gmt":"2022-11-21T14:47:08","slug":"how-is-information-gain-calculated","status":"publish","type":"post","link":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/how-is-information-gain-calculated\/","title":{"rendered":"How Is Information Gain Calculated?"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">This post will explore the mathematics behind information gain. We\u2019ll start with the base intuition behind information gain, but then explain why it has the calculation that it does.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-what-is-information-gain\"><strong>What is information gain?<\/strong><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Information gain is a measure frequently used in decision trees to determine which variable to split the input dataset on at each step in the tree. Before we formally define this measure we need to first understand the concept of&nbsp;<em>entropy<\/em>. Entropy measures the amount of information or uncertainty in a variable\u2019s possible values.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\" id=\"h-how-to-calculate-entropy\"><strong>How to calculate entropy<\/strong><\/h4>\n\n\n\n<p class=\"wp-block-paragraph\">Entropy of a random variable&nbsp;<strong>X<\/strong>&nbsp;is given by the following formula:<\/p>\n\n\n\n<p style=\"background-color:#fcfcdb;font-size:26px\" class=\"has-background\">-\u03a3<sub>i<\/sub>[p(X<sub>i<\/sub>) * log<sub>2<\/sub>(p(X<sub>i<\/sub>))]<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Here, each X<sub>i<\/sub>&nbsp;represents each possible (i<sup>th<\/sup>) value of&nbsp;<strong>X<\/strong>. p(x<sub>i<\/sub>) is the probability of a particular (the i<sup>th<\/sup>) possible value of&nbsp;<strong>X<\/strong>.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\" id=\"h-why-is-it-calculated-this-way\"><strong>Why is it calculated this way?<\/strong><\/h4>\n\n\n\n<p class=\"wp-block-paragraph\">First, let\u2019s build some intuition behind the entropy formula. The formula has several useful properties. For example, it\u2019s always non-negative. Also, entropy as a function is monotonically decreasing in the probability,&nbsp;<em>p<\/em>. In other words, the amount of information about an event (or value) of&nbsp;<strong>X<\/strong>&nbsp;decreases as the probability of that event&nbsp;<em>increases<\/em>.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">That may sound a little abstract at first, so let\u2019s consider a specific example. Suppose you\u2019re expecting how much a movie will make in revenue. One of your anticipators is a binary indicator \u2013 1 if the record refers to a movie, and 0 otherwise. Well \u2013 that anticipator is useless! Mathematically speaking, it\u2019s useless because every record in the dataset is a movie \u2013 so there\u2019s a 100% probability of that event (i.e. the record being a movie) occurring. This means that the variable provides no real information about the data. The closer you get to a variable having a single possible value, the less information that single value gives you.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Why the log<sub>2<\/sub>? Technically, entropy can be calculated using a logarithm of a different base (e.g. natural log). However, it\u2019s common to use base 2 because this returns a result in terms of&nbsp;<em>bits<\/em>. In this way, entropy can be thought of as the average number of&nbsp;<em>bits<\/em>&nbsp;needed to encode a value for a specific variable.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\" id=\"h-case-example\"><strong>Case Example<\/strong><\/h4>\n\n\n\n<p class=\"wp-block-paragraph\">Information gain in the context of decision trees is the reduction in entropy when splitting on variable\u00a0<strong>X<\/strong>. Let\u2019s do an example to make this clear. In the below mini-dataset, the label we\u2019re trying to anticipate is the type of fruit. This is based off the size, color, and shape variables.<\/p>\n\n\n<table>\n<tbody>\n<tr>\n<td><strong>Fruit<\/strong><\/td>\n<td><strong>Size<\/strong><\/td>\n<td><strong>Color<\/strong><\/td>\n<td><strong>Shape<\/strong><\/td>\n<\/tr>\n<tr>\n<td class=\"has-text-align-center\" data-align=\"center\">Watermelon<\/td>\n<td>Big<\/td>\n<td>Green<\/td>\n<td class=\"has-text-align-right\" data-align=\"right\">Round<\/td>\n<\/tr>\n<tr>\n<td class=\"has-text-align-center\" data-align=\"center\">Apple<\/td>\n<td>Medium<\/td>\n<td>Red<\/td>\n<td class=\"has-text-align-right\" data-align=\"right\">Round<\/td>\n<\/tr>\n<tr>\n<td class=\"has-text-align-center\" data-align=\"center\">Banana<\/td>\n<td>Medium<\/td>\n<td>Yellow<\/td>\n<td class=\"has-text-align-right\" data-align=\"right\">Thin<\/td>\n<\/tr>\n<tr>\n<td class=\"has-text-align-center\" data-align=\"center\">Grape<\/td>\n<td>Small<\/td>\n<td>Green<\/td>\n<td class=\"has-text-align-right\" data-align=\"right\">Round<\/td>\n<\/tr>\n<tr>\n<td class=\"has-text-align-center\" data-align=\"center\">Grapefruit<\/td>\n<td>Medium<\/td>\n<td>Yellow<\/td>\n<td class=\"has-text-align-right\" data-align=\"right\">Round<\/td>\n<\/tr>\n<tr>\n<td class=\"has-text-align-center\" data-align=\"center\">Lemon<\/td>\n<td>Small<\/td>\n<td>Yellow<\/td>\n<td class=\"has-text-align-right\" data-align=\"right\">Round<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n\n\n<p class=\"wp-block-paragraph\">Suppose we want to calculate the information gained if we select the\u00a0<em>color<\/em>\u00a0variable. 3 out of the 6 records are yellow, 2 are green, and 1 is red. Proportionally, the probability of a yellow fruit is 3 \/ 6 = 0.5; 2 \/ 6 = 0.333&#8230; for green, and 1 \/ 6 = 0.1666\u2026 for red. Using the formula from above, we can calculate it like this:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>-([3\/6 * log<sub>2<\/sub>(3\/6)] + [2\/6 * log<sub>2<\/sub>(2\/6)] + [1\/6 * log<sub>2<\/sub>(1\/6)])<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>= 1.459148<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Likewise, we want to get the information gain for the&nbsp;<em>size<\/em>&nbsp;variable.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>-([3\/6 * log<sub>2<\/sub>(3\/6)] + [2\/6 * log<sub>2<\/sub>(2\/6)] + [1\/6 * log<sub>2<\/sub>(1\/6)])<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>= 1.459148<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">In this case, 3 \/ 6 of the fruits are medium-sized, 2 \/ 6 are small, 1 \/ 6 is big.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Lastly, we have the&nbsp;<em>shape<\/em>&nbsp;variable. Here, 5 \/ 6 of the fruits are round and 1 \/ 6 is thin.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>-([5\/6 * log<sub>2<\/sub>(5\/6)] + [1\/6 * log<sub>2<\/sub>(1\/6)])<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>= 0.650022<\/strong><\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-how-to-calculate-information-gain-in-r\"><strong>How to calculate information gain in R<\/strong><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">So, how do we calculate information gain in&nbsp;<a href=\"https:\/\/theautomatic.net\/category\/r\/\">R<\/a>? Thankfully, this is fairly simple to do using the&nbsp;<strong>FSelector<\/strong>&nbsp;package:<\/p>\n\n\n\n<p style=\"background-color:#fcfcdb;font-size:11px\" class=\"has-background\">\t\nlibrary(FSelector)<br><br>\n \ninfo <- data.frame(fruits = c(\"watermelon\", \"apple\", \"banana\", \"grape\", \"grapefruit\", \"lemon\"))<br>\ninfo$sizes <- c(\"big\", \"medium\", \"medium\",\"small\" ,\"medium\", \"small\")<br>\ninfo$colors <- c(\"green\", \"red\", \"yellow\", \"green\", \"yellow\", \"yellow\")<br>\ninfo$shapes <- c(\"round\", \"round\", \"thin\", \"round\", \"round\", \"round\")<br><br>\n \n# get information gain results<br>\ninformation.gain(formula(info), info)\n<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>Conclusion<\/strong><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">That\u2019s all for now! Information gain is just one of many possible feature importance methods, and I\u2019ll have more articles in the future to explore other possibilities. If you liked this post, please&nbsp;<a href=\"https:\/\/twitter.com\/AutomaticSource\">follow my blog on Twitter!<\/a>.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><em>Visit TheAutomatic.net to get additional insight on this topic: <a href=\"https:\/\/theautomatic.net\/2020\/02\/18\/how-is-information-gain-calculated\/\">https:\/\/theautomatic.net\/2020\/02\/18\/how-is-information-gain-calculated\/<\/a><\/em><\/p>\n","protected":false},"excerpt":{"rendered":"<p>TheAutomatic.net explores the concept of entropy and offers a way to calculate information gain in R using the FSelector package.<\/p>\n","protected":false},"author":388,"featured_media":50791,"comment_status":"closed","ping_status":"open","sticky":true,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":"","jetpack_post_was_ever_published":false},"categories":[339,343,338,341,352,344,342],"tags":[806,9253,4941,9252,9254,8710,6591,508],"contributors-categories":[13695],"class_list":["post-76745","post","type-post","status-publish","format-standard","has-post-thumbnail","category-data-science","category-programing-languages","category-ibkr-quant-news","category-quant-development","category-quant-north-america","category-quant-regions","category-r-development","tag-data-science","tag-entropy","tag-financial-mathematics","tag-fselector-package","tag-information-gain","tag-mathematical-finance","tag-rstats","tag-rstudio","contributors-categories-theautomatic-net"],"pp_statuses_selecting_workflow":false,"pp_workflow_action":"current","pp_status_selection":"publish","acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v26.9 (Yoast SEO v28.0) - https:\/\/yoast.com\/product\/yoast-seo-premium-wordpress\/ -->\n<title>How Is Information Gain Calculated? | IBKR Quant<\/title>\n<meta name=\"description\" content=\"TheAutomatic.net explores the concept of entropy and offers a way to calculate information gain in R using the FSelector package.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.interactivebrokers.com\/campus\/wp-json\/wp\/v2\/posts\/76745\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"How Is Information Gain Calculated? | IBKR Quant Blog\" \/>\n<meta property=\"og:description\" content=\"TheAutomatic.net explores the concept of entropy and offers a way to calculate information gain in R using the FSelector package.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/how-is-information-gain-calculated\/\" \/>\n<meta property=\"og:site_name\" content=\"IBKR Campus US\" \/>\n<meta property=\"article:published_time\" content=\"2021-02-19T15:25:44+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2022-11-21T14:47:08+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2020\/07\/machine-learning-sphere.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"900\" \/>\n\t<meta property=\"og:image:height\" content=\"550\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Andrew Treadway\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Andrew Treadway\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"4 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\n\t    \"@context\": \"https:\\\/\\\/schema.org\",\n\t    \"@graph\": [\n\t        {\n\t            \"@type\": \"NewsArticle\",\n\t            \"@id\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/how-is-information-gain-calculated\\\/#article\",\n\t            \"isPartOf\": {\n\t                \"@id\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/how-is-information-gain-calculated\\\/\"\n\t            },\n\t            \"author\": {\n\t                \"name\": \"Andrew Treadway\",\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#\\\/schema\\\/person\\\/d4018570a16fb867f1c08412fc9c64bc\"\n\t            },\n\t            \"headline\": \"How Is Information Gain Calculated?\",\n\t            \"datePublished\": \"2021-02-19T15:25:44+00:00\",\n\t            \"dateModified\": \"2022-11-21T14:47:08+00:00\",\n\t            \"mainEntityOfPage\": {\n\t                \"@id\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/how-is-information-gain-calculated\\\/\"\n\t            },\n\t            \"wordCount\": 604,\n\t            \"publisher\": {\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#organization\"\n\t            },\n\t            \"image\": {\n\t                \"@id\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/how-is-information-gain-calculated\\\/#primaryimage\"\n\t            },\n\t            \"thumbnailUrl\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2020\\\/07\\\/machine-learning-sphere.jpg\",\n\t            \"keywords\": [\n\t                \"Data Science\",\n\t                \"Entropy\",\n\t                \"Financial Mathematics\",\n\t                \"FSelector package\",\n\t                \"Information Gain\",\n\t                \"Mathematical Finance\",\n\t                \"rstats\",\n\t                \"RStudio\"\n\t            ],\n\t            \"articleSection\": [\n\t                \"Data Science\",\n\t                \"Programming Languages\",\n\t                \"Quant\",\n\t                \"Quant Development\",\n\t                \"Quant North America\",\n\t                \"Quant Regions\",\n\t                \"R Development\"\n\t            ],\n\t            \"inLanguage\": \"en-US\"\n\t        },\n\t        {\n\t            \"@type\": \"WebPage\",\n\t            \"@id\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/how-is-information-gain-calculated\\\/\",\n\t            \"url\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/how-is-information-gain-calculated\\\/\",\n\t            \"name\": \"How Is Information Gain Calculated? | IBKR Quant Blog\",\n\t            \"isPartOf\": {\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#website\"\n\t            },\n\t            \"primaryImageOfPage\": {\n\t                \"@id\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/how-is-information-gain-calculated\\\/#primaryimage\"\n\t            },\n\t            \"image\": {\n\t                \"@id\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/how-is-information-gain-calculated\\\/#primaryimage\"\n\t            },\n\t            \"thumbnailUrl\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2020\\\/07\\\/machine-learning-sphere.jpg\",\n\t            \"datePublished\": \"2021-02-19T15:25:44+00:00\",\n\t            \"dateModified\": \"2022-11-21T14:47:08+00:00\",\n\t            \"description\": \"TheAutomatic.net explores the concept of entropy and offers a way to calculate information gain in R using the FSelector package.\",\n\t            \"inLanguage\": \"en-US\",\n\t            \"potentialAction\": [\n\t                {\n\t                    \"@type\": \"ReadAction\",\n\t                    \"target\": [\n\t                        \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/how-is-information-gain-calculated\\\/\"\n\t                    ]\n\t                }\n\t            ]\n\t        },\n\t        {\n\t            \"@type\": \"ImageObject\",\n\t            \"inLanguage\": \"en-US\",\n\t            \"@id\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/how-is-information-gain-calculated\\\/#primaryimage\",\n\t            \"url\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2020\\\/07\\\/machine-learning-sphere.jpg\",\n\t            \"contentUrl\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2020\\\/07\\\/machine-learning-sphere.jpg\",\n\t            \"width\": 900,\n\t            \"height\": 550,\n\t            \"caption\": \"Machine Learning\"\n\t        },\n\t        {\n\t            \"@type\": \"WebSite\",\n\t            \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#website\",\n\t            \"url\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/\",\n\t            \"name\": \"IBKR Campus US\",\n\t            \"description\": \"Financial Education from Interactive Brokers\",\n\t            \"publisher\": {\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#organization\"\n\t            },\n\t            \"potentialAction\": [\n\t                {\n\t                    \"@type\": \"SearchAction\",\n\t                    \"target\": {\n\t                        \"@type\": \"EntryPoint\",\n\t                        \"urlTemplate\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/?s={search_term_string}\"\n\t                    },\n\t                    \"query-input\": {\n\t                        \"@type\": \"PropertyValueSpecification\",\n\t                        \"valueRequired\": true,\n\t                        \"valueName\": \"search_term_string\"\n\t                    }\n\t                }\n\t            ],\n\t            \"inLanguage\": \"en-US\"\n\t        },\n\t        {\n\t            \"@type\": \"Organization\",\n\t            \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#organization\",\n\t            \"name\": \"Interactive Brokers\",\n\t            \"alternateName\": \"IBKR\",\n\t            \"url\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/\",\n\t            \"logo\": {\n\t                \"@type\": \"ImageObject\",\n\t                \"inLanguage\": \"en-US\",\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#\\\/schema\\\/logo\\\/image\\\/\",\n\t                \"url\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2024\\\/05\\\/ibkr-campus-logo.jpg\",\n\t                \"contentUrl\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2024\\\/05\\\/ibkr-campus-logo.jpg\",\n\t                \"width\": 669,\n\t                \"height\": 669,\n\t                \"caption\": \"Interactive Brokers\"\n\t            },\n\t            \"image\": {\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#\\\/schema\\\/logo\\\/image\\\/\"\n\t            },\n\t            \"publishingPrinciples\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/about-ibkr-campus\\\/\",\n\t            \"ethicsPolicy\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/cyber-security-notice\\\/\"\n\t        },\n\t        {\n\t            \"@type\": \"Person\",\n\t            \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#\\\/schema\\\/person\\\/d4018570a16fb867f1c08412fc9c64bc\",\n\t            \"name\": \"Andrew Treadway\",\n\t            \"description\": \"Andrew Treadway currently works as a Senior Data Scientist, and has experience doing analytics, software automation, and ETL. He completed a master\u2019s degree in computer science \\\/ machine learning, and an undergraduate degree in pure mathematics. Connect with him on LinkedIn: https:\\\/\\\/www.linkedin.com\\\/in\\\/andrew-treadway-a3b19b103\\\/In addition to TheAutomatic.net blog, he also teaches in-person courses on Python and R through my NYC meetup: more details.\",\n\t            \"sameAs\": [\n\t                \"https:\\\/\\\/theautomatic.net\\\/about-me\\\/\"\n\t            ],\n\t            \"url\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/author\\\/andrewtreadway\\\/\"\n\t        }\n\t    ]\n\t}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"How Is Information Gain Calculated? | IBKR Quant","description":"TheAutomatic.net explores the concept of entropy and offers a way to calculate information gain in R using the FSelector package.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.interactivebrokers.com\/campus\/wp-json\/wp\/v2\/posts\/76745\/","og_locale":"en_US","og_type":"article","og_title":"How Is Information Gain Calculated? | IBKR Quant Blog","og_description":"TheAutomatic.net explores the concept of entropy and offers a way to calculate information gain in R using the FSelector package.","og_url":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/how-is-information-gain-calculated\/","og_site_name":"IBKR Campus US","article_published_time":"2021-02-19T15:25:44+00:00","article_modified_time":"2022-11-21T14:47:08+00:00","og_image":[{"width":900,"height":550,"url":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2020\/07\/machine-learning-sphere.jpg","type":"image\/jpeg"}],"author":"Andrew Treadway","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Andrew Treadway","Est. reading time":"4 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"NewsArticle","@id":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/how-is-information-gain-calculated\/#article","isPartOf":{"@id":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/how-is-information-gain-calculated\/"},"author":{"name":"Andrew Treadway","@id":"https:\/\/ibkrcampus.com\/campus\/#\/schema\/person\/d4018570a16fb867f1c08412fc9c64bc"},"headline":"How Is Information Gain Calculated?","datePublished":"2021-02-19T15:25:44+00:00","dateModified":"2022-11-21T14:47:08+00:00","mainEntityOfPage":{"@id":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/how-is-information-gain-calculated\/"},"wordCount":604,"publisher":{"@id":"https:\/\/ibkrcampus.com\/campus\/#organization"},"image":{"@id":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/how-is-information-gain-calculated\/#primaryimage"},"thumbnailUrl":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2020\/07\/machine-learning-sphere.jpg","keywords":["Data Science","Entropy","Financial Mathematics","FSelector package","Information Gain","Mathematical Finance","rstats","RStudio"],"articleSection":["Data Science","Programming Languages","Quant","Quant Development","Quant North America","Quant Regions","R Development"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/how-is-information-gain-calculated\/","url":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/how-is-information-gain-calculated\/","name":"How Is Information Gain Calculated? | IBKR Quant Blog","isPartOf":{"@id":"https:\/\/ibkrcampus.com\/campus\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/how-is-information-gain-calculated\/#primaryimage"},"image":{"@id":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/how-is-information-gain-calculated\/#primaryimage"},"thumbnailUrl":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2020\/07\/machine-learning-sphere.jpg","datePublished":"2021-02-19T15:25:44+00:00","dateModified":"2022-11-21T14:47:08+00:00","description":"TheAutomatic.net explores the concept of entropy and offers a way to calculate information gain in R using the FSelector package.","inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/how-is-information-gain-calculated\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/how-is-information-gain-calculated\/#primaryimage","url":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2020\/07\/machine-learning-sphere.jpg","contentUrl":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2020\/07\/machine-learning-sphere.jpg","width":900,"height":550,"caption":"Machine Learning"},{"@type":"WebSite","@id":"https:\/\/ibkrcampus.com\/campus\/#website","url":"https:\/\/ibkrcampus.com\/campus\/","name":"IBKR Campus US","description":"Financial Education from Interactive Brokers","publisher":{"@id":"https:\/\/ibkrcampus.com\/campus\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/ibkrcampus.com\/campus\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/ibkrcampus.com\/campus\/#organization","name":"Interactive Brokers","alternateName":"IBKR","url":"https:\/\/ibkrcampus.com\/campus\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/ibkrcampus.com\/campus\/#\/schema\/logo\/image\/","url":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/05\/ibkr-campus-logo.jpg","contentUrl":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/05\/ibkr-campus-logo.jpg","width":669,"height":669,"caption":"Interactive Brokers"},"image":{"@id":"https:\/\/ibkrcampus.com\/campus\/#\/schema\/logo\/image\/"},"publishingPrinciples":"https:\/\/www.interactivebrokers.com\/campus\/about-ibkr-campus\/","ethicsPolicy":"https:\/\/www.interactivebrokers.com\/campus\/cyber-security-notice\/"},{"@type":"Person","@id":"https:\/\/ibkrcampus.com\/campus\/#\/schema\/person\/d4018570a16fb867f1c08412fc9c64bc","name":"Andrew Treadway","description":"Andrew Treadway currently works as a Senior Data Scientist, and has experience doing analytics, software automation, and ETL. He completed a master\u2019s degree in computer science \/ machine learning, and an undergraduate degree in pure mathematics. Connect with him on LinkedIn: https:\/\/www.linkedin.com\/in\/andrew-treadway-a3b19b103\/In addition to TheAutomatic.net blog, he also teaches in-person courses on Python and R through my NYC meetup: more details.","sameAs":["https:\/\/theautomatic.net\/about-me\/"],"url":"https:\/\/www.interactivebrokers.com\/campus\/author\/andrewtreadway\/"}]}},"jetpack_featured_media_url":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2020\/07\/machine-learning-sphere.jpg","_links":{"self":[{"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/posts\/76745","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/users\/388"}],"replies":[{"embeddable":true,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/comments?post=76745"}],"version-history":[{"count":0,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/posts\/76745\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/media\/50791"}],"wp:attachment":[{"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/media?parent=76745"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/categories?post=76745"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/tags?post=76745"},{"taxonomy":"contributors-categories","embeddable":true,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/contributors-categories?post=76745"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}