{"id":97892,"date":"2021-08-10T10:51:00","date_gmt":"2021-08-10T14:51:00","guid":{"rendered":"https:\/\/ibkrcampus.com\/?p=97892"},"modified":"2023-02-16T16:00:47","modified_gmt":"2023-02-16T21:00:47","slug":"how-to-fill-gaps-in-large-stock-data-universes-using-tidyr-and-dplyr-part-i","status":"publish","type":"post","link":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/how-to-fill-gaps-in-large-stock-data-universes-using-tidyr-and-dplyr-part-i\/","title":{"rendered":"How to Fill Gaps in Large Stock Data Universes Using tidyr and dplyr &#8211; Part I"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">When you\u2019re working with&nbsp;<a href=\"https:\/\/robotwealth.com\/how-to-get-historical-spx-constituents-data-for-free\/\" target=\"_blank\" rel=\"noreferrer noopener\">large universes of stock data<\/a>&nbsp;you\u2019ll come across a lot of challenges:<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li>Stocks pay dividends and other distributions that have to be accounted for.<\/li><li>Stocks are subject to splits and other corporate actions which also have to be accounted for.<\/li><li>New stocks are listed all the time \u2013 you won\u2019t have as much history for these stocks as for other stocks.<\/li><li>Stocks are delisted, and many datasets do not include the price history of delisted stocks<\/li><li>Stocks can be suspended or halted for a period of time, leading to trading gaps.<\/li><li>Companies grow and shrink: the \u201ctop 100 stocks by market cap\u201d in 1990 looks very different to the same group in 2020; \u201cgrowth stocks\u201d in 1990 look very different to \u201cgrowth stocks\u201d in 2020 etc.<\/li><\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">The challenges are well understood, but dealing with them is not always straightforward.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">One significant challenge is gaps in data.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Quant analysis gets very hard if you have missing or misaligned data.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">If you\u2019re working with a universe of 1,000 stocks life is a lot easier if you have an observation for each stock for each trading date, regardless of whether it actually traded that day. That way:<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li>you can always do look-ups by date<\/li><li>any grouped aggregations or rolling window aggregations will be operating on the date range for every ticker<\/li><li>you can easily sense check the size of your data to have&nbsp;<code>trading_days * number_of_stocks<\/code>&nbsp;rows.<\/li><\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">If you work with \u201cwide\u201d matrix-like data, these challenges are obvious because you have one row for every date in your data set, and the columns represent an observation for each ticker.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">We usually work with&nbsp;<a href=\"https:\/\/robotwealth.com\/financial-data-manipulation-in-dplyr-for-quant-traders\/\" target=\"_blank\" rel=\"noreferrer noopener\">long or \u201ctidy\u201d data<\/a>&nbsp;\u2013 where each observation is an observation for a stock for a given day.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">How do we work productively in this data, whilst still ensuring that we fill in any gaps in our long data with NAs?<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The tidyverse makes this very straightforward. Let me show you!<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">First, here\u2019s some dummy data to illustrate the problem:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>library(tidyverse)\ntestdata &lt;- tibble(date = c(1,1,2,2,2,3,3),\n                       ticker = c('AMZN','FB','AMZN','FB','TSLA','AMZN','TSLA'),\n                       returns = 1:7 \/ 100)\ntestdata<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>## # A tibble: 7 x 3\n##    date ticker returns\n##   &lt;dbl&gt; &lt;chr&gt;    &lt;dbl&gt;\n## 1     1 AMZN      0.01\n## 2     1 FB        0.02\n## 3     2 AMZN      0.03\n## 4     2 FB        0.04\n## 5     2 TSLA      0.05\n## 6     3 AMZN      0.06\n## 7     3 TSLA      0.07<\/code><\/pre>\n\n\n\n<ul class=\"wp-block-list\"><li>TSLA is missing from date 1 as it only started trading after the others<\/li><li>FB is missing from date 3 as it was put on trading halt after Citron Research hacked into Zuck\u2019s memory banks<\/li><\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Ideally we want a row for every date for every stock \u2013 with returns set to NA in the case where data is missing.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">That way we can always look up a price by date. And we can always be sure that any grouped operations by ticker return the same size data set.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Turns out that the&nbsp;<code>tidyr::complete<\/code>&nbsp;function is exactly what we\u2019re looking for. It turns&nbsp;<em>implicit<\/em>&nbsp;missing values \u2013 like the returns for TSLA on date 1 and FB on date 3 \u2013 into&nbsp;<em>explicit<\/em>&nbsp;missing values:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>tidydata &lt;- testdata %&gt;%\n  complete(date, ticker)\ntidydata<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>## # A tibble: 9 x 3\n##    date ticker returns\n##   &lt;dbl&gt; &lt;chr&gt;    &lt;dbl&gt;\n## 1     1 AMZN      0.01\n## 2     1 FB        0.02\n## 3     1 TSLA     NA   \n## 4     2 AMZN      0.03\n## 5     2 FB        0.04\n## 6     2 TSLA      0.05\n## 7     3 AMZN      0.06\n## 8     3 FB       NA   \n## 9     3 TSLA      0.07<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Easy!<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Now we have a row for every date for every stock.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Now we can safely do grouped aggregations by ticker, on the understanding that the data is the same size for all tickers, and we\u2019ve removed one large source of potential analysis mishap\u2026<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>tidydata %&gt;%\n  group_by(ticker) %&gt;%\n  summarise(count = n())<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>## # A tibble: 3 x 2\n##   ticker count\n##   &lt;chr&gt;  &lt;int&gt;\n## 1 AMZN       3\n## 2 FB         3\n## 3 TSLA       3<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\"><em>Visit Robot Wealth website for additional insight on this topic and to download the complete set of scripts: <a href=\"https:\/\/robotwealth.com\/how-to-fill-gaps-in-large-stock-data-universes-using-tidyr-and-dplyr\/\">https:\/\/robotwealth.com\/how-to-fill-gaps-in-large-stock-data-universes-using-tidyr-and-dplyr\/<\/a><\/em><\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><em>Past performance is not indicative of future results.<\/em><\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><em>Any stock, options or futures symbols displayed are for illustrative purposes only and are not intended to portray recommendations.<\/em><\/p>\n","protected":false},"excerpt":{"rendered":"<p>How do we work productively in this data, whilst still ensuring that we fill in any gaps in our long data with NAs? The tidyverse makes this very straightforward. <\/p>\n","protected":false},"author":421,"featured_media":41647,"comment_status":"closed","ping_status":"open","sticky":true,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":"","jetpack_post_was_ever_published":false},"categories":[339,343,338,350,341,344,342],"tags":[806,2535,6591,508,10114,8594,1045],"contributors-categories":[13676],"class_list":["post-97892","post","type-post","status-publish","format-standard","has-post-thumbnail","category-data-science","category-programing-languages","category-ibkr-quant-news","category-quant-asia-pacific","category-quant-development","category-quant-regions","category-r-development","tag-data-science","tag-dplyr","tag-rstats","tag-rstudio","tag-tidydata","tag-tidyr","tag-tidyverse","contributors-categories-robot-wealth"],"pp_statuses_selecting_workflow":false,"pp_workflow_action":"current","pp_status_selection":"publish","acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v26.9 (Yoast SEO v27.8) - https:\/\/yoast.com\/product\/yoast-seo-premium-wordpress\/ -->\n<title>How to Fill Gaps in Large Stock Data Universes Using tidyr and dplyr &#8211; Part I<\/title>\n<meta name=\"description\" content=\"How do we work productively in this data, whilst still ensuring that we fill in any gaps in our long data with NAs? The tidyverse makes this very...\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.interactivebrokers.com\/campus\/wp-json\/wp\/v2\/posts\/97892\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"How to Fill Gaps in Large Stock Data Universes Using tidyr and dplyr - Part I | IBKR Quant Blog\" \/>\n<meta property=\"og:description\" content=\"How do we work productively in this data, whilst still ensuring that we fill in any gaps in our long data with NAs? The tidyverse makes this very straightforward.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/how-to-fill-gaps-in-large-stock-data-universes-using-tidyr-and-dplyr-part-i\/\" \/>\n<meta property=\"og:site_name\" content=\"IBKR Campus US\" \/>\n<meta property=\"article:published_time\" content=\"2021-08-10T14:51:00+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2023-02-16T21:00:47+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2020\/04\/world-social-city.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"900\" \/>\n\t<meta property=\"og:image:height\" content=\"550\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Robot James\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Robot James\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"4 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\n\t    \"@context\": \"https:\\\/\\\/schema.org\",\n\t    \"@graph\": [\n\t        {\n\t            \"@type\": \"NewsArticle\",\n\t            \"@id\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/how-to-fill-gaps-in-large-stock-data-universes-using-tidyr-and-dplyr-part-i\\\/#article\",\n\t            \"isPartOf\": {\n\t                \"@id\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/how-to-fill-gaps-in-large-stock-data-universes-using-tidyr-and-dplyr-part-i\\\/\"\n\t            },\n\t            \"author\": {\n\t                \"name\": \"Robot James\",\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#\\\/schema\\\/person\\\/0fcf297a219d9485e2efa476d632b468\"\n\t            },\n\t            \"headline\": \"How to Fill Gaps in Large Stock Data Universes Using tidyr and dplyr &#8211; Part I\",\n\t            \"datePublished\": \"2021-08-10T14:51:00+00:00\",\n\t            \"dateModified\": \"2023-02-16T21:00:47+00:00\",\n\t            \"mainEntityOfPage\": {\n\t                \"@id\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/how-to-fill-gaps-in-large-stock-data-universes-using-tidyr-and-dplyr-part-i\\\/\"\n\t            },\n\t            \"wordCount\": 568,\n\t            \"publisher\": {\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#organization\"\n\t            },\n\t            \"image\": {\n\t                \"@id\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/how-to-fill-gaps-in-large-stock-data-universes-using-tidyr-and-dplyr-part-i\\\/#primaryimage\"\n\t            },\n\t            \"thumbnailUrl\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2020\\\/04\\\/world-social-city.jpg\",\n\t            \"keywords\": [\n\t                \"Data Science\",\n\t                \"dplyr\",\n\t                \"rstats\",\n\t                \"RStudio\",\n\t                \"tidydata\",\n\t                \"tidyr\",\n\t                \"tidyverse\"\n\t            ],\n\t            \"articleSection\": [\n\t                \"Data Science\",\n\t                \"Programming Languages\",\n\t                \"Quant\",\n\t                \"Quant Asia Pacific\",\n\t                \"Quant Development\",\n\t                \"Quant Regions\",\n\t                \"R Development\"\n\t            ],\n\t            \"inLanguage\": \"en-US\"\n\t        },\n\t        {\n\t            \"@type\": \"WebPage\",\n\t            \"@id\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/how-to-fill-gaps-in-large-stock-data-universes-using-tidyr-and-dplyr-part-i\\\/\",\n\t            \"url\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/how-to-fill-gaps-in-large-stock-data-universes-using-tidyr-and-dplyr-part-i\\\/\",\n\t            \"name\": \"How to Fill Gaps in Large Stock Data Universes Using tidyr and dplyr - Part I | IBKR Quant Blog\",\n\t            \"isPartOf\": {\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#website\"\n\t            },\n\t            \"primaryImageOfPage\": {\n\t                \"@id\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/how-to-fill-gaps-in-large-stock-data-universes-using-tidyr-and-dplyr-part-i\\\/#primaryimage\"\n\t            },\n\t            \"image\": {\n\t                \"@id\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/how-to-fill-gaps-in-large-stock-data-universes-using-tidyr-and-dplyr-part-i\\\/#primaryimage\"\n\t            },\n\t            \"thumbnailUrl\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2020\\\/04\\\/world-social-city.jpg\",\n\t            \"datePublished\": \"2021-08-10T14:51:00+00:00\",\n\t            \"dateModified\": \"2023-02-16T21:00:47+00:00\",\n\t            \"description\": \"How do we work productively in this data, whilst still ensuring that we fill in any gaps in our long data with NAs? The tidyverse makes this very straightforward.\",\n\t            \"inLanguage\": \"en-US\",\n\t            \"potentialAction\": [\n\t                {\n\t                    \"@type\": \"ReadAction\",\n\t                    \"target\": [\n\t                        \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/how-to-fill-gaps-in-large-stock-data-universes-using-tidyr-and-dplyr-part-i\\\/\"\n\t                    ]\n\t                }\n\t            ]\n\t        },\n\t        {\n\t            \"@type\": \"ImageObject\",\n\t            \"inLanguage\": \"en-US\",\n\t            \"@id\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/how-to-fill-gaps-in-large-stock-data-universes-using-tidyr-and-dplyr-part-i\\\/#primaryimage\",\n\t            \"url\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2020\\\/04\\\/world-social-city.jpg\",\n\t            \"contentUrl\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2020\\\/04\\\/world-social-city.jpg\",\n\t            \"width\": 900,\n\t            \"height\": 550,\n\t            \"caption\": \"Quant\"\n\t        },\n\t        {\n\t            \"@type\": \"WebSite\",\n\t            \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#website\",\n\t            \"url\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/\",\n\t            \"name\": \"IBKR Campus US\",\n\t            \"description\": \"Financial Education from Interactive Brokers\",\n\t            \"publisher\": {\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#organization\"\n\t            },\n\t            \"potentialAction\": [\n\t                {\n\t                    \"@type\": \"SearchAction\",\n\t                    \"target\": {\n\t                        \"@type\": \"EntryPoint\",\n\t                        \"urlTemplate\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/?s={search_term_string}\"\n\t                    },\n\t                    \"query-input\": {\n\t                        \"@type\": \"PropertyValueSpecification\",\n\t                        \"valueRequired\": true,\n\t                        \"valueName\": \"search_term_string\"\n\t                    }\n\t                }\n\t            ],\n\t            \"inLanguage\": \"en-US\"\n\t        },\n\t        {\n\t            \"@type\": \"Organization\",\n\t            \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#organization\",\n\t            \"name\": \"Interactive Brokers\",\n\t            \"alternateName\": \"IBKR\",\n\t            \"url\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/\",\n\t            \"logo\": {\n\t                \"@type\": \"ImageObject\",\n\t                \"inLanguage\": \"en-US\",\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#\\\/schema\\\/logo\\\/image\\\/\",\n\t                \"url\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2024\\\/05\\\/ibkr-campus-logo.jpg\",\n\t                \"contentUrl\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2024\\\/05\\\/ibkr-campus-logo.jpg\",\n\t                \"width\": 669,\n\t                \"height\": 669,\n\t                \"caption\": \"Interactive Brokers\"\n\t            },\n\t            \"image\": {\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#\\\/schema\\\/logo\\\/image\\\/\"\n\t            },\n\t            \"publishingPrinciples\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/about-ibkr-campus\\\/\",\n\t            \"ethicsPolicy\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/cyber-security-notice\\\/\"\n\t        },\n\t        {\n\t            \"@type\": \"Person\",\n\t            \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#\\\/schema\\\/person\\\/0fcf297a219d9485e2efa476d632b468\",\n\t            \"name\": \"Robot James\",\n\t            \"url\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/author\\\/robotjames\\\/\"\n\t        }\n\t    ]\n\t}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"How to Fill Gaps in Large Stock Data Universes Using tidyr and dplyr &#8211; Part I","description":"How do we work productively in this data, whilst still ensuring that we fill in any gaps in our long data with NAs? The tidyverse makes this very...","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.interactivebrokers.com\/campus\/wp-json\/wp\/v2\/posts\/97892\/","og_locale":"en_US","og_type":"article","og_title":"How to Fill Gaps in Large Stock Data Universes Using tidyr and dplyr - Part I | IBKR Quant Blog","og_description":"How do we work productively in this data, whilst still ensuring that we fill in any gaps in our long data with NAs? The tidyverse makes this very straightforward.","og_url":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/how-to-fill-gaps-in-large-stock-data-universes-using-tidyr-and-dplyr-part-i\/","og_site_name":"IBKR Campus US","article_published_time":"2021-08-10T14:51:00+00:00","article_modified_time":"2023-02-16T21:00:47+00:00","og_image":[{"width":900,"height":550,"url":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2020\/04\/world-social-city.jpg","type":"image\/jpeg"}],"author":"Robot James","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Robot James","Est. reading time":"4 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"NewsArticle","@id":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/how-to-fill-gaps-in-large-stock-data-universes-using-tidyr-and-dplyr-part-i\/#article","isPartOf":{"@id":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/how-to-fill-gaps-in-large-stock-data-universes-using-tidyr-and-dplyr-part-i\/"},"author":{"name":"Robot James","@id":"https:\/\/ibkrcampus.com\/campus\/#\/schema\/person\/0fcf297a219d9485e2efa476d632b468"},"headline":"How to Fill Gaps in Large Stock Data Universes Using tidyr and dplyr &#8211; Part I","datePublished":"2021-08-10T14:51:00+00:00","dateModified":"2023-02-16T21:00:47+00:00","mainEntityOfPage":{"@id":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/how-to-fill-gaps-in-large-stock-data-universes-using-tidyr-and-dplyr-part-i\/"},"wordCount":568,"publisher":{"@id":"https:\/\/ibkrcampus.com\/campus\/#organization"},"image":{"@id":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/how-to-fill-gaps-in-large-stock-data-universes-using-tidyr-and-dplyr-part-i\/#primaryimage"},"thumbnailUrl":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2020\/04\/world-social-city.jpg","keywords":["Data Science","dplyr","rstats","RStudio","tidydata","tidyr","tidyverse"],"articleSection":["Data Science","Programming Languages","Quant","Quant Asia Pacific","Quant Development","Quant Regions","R Development"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/how-to-fill-gaps-in-large-stock-data-universes-using-tidyr-and-dplyr-part-i\/","url":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/how-to-fill-gaps-in-large-stock-data-universes-using-tidyr-and-dplyr-part-i\/","name":"How to Fill Gaps in Large Stock Data Universes Using tidyr and dplyr - Part I | IBKR Quant Blog","isPartOf":{"@id":"https:\/\/ibkrcampus.com\/campus\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/how-to-fill-gaps-in-large-stock-data-universes-using-tidyr-and-dplyr-part-i\/#primaryimage"},"image":{"@id":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/how-to-fill-gaps-in-large-stock-data-universes-using-tidyr-and-dplyr-part-i\/#primaryimage"},"thumbnailUrl":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2020\/04\/world-social-city.jpg","datePublished":"2021-08-10T14:51:00+00:00","dateModified":"2023-02-16T21:00:47+00:00","description":"How do we work productively in this data, whilst still ensuring that we fill in any gaps in our long data with NAs? The tidyverse makes this very straightforward.","inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/how-to-fill-gaps-in-large-stock-data-universes-using-tidyr-and-dplyr-part-i\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/how-to-fill-gaps-in-large-stock-data-universes-using-tidyr-and-dplyr-part-i\/#primaryimage","url":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2020\/04\/world-social-city.jpg","contentUrl":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2020\/04\/world-social-city.jpg","width":900,"height":550,"caption":"Quant"},{"@type":"WebSite","@id":"https:\/\/ibkrcampus.com\/campus\/#website","url":"https:\/\/ibkrcampus.com\/campus\/","name":"IBKR Campus US","description":"Financial Education from Interactive Brokers","publisher":{"@id":"https:\/\/ibkrcampus.com\/campus\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/ibkrcampus.com\/campus\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/ibkrcampus.com\/campus\/#organization","name":"Interactive Brokers","alternateName":"IBKR","url":"https:\/\/ibkrcampus.com\/campus\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/ibkrcampus.com\/campus\/#\/schema\/logo\/image\/","url":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/05\/ibkr-campus-logo.jpg","contentUrl":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/05\/ibkr-campus-logo.jpg","width":669,"height":669,"caption":"Interactive Brokers"},"image":{"@id":"https:\/\/ibkrcampus.com\/campus\/#\/schema\/logo\/image\/"},"publishingPrinciples":"https:\/\/www.interactivebrokers.com\/campus\/about-ibkr-campus\/","ethicsPolicy":"https:\/\/www.interactivebrokers.com\/campus\/cyber-security-notice\/"},{"@type":"Person","@id":"https:\/\/ibkrcampus.com\/campus\/#\/schema\/person\/0fcf297a219d9485e2efa476d632b468","name":"Robot James","url":"https:\/\/www.interactivebrokers.com\/campus\/author\/robotjames\/"}]}},"jetpack_featured_media_url":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2020\/04\/world-social-city.jpg","_links":{"self":[{"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/posts\/97892","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/users\/421"}],"replies":[{"embeddable":true,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/comments?post=97892"}],"version-history":[{"count":0,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/posts\/97892\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/media\/41647"}],"wp:attachment":[{"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/media?parent=97892"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/categories?post=97892"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/tags?post=97892"},{"taxonomy":"contributors-categories","embeddable":true,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/contributors-categories?post=97892"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}