{"id":227216,"date":"2025-07-17T08:15:26","date_gmt":"2025-07-17T12:15:26","guid":{"rendered":"https:\/\/ibkrcampus.com\/campus\/?p=227216"},"modified":"2025-07-18T17:11:11","modified_gmt":"2025-07-18T21:11:11","slug":"how-to-ingest-premium-market-data-with-zipline-reloaded","status":"publish","type":"post","link":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/how-to-ingest-premium-market-data-with-zipline-reloaded\/","title":{"rendered":"How to Ingest Premium Market Data with Zipline Reloaded"},"content":{"rendered":"\n

The article “How to Ingest Premium Market Data with Zipline Reloaded” was first published on PyQuant News<\/a> blog<\/em><\/p>\n\n\n\n

The author of this article is not affiliated with Interactive Brokers. The software is in no way affiliated, endorsed, or approved by Interactive Brokers or any of its affiliates. It comes with absolutely no warranty and should not be used in actual trading unless the user can read and understand the source. The IBKR API team does not support this software.<\/strong><\/p>\n\n\n\n

How to ingest premium market data with Zipline Reloaded<\/h2>\n\n\n\n

This article explains how to build the two Python scripts you need to use premium data to create a custom data bundle using Zipline Reloaded.<\/p>\n\n\n\n

Step 1: Subscribe to premium data<\/h2>\n\n\n\n

By now you should already have an account with Nasdaq Data Link. If not, head over to https:\/\/data.nasdaq.com<\/a> and set one up.<\/p>\n\n\n\n

You\u2019re looking for QuoteMedia End of Day US Stock Prices. This product offers end-of-day prices, dividends, adjustments and splits for US publicly traded stocks with history to 1996. Prices are provided both adjusted and unadjusted. The product covers all stocks with primary listing on NASDAQ, AMEX, NYSE, and ARCA.<\/p>\n\n\n\n

You can find the page to subscribe here: https:\/\/data.nasdaq.com\/databases\/EOD\/data<\/a><\/p>\n\n\n\n

Once subscribed, you\u2019ll be able to use it through your API key.<\/p>\n\n\n\n

Step 2: Create\/Edit extension.py<\/strong><\/h2>\n\n\n\n

Now we\u2019ll create the two files we need to create the bundle.<\/p>\n\n\n\n

For Windows users<\/h3>\n\n\n\n

In the .zipline<\/strong> directory, you will store the extension.py<\/strong> file, which informs Zipline about the custom data bundle.<\/p>\n\n\n\n

    \n
  1. Open the File Explorer and navigate to your home directory. You should find the .zipline<\/strong> folder there. If you’re not sure where your home directory is, it’s usually C:\\Users\\[YourUsername]<\/strong>.<\/li>\n\n\n\n
  2. Open the .zipline<\/strong> folder.<\/li>\n\n\n\n
  3. Right-click within the folder, select New<\/strong>, then choose Text Document<\/strong>. Rename the newly created file to extension.py<\/strong>. Make sure you change the file extension from .txt<\/strong> to .py<\/strong>.<\/li>\n<\/ol>\n\n\n\n

    Note:<\/strong> If you can’t see file extensions in your File Explorer, you’ll need to enable them. To do this, click on the View<\/strong> tab in File Explorer, and then check the box for File name extensions<\/strong>.<\/p>\n\n\n\n

    For Mac\/Linux\/Unix users<\/h3>\n\n\n\n
      \n
    1. Open Terminal<\/strong>: You can do this by searching for “Terminal” using Spotlight (Cmd + Space) on Mac or by accessing it from the Applications folder.<\/li>\n\n\n\n
    2. Navigate to .zipline Directory<\/strong>: By default, the terminal opens in your home directory. To ensure you’re in the home directory and then navigate to the .zipline<\/strong> directory, you can use the following commands:cd ~ cd .zipline<\/code><\/li>\n\n\n\n
    3. Create\/Edit the extension.py File<\/strong>:\n
        \n
      • If the file doesn’t exist: You can create it using the touch<\/strong> command followed by opening it with a text editor of your choice. touch extension.py<\/code><\/li>\n\n\n\n
      • If the file already exists: Simply open it with a text editor.<\/li>\n<\/ul>\n<\/li>\n<\/ol>\n\n\n\n

        For all users<\/h3>\n\n\n\n

        Within the editor, you can now proceed to input or edit the necessary content. In the file, add the following content:<\/p>\n\n\n\n

        import sys\nfrom pathlib import Path\nsys.path.append(Path(\"~\", \".zipline\").expanduser().as_posix())\n\nfrom zipline.data.bundles import register\n\nfrom daily_us_equities import daily_us_equities_bundle\n\nregister(\"quotemedia\", daily_us_equities_bundle, calendar_name=\"XNYS\")<\/pre>\n\n\n\n
        Save and close the file<\/p>\n\n\n\n
        Step 3: Create the code to build the bundle<\/h2>\n\n\n\n
        Use the instructions above to create a file called daily_us_equities.py.<\/strong><\/p>\n\n\n\n
        In the file, add the following code exactly as is (do not alter!):<\/p>\n\n\n\n
        import time\nfrom io import BytesIO\nfrom zipfile import ZipFile\n\nimport numpy as np\nimport pandas as pd\nimport requests\nfrom click import progressbar\nfrom logbook import Logger\nfrom six import iteritems\nfrom six.moves.urllib.parse import urlencode\n\nlog = Logger(__name__)\n\nDATA_START_DATE = \"2000-01-01\"\nONE_MEGABYTE = 1024 * 1024\nDATALINK_DATA_URL = \"https:\/\/data.nasdaq.com\/api\/v3\/datatables\/QUOTEMEDIA\/PRICES\"\nMAX_DOWNLOAD_TRIES = 5\n\n\ndef format_metadata_url(api_key):\n    \"\"\"Build the query URL for Quandl WIKI Prices metadata.\"\"\"\n    columns = \",\".join(\n        [\n            \"ticker\",\n            \"date\",\n            \"open\",\n            \"high\",\n            \"low\",\n            \"close\",\n            \"volume\",\n            \"dividend\",\n            \"split\",\n        ]\n    )\n\n    query_params = [\n        (\"date.gte\", DATA_START_DATE),\n        (\"api_key\", api_key),\n        (\"qopts.export\", \"true\"),\n        (\"qopts.columns\", columns),\n    ]\n    return f\"{DATALINK_DATA_URL}?{urlencode(query_params)}\"\n\n\ndef fetch_download_link(table_url, max_download_tries=MAX_DOWNLOAD_TRIES):\n    log.info(f\"Attempting to fetch download link with ...\")\n\n    status = None\n    cnt = 0\n\n    while status != \"fresh\" and cnt < max_download_tries:\n        log.info(f\"Fetching download link...\")\n        try:\n            resp = requests.get(table_url)\n            resp.raise_for_status()\n        except:\n            log.info(\"Failed to get download link from Quandl\")\n\n        payload = resp.json()\n\n        status = payload[\"datatable_bulk_download\"][\"file\"][\"status\"]\n\n        if status == \"fresh\":\n            link = payload[\"datatable_bulk_download\"][\"file\"][\"link\"]\n            log.info(f\"Status is {status}. Returning download link: {link}\")\n            return link\n\n        log.info(f\"Status is {status}. Retrying in 10 seconds...\")\n\n        time.sleep(10)\n\n\ndef load_data_table(file, index_col=None):\n    \"\"\"Load data table from zip file provided by Quandl.\"\"\"\n    with ZipFile(file) as zip_file:\n        file_names = zip_file.namelist()\n        assert len(file_names) == 1, \"Expected a single file from Quandl.\"\n        eod_prices = file_names.pop()\n        with zip_file.open(eod_prices) as table_file:\n            log.info(\"Parsing raw data.\")\n            data_table = pd.read_csv(\n                table_file,\n                header=0,\n                names=[\n                    \"ticker\",\n                    \"date\",\n                    \"open\",\n                    \"high\",\n                    \"low\",\n                    \"close\",\n                    \"volume\",\n                    \"dividend\",\n                    \"split\",\n                ],\n                parse_dates=[\"date\"],\n                index_col=index_col,\n                usecols=[\n                    \"ticker\",\n                    \"date\",\n                    \"open\",\n                    \"high\",\n                    \"low\",\n                    \"close\",\n                    \"volume\",\n                    \"dividend\",\n                    \"split\",\n                ]\n            ).rename(\n                columns={\n                    \"ticker\": \"symbol\",\n                    \"dividend\": \"ex_dividend\",\n                    \"split\": \"split_ratio\",\n                }\n            )\n    data_table.volume = data_table.volume.astype(\"Int64\")\n    return data_table\n\n\ndef fetch_data_table(api_key):\n    \"\"\"Fetch WIKI Prices data table from Quandl\"\"\"\n    log.info(f\"Fetching data table...\")\n\n    table_url = format_metadata_url(api_key)\n    download_link = fetch_download_link(table_url)\n    raw_file = download_with_progress(download_link, chunk_size=ONE_MEGABYTE)\n\n    return load_data_table(file=raw_file)\n\n\ndef gen_asset_metadata(data, show_progress):\n    if show_progress:\n        log.info(\"Generating asset metadata.\")\n\n    data = data.groupby(by=\"symbol\").agg({\"date\": [\"min\", \"max\"]})\n    data.reset_index(inplace=True)\n    data[\"start_date\"] = data.date.min(axis=1)\n    data[\"end_date\"] = data.date.max(axis=1)\n    del data[\"date\"]\n    data.columns = data.columns.get_level_values(0)\n\n    data[\"exchange\"] = \"QUOTEMEDIA\"\n    data[\"auto_close_date\"] = data[\"end_date\"].values + pd.Timedelta(days=1)\n    return data\n\n\ndef parse_splits(data, show_progress):\n    if show_progress:\n        log.info(\"Parsing split data.\")\n\n    data[\"split_ratio\"] = 1.0 \/ data.split_ratio\n    data.rename(\n        columns={\"split_ratio\": \"ratio\", \"date\": \"effective_date\"},\n        inplace=True,\n        copy=False,\n    )\n    return data\n\n\ndef parse_dividends(data, show_progress):\n    if show_progress:\n        log.info(\"Parsing dividend data.\")\n\n    data[\"record_date\"] = data[\"declared_date\"] = data[\"pay_date\"] = pd.NaT\n    data.rename(\n        columns={\"ex_dividend\": \"amount\", \"date\": \"ex_date\"}, inplace=True, copy=False\n    )\n    return data\n\n\ndef parse_pricing_and_vol(data, sessions, symbol_map):\n    for asset_id, symbol in iteritems(symbol_map):\n        asset_data = (\n            data.xs(symbol, level=1).reindex(sessions.tz_localize(None)).fillna(0.0)\n        )\n        yield asset_id, asset_data\n\n\ndef daily_us_equities_bundle(\n    environ,\n    asset_db_writer,\n    minute_bar_writer,\n    daily_bar_writer,\n    adjustment_writer,\n    calendar,\n    start_session,\n    end_session,\n    cache,\n    show_progress,\n    output_dir,\n):\n    \"\"\"\n    daily_us_equities_bundle builds a daily dataset using Quotemedia\n    end of day equities data. For more information on the Quotemedia\n    data see here: https:\/\/data.nasdaq.com\/databases\/EOD\n    \"\"\"\n    api_key = environ.get(\"DATALINK_API_KEY\")\n    if api_key is None:\n        raise ValueError(\n            \"Please set your DATALINK_API_KEY environment variable and retry.\"\n        )\n\n    raw_data = fetch_data_table(api_key)\n\n    start_session, end_session = raw_data.date.min(), raw_data.date.max()\n    asset_metadata = gen_asset_metadata(raw_data[[\"symbol\", \"date\"]], show_progress)\n\n    exchanges = pd.DataFrame(\n        data=[[\"QUOTEMEDIA\", \"QUOTEMEDIA\", \"US\"]],\n        columns=[\"exchange\", \"canonical_name\", \"country_code\"],\n    )\n    asset_db_writer.write(equities=asset_metadata, exchanges=exchanges)\n\n    symbol_map = asset_metadata.symbol\n    sessions = calendar.sessions_in_range(start_session, end_session)\n\n    raw_data.set_index([\"date\", \"symbol\"], inplace=True)\n    daily_bar_writer.write(\n        parse_pricing_and_vol(raw_data, sessions, symbol_map),\n        show_progress=show_progress,\n    )\n\n    raw_data.reset_index(inplace=True)\n    raw_data[\"symbol\"] = raw_data[\"symbol\"].astype(\"category\")\n    raw_data[\"sid\"] = raw_data.symbol.cat.codes\n    adjustment_writer.write(\n        splits=parse_splits(\n            raw_data[[\"sid\", \"date\", \"split_ratio\"]].loc[raw_data.split_ratio != 1],\n            show_progress=show_progress,\n        ),\n        dividends=parse_dividends(\n            raw_data[[\"sid\", \"date\", \"ex_dividend\"]].loc[raw_data.ex_dividend != 0],\n            show_progress=show_progress,\n        ),\n    )\n\n\ndef download_with_progress(url, chunk_size, **progress_kwargs):\n    \"\"\"\n    Download streaming data from a URL, printing progress information to the\n    terminal.\n    Parameters\n    ----------\n    url : str\n        A URL that can be understood by ``requests.get``.\n    chunk_size : int\n        Number of bytes to read at a time from requests.\n    **progress_kwargs\n        Forwarded to click.progressbar.\n    Returns\n    -------\n    data : BytesIO\n        A BytesIO containing the downloaded data.\n    \"\"\"\n    resp = requests.get(url, stream=True)\n    resp.raise_for_status()\n\n    total_size = int(resp.headers[\"content-length\"])\n    data = BytesIO()\n    with progressbar(length=total_size, **progress_kwargs) as pbar:\n        for chunk in resp.iter_content(chunk_size=chunk_size):\n            data.write(chunk)\n            pbar.update(len(chunk))\n\n    data.seek(0)\n    return data<\/pre>\n\n\n\n
        The format_metadata_url<\/strong> function constructs the URL for querying Nasdaq Data Link based on a provided API key and selects specific columns of data to retrieve, including ticker information, date, and price metrics.<\/p>\n\n\n\n
        The fetch_download_link<\/strong> function attempts to retrieve the actual data download link from Nasdaq Data Link. This link is dynamic and can change, so the function continually checks the status of the data until it is ready for download. If the data isn’t ready after a certain number of tries (defined by MAX_DOWNLOAD_TRIES<\/strong>), the function waits for a set interval before trying again.<\/p>\n\n\n\n
        load_data_table<\/strong> extracts and processes data from a downloaded ZIP file. It assumes the ZIP file contains a single CSV file, from which data is read into a Pandas DataFrame. The columns are renamed to be compatible with Zipline’s naming conventions.<\/p>\n\n\n\n
        In the fetch_data_table<\/strong> function, the data table is fetched by constructing the appropriate metadata URL and then downloading the data, leveraging the previously mentioned functions.<\/p>\n\n\n\n
        Subsequent functions like gen_asset_metadata<\/strong>, parse_splits<\/strong>, parse_dividends<\/strong>, and parse_pricing_and_vol<\/strong> provide parsing and transformation capabilities to process the raw data into a format suitable for Zipline. They generate asset metadata, handle stock split and dividend data, and parse pricing and volume data, respectively.<\/p>\n\n\n\n
        The core function, daily_us_equities_bundle<\/strong>, integrates all the functionalities to fetch and prepare the QuoteMedia End of Day US Stock Prices dataset for Zipline’s consumption. It checks for the required API key, fetches the raw data table, processes it, and writes the formatted data to disk. This function is the primary interface that a user or system might call to get Quandl data into Zipline’s bundle format.<\/p>\n\n\n\n
        Lastly, the download_with_progress<\/strong> functions facilitate the actual data download. The function provides a visual progress bar for tracking download progress. It returns the downloaded data as a BytesIO object, making it easier to subsequently process or store the data.<\/p>\n\n\n\n
        <\/p>\n","protected":false},"excerpt":{"rendered":"
        This article explains how to build the two Python scripts you need to use premium data to create a custom data bundle using Zipline Reloaded.<\/p>\n","protected":false},"author":1518,"featured_media":182059,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":"","jetpack_post_was_ever_published":false},"categories":[339,343,349,338,341],"tags":[851,806,595,20353],"contributors-categories":[17813],"class_list":["post-227216","post","type-post","status-publish","format-standard","has-post-thumbnail","category-data-science","category-programing-languages","category-python-development","category-ibkr-quant-news","category-quant-development","tag-algo-trading","tag-data-science","tag-python","tag-zipline-reloaded","contributors-categories-pyquantnews"],"pp_statuses_selecting_workflow":false,"pp_workflow_action":"current","pp_status_selection":"publish","acf":[],"yoast_head":"\nHow to Ingest Premium Market Data with Zipline Reloaded<\/title>\n<meta name=\"description\" content=\"This article explains how to build the two Python scripts you need to use premium data to create a custom data bundle using Zipline Reloaded.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.interactivebrokers.com\/campus\/wp-json\/wp\/v2\/posts\/227216\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"How to Ingest Premium Market Data with Zipline Reloaded\" \/>\n<meta property=\"og:description\" content=\"This article explains how to build the two Python scripts you need to use premium data to create a custom data bundle using Zipline Reloaded.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/how-to-ingest-premium-market-data-with-zipline-reloaded\/\" \/>\n<meta property=\"og:site_name\" content=\"IBKR Campus US\" \/>\n<meta property=\"article:published_time\" content=\"2025-07-17T12:15:26+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2025-07-18T21:11:11+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2023\/02\/python-blue-dots-opaque.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1000\" \/>\n\t<meta property=\"og:image:height\" content=\"563\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Jason\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Jason\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"4 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\n\t    \"@context\": \"https:\\\/\\\/schema.org\",\n\t    \"@graph\": [\n\t        {\n\t            \"@type\": \"NewsArticle\",\n\t            \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/ibkr-quant-news\\\/how-to-ingest-premium-market-data-with-zipline-reloaded\\\/#article\",\n\t            \"isPartOf\": {\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/ibkr-quant-news\\\/how-to-ingest-premium-market-data-with-zipline-reloaded\\\/\"\n\t            },\n\t            \"author\": {\n\t                \"name\": \"Jason\",\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#\\\/schema\\\/person\\\/41e9bacc875edb13ed6288f4ffb2afec\"\n\t            },\n\t            \"headline\": \"How to Ingest Premium Market Data with Zipline Reloaded\",\n\t            \"datePublished\": \"2025-07-17T12:15:26+00:00\",\n\t            \"dateModified\": \"2025-07-18T21:11:11+00:00\",\n\t            \"mainEntityOfPage\": {\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/ibkr-quant-news\\\/how-to-ingest-premium-market-data-with-zipline-reloaded\\\/\"\n\t            },\n\t            \"wordCount\": 917,\n\t            \"commentCount\": 3,\n\t            \"publisher\": {\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#organization\"\n\t            },\n\t            \"image\": {\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/ibkr-quant-news\\\/how-to-ingest-premium-market-data-with-zipline-reloaded\\\/#primaryimage\"\n\t            },\n\t            \"thumbnailUrl\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2023\\\/02\\\/python-blue-dots-opaque.jpg\",\n\t            \"keywords\": [\n\t                \"Algo Trading\",\n\t                \"Data Science\",\n\t                \"Python\",\n\t                \"Zipline Reloaded\"\n\t            ],\n\t            \"articleSection\": [\n\t                \"Data Science\",\n\t                \"Programming Languages\",\n\t                \"Python Development\",\n\t                \"Quant\",\n\t                \"Quant Development\"\n\t            ],\n\t            \"inLanguage\": \"en-US\",\n\t            \"potentialAction\": [\n\t                {\n\t                    \"@type\": \"CommentAction\",\n\t                    \"name\": \"Comment\",\n\t                    \"target\": [\n\t                        \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/ibkr-quant-news\\\/how-to-ingest-premium-market-data-with-zipline-reloaded\\\/#respond\"\n\t                    ]\n\t                }\n\t            ]\n\t        },\n\t        {\n\t            \"@type\": \"WebPage\",\n\t            \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/ibkr-quant-news\\\/how-to-ingest-premium-market-data-with-zipline-reloaded\\\/\",\n\t            \"url\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/ibkr-quant-news\\\/how-to-ingest-premium-market-data-with-zipline-reloaded\\\/\",\n\t            \"name\": \"How to Ingest Premium Market Data with Zipline Reloaded | IBKR Campus US\",\n\t            \"isPartOf\": {\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#website\"\n\t            },\n\t            \"primaryImageOfPage\": {\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/ibkr-quant-news\\\/how-to-ingest-premium-market-data-with-zipline-reloaded\\\/#primaryimage\"\n\t            },\n\t            \"image\": {\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/ibkr-quant-news\\\/how-to-ingest-premium-market-data-with-zipline-reloaded\\\/#primaryimage\"\n\t            },\n\t            \"thumbnailUrl\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2023\\\/02\\\/python-blue-dots-opaque.jpg\",\n\t            \"datePublished\": \"2025-07-17T12:15:26+00:00\",\n\t            \"dateModified\": \"2025-07-18T21:11:11+00:00\",\n\t            \"description\": \"This article explains how to build the two Python scripts you need to use premium data to create a custom data bundle using Zipline Reloaded.\",\n\t            \"inLanguage\": \"en-US\",\n\t            \"potentialAction\": [\n\t                {\n\t                    \"@type\": \"ReadAction\",\n\t                    \"target\": [\n\t                        \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/ibkr-quant-news\\\/how-to-ingest-premium-market-data-with-zipline-reloaded\\\/\"\n\t                    ]\n\t                }\n\t            ]\n\t        },\n\t        {\n\t            \"@type\": \"ImageObject\",\n\t            \"inLanguage\": \"en-US\",\n\t            \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/ibkr-quant-news\\\/how-to-ingest-premium-market-data-with-zipline-reloaded\\\/#primaryimage\",\n\t            \"url\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2023\\\/02\\\/python-blue-dots-opaque.jpg\",\n\t            \"contentUrl\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2023\\\/02\\\/python-blue-dots-opaque.jpg\",\n\t            \"width\": 1000,\n\t            \"height\": 563,\n\t            \"caption\": \"PySpark \u2013 A Beginner's Guide to Apache Spark and Big Data\"\n\t        },\n\t        {\n\t            \"@type\": \"WebSite\",\n\t            \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#website\",\n\t            \"url\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/\",\n\t            \"name\": \"IBKR Campus US\",\n\t            \"description\": \"Financial Education from Interactive Brokers\",\n\t            \"publisher\": {\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#organization\"\n\t            },\n\t            \"potentialAction\": [\n\t                {\n\t                    \"@type\": \"SearchAction\",\n\t                    \"target\": {\n\t                        \"@type\": \"EntryPoint\",\n\t                        \"urlTemplate\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/?s={search_term_string}\"\n\t                    },\n\t                    \"query-input\": {\n\t                        \"@type\": \"PropertyValueSpecification\",\n\t                        \"valueRequired\": true,\n\t                        \"valueName\": \"search_term_string\"\n\t                    }\n\t                }\n\t            ],\n\t            \"inLanguage\": \"en-US\"\n\t        },\n\t        {\n\t            \"@type\": \"Organization\",\n\t            \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#organization\",\n\t            \"name\": \"Interactive Brokers\",\n\t            \"alternateName\": \"IBKR\",\n\t            \"url\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/\",\n\t            \"logo\": {\n\t                \"@type\": \"ImageObject\",\n\t                \"inLanguage\": \"en-US\",\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#\\\/schema\\\/logo\\\/image\\\/\",\n\t                \"url\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2024\\\/05\\\/ibkr-campus-logo.jpg\",\n\t                \"contentUrl\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2024\\\/05\\\/ibkr-campus-logo.jpg\",\n\t                \"width\": 669,\n\t                \"height\": 669,\n\t                \"caption\": \"Interactive Brokers\"\n\t            },\n\t            \"image\": {\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#\\\/schema\\\/logo\\\/image\\\/\"\n\t            },\n\t            \"publishingPrinciples\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/about-ibkr-campus\\\/\",\n\t            \"ethicsPolicy\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/cyber-security-notice\\\/\"\n\t        },\n\t        {\n\t            \"@type\": \"Person\",\n\t            \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#\\\/schema\\\/person\\\/41e9bacc875edb13ed6288f4ffb2afec\",\n\t            \"name\": \"Jason\",\n\t            \"url\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/author\\\/jasonpyquantnews\\\/\"\n\t        }\n\t    ]\n\t}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"How to Ingest Premium Market Data with Zipline Reloaded","description":"This article explains how to build the two Python scripts you need to use premium data to create a custom data bundle using Zipline Reloaded.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.interactivebrokers.com\/campus\/wp-json\/wp\/v2\/posts\/227216\/","og_locale":"en_US","og_type":"article","og_title":"How to Ingest Premium Market Data with Zipline Reloaded","og_description":"This article explains how to build the two Python scripts you need to use premium data to create a custom data bundle using Zipline Reloaded.","og_url":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/how-to-ingest-premium-market-data-with-zipline-reloaded\/","og_site_name":"IBKR Campus US","article_published_time":"2025-07-17T12:15:26+00:00","article_modified_time":"2025-07-18T21:11:11+00:00","og_image":[{"width":1000,"height":563,"url":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2023\/02\/python-blue-dots-opaque.jpg","type":"image\/jpeg"}],"author":"Jason","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Jason","Est. reading time":"4 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"NewsArticle","@id":"https:\/\/ibkrcampus.com\/campus\/ibkr-quant-news\/how-to-ingest-premium-market-data-with-zipline-reloaded\/#article","isPartOf":{"@id":"https:\/\/ibkrcampus.com\/campus\/ibkr-quant-news\/how-to-ingest-premium-market-data-with-zipline-reloaded\/"},"author":{"name":"Jason","@id":"https:\/\/ibkrcampus.com\/campus\/#\/schema\/person\/41e9bacc875edb13ed6288f4ffb2afec"},"headline":"How to Ingest Premium Market Data with Zipline Reloaded","datePublished":"2025-07-17T12:15:26+00:00","dateModified":"2025-07-18T21:11:11+00:00","mainEntityOfPage":{"@id":"https:\/\/ibkrcampus.com\/campus\/ibkr-quant-news\/how-to-ingest-premium-market-data-with-zipline-reloaded\/"},"wordCount":917,"commentCount":3,"publisher":{"@id":"https:\/\/ibkrcampus.com\/campus\/#organization"},"image":{"@id":"https:\/\/ibkrcampus.com\/campus\/ibkr-quant-news\/how-to-ingest-premium-market-data-with-zipline-reloaded\/#primaryimage"},"thumbnailUrl":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2023\/02\/python-blue-dots-opaque.jpg","keywords":["Algo Trading","Data Science","Python","Zipline Reloaded"],"articleSection":["Data Science","Programming Languages","Python Development","Quant","Quant Development"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/ibkrcampus.com\/campus\/ibkr-quant-news\/how-to-ingest-premium-market-data-with-zipline-reloaded\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/ibkrcampus.com\/campus\/ibkr-quant-news\/how-to-ingest-premium-market-data-with-zipline-reloaded\/","url":"https:\/\/ibkrcampus.com\/campus\/ibkr-quant-news\/how-to-ingest-premium-market-data-with-zipline-reloaded\/","name":"How to Ingest Premium Market Data with Zipline Reloaded | IBKR Campus US","isPartOf":{"@id":"https:\/\/ibkrcampus.com\/campus\/#website"},"primaryImageOfPage":{"@id":"https:\/\/ibkrcampus.com\/campus\/ibkr-quant-news\/how-to-ingest-premium-market-data-with-zipline-reloaded\/#primaryimage"},"image":{"@id":"https:\/\/ibkrcampus.com\/campus\/ibkr-quant-news\/how-to-ingest-premium-market-data-with-zipline-reloaded\/#primaryimage"},"thumbnailUrl":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2023\/02\/python-blue-dots-opaque.jpg","datePublished":"2025-07-17T12:15:26+00:00","dateModified":"2025-07-18T21:11:11+00:00","description":"This article explains how to build the two Python scripts you need to use premium data to create a custom data bundle using Zipline Reloaded.","inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/ibkrcampus.com\/campus\/ibkr-quant-news\/how-to-ingest-premium-market-data-with-zipline-reloaded\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/ibkrcampus.com\/campus\/ibkr-quant-news\/how-to-ingest-premium-market-data-with-zipline-reloaded\/#primaryimage","url":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2023\/02\/python-blue-dots-opaque.jpg","contentUrl":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2023\/02\/python-blue-dots-opaque.jpg","width":1000,"height":563,"caption":"PySpark \u2013 A Beginner's Guide to Apache Spark and Big Data"},{"@type":"WebSite","@id":"https:\/\/ibkrcampus.com\/campus\/#website","url":"https:\/\/ibkrcampus.com\/campus\/","name":"IBKR Campus US","description":"Financial Education from Interactive Brokers","publisher":{"@id":"https:\/\/ibkrcampus.com\/campus\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/ibkrcampus.com\/campus\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/ibkrcampus.com\/campus\/#organization","name":"Interactive Brokers","alternateName":"IBKR","url":"https:\/\/ibkrcampus.com\/campus\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/ibkrcampus.com\/campus\/#\/schema\/logo\/image\/","url":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/05\/ibkr-campus-logo.jpg","contentUrl":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/05\/ibkr-campus-logo.jpg","width":669,"height":669,"caption":"Interactive Brokers"},"image":{"@id":"https:\/\/ibkrcampus.com\/campus\/#\/schema\/logo\/image\/"},"publishingPrinciples":"https:\/\/www.interactivebrokers.com\/campus\/about-ibkr-campus\/","ethicsPolicy":"https:\/\/www.interactivebrokers.com\/campus\/cyber-security-notice\/"},{"@type":"Person","@id":"https:\/\/ibkrcampus.com\/campus\/#\/schema\/person\/41e9bacc875edb13ed6288f4ffb2afec","name":"Jason","url":"https:\/\/www.interactivebrokers.com\/campus\/author\/jasonpyquantnews\/"}]}},"jetpack_featured_media_url":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2023\/02\/python-blue-dots-opaque.jpg","_links":{"self":[{"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/posts\/227216","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/users\/1518"}],"replies":[{"embeddable":true,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/comments?post=227216"}],"version-history":[{"count":0,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/posts\/227216\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/media\/182059"}],"wp:attachment":[{"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/media?parent=227216"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/categories?post=227216"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/tags?post=227216"},{"taxonomy":"contributors-categories","embeddable":true,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/contributors-categories?post=227216"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}