{"id":156606,"date":"2022-09-09T11:33:16","date_gmt":"2022-09-09T15:33:16","guid":{"rendered":"https:\/\/ibkrcampus.com\/?p=156606"},"modified":"2022-11-21T09:58:04","modified_gmt":"2022-11-21T14:58:04","slug":"natural-language-processing-in-python-using-spacy-part-i","status":"publish","type":"post","link":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/natural-language-processing-in-python-using-spacy-part-i\/","title":{"rendered":"Natural Language Processing in Python using spaCy &#8211; Part I"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">The human mind is an amazing place. Umpteen ideas originate there in a split second, coloured with various emotions. Many such thoughts and emotions are splattered across the \u2018walls\u2019 and \u2018feeds\u2019 of increasingly popular social media platforms.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">In the quest to find the elusive alpha, data scientists and quant analysts have now shifted their focus on processing the tons of \u2018big data\u2019 churned out there by internet users. Using programs to understand and analyse the human language is called&nbsp;<a href=\"https:\/\/blog.quantinsti.com\/natural-language-processing-trading\/\">natural language processing<\/a>&nbsp;(NLP).<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">In this post, we\u2019ll look at one of the popular libraries for natural language processing in Python-&nbsp;<strong>spaCy<\/strong>.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The topics we will cover are:<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li>What is spaCy?<\/li><li>How to install spaCy?<\/li><li>NLTK vs spaCy<\/li><li>spaCy trained pipelines<\/li><li>Tokenization using spaCy<\/li><li>Lemmatization using spaCy<\/li><li>Split Text into sentences using spaCy<\/li><li>Removing punctuation using spaCy<\/li><li>Removing stop words using spaCy<\/li><li>POS tagging using spaCy<\/li><li>Named Entity Recognition using spaCy<\/li><li>Dependency Visualization using displaCy<\/li><li>Getting linguistic annotations using spaCy<\/li><li>spaCy examples on Github<\/li><\/ul>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"what-is-spacy\">What is spaCy?<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">spaCy is a free, open-source library for natural language processing in Python. It is one of the two most popular libraries for NLP, the other one being&nbsp;<a href=\"https:\/\/blog.quantinsti.com\/nltk\/\">NLTK<\/a>. We will look at the important differences between the two in a later section.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The spaCy&nbsp;<a href=\"https:\/\/spacy.io\/\" target=\"_blank\" rel=\"noreferrer noopener\">website<\/a>&nbsp;describes it as the preferred tool for \u201c<em>industrial strength natural language processing<\/em>\u201d. The rich features offered by spaCy make it an excellent choice for NLP, information extraction, and natural language understanding.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The key advantage of spaCy is that it is designed to work with large amounts of data in an optimal and robust manner.<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"how-to-install-spacy\">How to install spaCy?<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">The simplest way to install spaCy is to follow the following steps:<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li>Open&nbsp;<a href=\"https:\/\/spacy.io\/usage\" target=\"_blank\" rel=\"noreferrer noopener\">this<\/a>&nbsp;page from spaCy\u2019s website on your browser.<\/li><li>Select the appropriate options for the operating system, platform, package manager, etc.<\/li><li>The appropriate commands will be displayed in the black panel under the options. Click on the \u2018Copy\u2019 icon on the lower right corner of the black panel to copy the installation commands, and paste them on your terminal\/command prompt.<\/li><\/ul>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Note:<\/strong>&nbsp;If you are doing the installation from a Jupyter notebook, don\u2019t forget to prefix the commands with a \u2018!\u2019 sign.<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"nltk-vs-spacy\">NLTK vs spaCy<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/www.nltk.org\/\" target=\"_blank\" rel=\"noreferrer noopener\">Natural Language Toolkit<\/a>&nbsp;(NLTK) is the largest natural language processing library that supports many languages. Let us compare NLTK and spaCy.<\/p>\n\n\n\n<figure class=\"wp-block-table\"><table><tbody><tr><td><strong>S.No.<\/strong><\/td><td><strong>NLTK<\/strong><\/td><td><strong>spaCy<\/strong><\/td><\/tr><tr><td>1.<\/td><td>NLTK is primarily designed for research.<\/td><td>spaCy is designed for production use.<\/td><\/tr><tr><td>2.<\/td><td>NLTK provides support for many languages.<\/td><td>Currently, spaCy provides trained pipelines for 23 languages and supports 66+ languages.<\/td><\/tr><tr><td>3.<\/td><td>NLTK follows a string processing approach and has a modular architecture.<\/td><td>spaCy follows an object-oriented approach.<\/td><\/tr><tr><td>4.<\/td><td>NLTK provides a large number of different NLP algorithms and hence is preferred for research and building innovative solutions. The user can select a particular algorithm from the available options for a particular task.<\/td><td>spaCy uses the best algorithm for a particular task. The user does not have to select an algorithm.&nbsp;<\/td><\/tr><tr><td>5.<\/td><td>NLTK can be slower.<\/td><td>spaCy is optimized for speed.<\/td><\/tr><tr><td>6.<\/td><td>It is built using Python.<\/td><td>It is built using Cython.<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<figure class=\"wp-block-image is-resized\"><img decoding=\"async\" data-src=\"https:\/\/d1rwhvwstyk9gu.cloudfront.net\/2022\/09\/spaCy-features.png\" alt=\"spacy features\" width=\"722\" height=\"447\" src=\"data:image\/svg+xml;base64,PHN2ZyB3aWR0aD0iMSIgaGVpZ2h0PSIxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPjwvc3ZnPg==\" class=\"lazyload\" style=\"--smush-placeholder-width: 722px; aspect-ratio: 722\/447;\" \/><\/figure>\n\n\n\n<p class=\"has-text-align-center wp-block-paragraph\">Source: <a href=\"https:\/\/spacy.io\/\">https:\/\/spacy.io\/<\/a><\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"spacy-trained-pipelines\">spaCy trained pipelines<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">spaCy introduces the concept of pipelines. When you pass a text through a pipeline, it goes through different steps (or pipes) of processing. The output from one step (or pipe) is fed into the next step (or pipe).<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">spaCy offers many trained pipelines for different&nbsp;<a href=\"https:\/\/spacy.io\/usage\/models#languages\" target=\"_blank\" rel=\"noreferrer noopener\">languages<\/a>. Typically, a trained pipeline includes a tagger, a lemmatizer, a parser, and an entity recognizer.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">We can also design our own custom pipelines in spaCy.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" data-src=\"https:\/\/d1rwhvwstyk9gu.cloudfront.net\/2022\/09\/Pipelines-in-spaCy.png\" alt=\"pipelines in spacy\" src=\"data:image\/svg+xml;base64,PHN2ZyB3aWR0aD0iMSIgaGVpZ2h0PSIxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPjwvc3ZnPg==\" class=\"lazyload\" \/><\/figure>\n\n\n\n<p class=\"has-text-align-center wp-block-paragraph\">Source: <a href=\"https:\/\/spacy.io\/usage\/processing-pipelines\">https:\/\/spacy.io\/usage\/processing-pipelines<\/a><\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"getting-started-with-spacy\">Getting started with spaCy<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Let us now do some natural language processing and see how some of these components work in the next few sections.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">We need to have installed spaCy and the trained model that we want to use. In this blog, we will be working with the model for the English language, the en_core_web_sm.<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"tokenization-using-spacy\">Tokenization using spaCy<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Passing a text to a trained model produces the doc container. Though it may appear to be similar to the text, the doc contains valuable metadata related to the text.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Yes, I know! You can\u2019t spot any difference between the text and the doc from the above code snippet. But let us explore a bit more.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Okay, so the length is different. What else? Let us now print the tokens from the doc.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The output for the above line of code is:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Jennifer<br>is<br>learning<br>quantitative<br>analysis.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">We have now seen that the doc container contains tokens. Tokens are the basic building blocks of the spaCy NLP ecosystem. They may be a word or a punctuation mark.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Tokenization<\/strong>&nbsp;is the process of breaking down a text into words, punctuations, etc. This is done using the rules for the specific language whose model we are using.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The tokens have different attributes, which are the foundation of natural language processing using spaCy. We will look at some of these in the following sections.<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"lemmatization-using-spacy\">Lemmatization using spaCy<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">A lemma is the base form of a token, with no inflectional suffixes. E.g., the lemma for \u2018going\u2019 and \u2018went\u2019 will be \u2018go\u2019. This process of deducing the lemma of each token is called lemmatization.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Output:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">I &#8211; I<br>am &#8211; be<br>going &#8211; go<br>where &#8211; where<br>Jennifer &#8211; Jennifer<br>went &#8211; go<br>yesterday &#8211; yesterday<br>. &#8211; .<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><em>Stay tuned for the next installment, in which Udisha Alok will show how to split text into sentences using spaCy<\/em>.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><em>Visit QuantInsti website for additional insight on this topic: <a href=\"https:\/\/blog.quantinsti.com\/spacy-python\/\">https:\/\/blog.quantinsti.com\/spacy-python\/<\/a>.<\/em><\/p>\n","protected":false},"excerpt":{"rendered":"<p>In the quest to find the elusive alpha, data scientists and quant analysts have now shifted their focus on processing the tons of \u2018big data\u2019 churned out there by internet users. Using programs to understand and analyse the human language is called\u00a0natural language processing\u00a0(NLP).<\/p>\n","protected":false},"author":731,"featured_media":156644,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":"","jetpack_post_was_ever_published":false},"categories":[339,343,349,338,350,341,344],"tags":[806,12705,12706,595,12704],"contributors-categories":[13654],"class_list":["post-156606","post","type-post","status-publish","format-standard","has-post-thumbnail","category-data-science","category-programing-languages","category-python-development","category-ibkr-quant-news","category-quant-asia-pacific","category-quant-development","category-quant-regions","tag-data-science","tag-natural-language-processing-nlp","tag-natural-language-toolkit-nltk","tag-python","tag-spacy","contributors-categories-quantinsti"],"pp_statuses_selecting_workflow":false,"pp_workflow_action":"current","pp_status_selection":"publish","acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v26.9 (Yoast SEO v28.0) - https:\/\/yoast.com\/product\/yoast-seo-premium-wordpress\/ -->\n<title>Natural Language Processing in Python using spaCy &#8211; Part I<\/title>\n<meta name=\"description\" content=\"In the quest to find the elusive alpha, data scientists and quant analysts have now shifted their focus on processing the tons of \u2018big data\u2019 churned...\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.interactivebrokers.com\/campus\/wp-json\/wp\/v2\/posts\/156606\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Natural Language Processing in Python using spaCy - Part I | IBKR Quant Blog\" \/>\n<meta property=\"og:description\" content=\"In the quest to find the elusive alpha, data scientists and quant analysts have now shifted their focus on processing the tons of \u2018big data\u2019 churned out there by internet users. Using programs to understand and analyse the human language is called\u00a0natural language processing\u00a0(NLP).\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/natural-language-processing-in-python-using-spacy-part-i\/\" \/>\n<meta property=\"og:site_name\" content=\"IBKR Campus US\" \/>\n<meta property=\"article:published_time\" content=\"2022-09-09T15:33:16+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2022-11-21T14:58:04+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2022\/09\/python-blue-button.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1000\" \/>\n\t<meta property=\"og:image:height\" content=\"563\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Udisha Alok\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Udisha Alok\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"5 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\n\t    \"@context\": \"https:\\\/\\\/schema.org\",\n\t    \"@graph\": [\n\t        {\n\t            \"@type\": \"NewsArticle\",\n\t            \"@id\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/natural-language-processing-in-python-using-spacy-part-i\\\/#article\",\n\t            \"isPartOf\": {\n\t                \"@id\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/natural-language-processing-in-python-using-spacy-part-i\\\/\"\n\t            },\n\t            \"author\": {\n\t                \"name\": \"Udisha Alok\",\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#\\\/schema\\\/person\\\/7faa788f12ff54d5d598292f5a252fab\"\n\t            },\n\t            \"headline\": \"Natural Language Processing in Python using spaCy &#8211; Part I\",\n\t            \"datePublished\": \"2022-09-09T15:33:16+00:00\",\n\t            \"dateModified\": \"2022-11-21T14:58:04+00:00\",\n\t            \"mainEntityOfPage\": {\n\t                \"@id\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/natural-language-processing-in-python-using-spacy-part-i\\\/\"\n\t            },\n\t            \"wordCount\": 935,\n\t            \"publisher\": {\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#organization\"\n\t            },\n\t            \"image\": {\n\t                \"@id\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/natural-language-processing-in-python-using-spacy-part-i\\\/#primaryimage\"\n\t            },\n\t            \"thumbnailUrl\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2022\\\/09\\\/python-blue-button.jpg\",\n\t            \"keywords\": [\n\t                \"Data Science\",\n\t                \"Natural Language Processing (NLP)\",\n\t                \"Natural Language Toolkit (NLTK)\",\n\t                \"Python\",\n\t                \"spaCy\"\n\t            ],\n\t            \"articleSection\": [\n\t                \"Data Science\",\n\t                \"Programming Languages\",\n\t                \"Python Development\",\n\t                \"Quant\",\n\t                \"Quant Asia Pacific\",\n\t                \"Quant Development\",\n\t                \"Quant Regions\"\n\t            ],\n\t            \"inLanguage\": \"en-US\"\n\t        },\n\t        {\n\t            \"@type\": \"WebPage\",\n\t            \"@id\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/natural-language-processing-in-python-using-spacy-part-i\\\/\",\n\t            \"url\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/natural-language-processing-in-python-using-spacy-part-i\\\/\",\n\t            \"name\": \"Natural Language Processing in Python using spaCy - Part I | IBKR Quant Blog\",\n\t            \"isPartOf\": {\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#website\"\n\t            },\n\t            \"primaryImageOfPage\": {\n\t                \"@id\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/natural-language-processing-in-python-using-spacy-part-i\\\/#primaryimage\"\n\t            },\n\t            \"image\": {\n\t                \"@id\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/natural-language-processing-in-python-using-spacy-part-i\\\/#primaryimage\"\n\t            },\n\t            \"thumbnailUrl\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2022\\\/09\\\/python-blue-button.jpg\",\n\t            \"datePublished\": \"2022-09-09T15:33:16+00:00\",\n\t            \"dateModified\": \"2022-11-21T14:58:04+00:00\",\n\t            \"description\": \"In the quest to find the elusive alpha, data scientists and quant analysts have now shifted their focus on processing the tons of \u2018big data\u2019 churned out there by internet users. Using programs to understand and analyse the human language is called\u00a0natural language processing\u00a0(NLP).\",\n\t            \"inLanguage\": \"en-US\",\n\t            \"potentialAction\": [\n\t                {\n\t                    \"@type\": \"ReadAction\",\n\t                    \"target\": [\n\t                        \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/natural-language-processing-in-python-using-spacy-part-i\\\/\"\n\t                    ]\n\t                }\n\t            ]\n\t        },\n\t        {\n\t            \"@type\": \"ImageObject\",\n\t            \"inLanguage\": \"en-US\",\n\t            \"@id\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/natural-language-processing-in-python-using-spacy-part-i\\\/#primaryimage\",\n\t            \"url\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2022\\\/09\\\/python-blue-button.jpg\",\n\t            \"contentUrl\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2022\\\/09\\\/python-blue-button.jpg\",\n\t            \"width\": 1000,\n\t            \"height\": 563,\n\t            \"caption\": \"Python\"\n\t        },\n\t        {\n\t            \"@type\": \"WebSite\",\n\t            \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#website\",\n\t            \"url\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/\",\n\t            \"name\": \"IBKR Campus US\",\n\t            \"description\": \"Financial Education from Interactive Brokers\",\n\t            \"publisher\": {\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#organization\"\n\t            },\n\t            \"potentialAction\": [\n\t                {\n\t                    \"@type\": \"SearchAction\",\n\t                    \"target\": {\n\t                        \"@type\": \"EntryPoint\",\n\t                        \"urlTemplate\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/?s={search_term_string}\"\n\t                    },\n\t                    \"query-input\": {\n\t                        \"@type\": \"PropertyValueSpecification\",\n\t                        \"valueRequired\": true,\n\t                        \"valueName\": \"search_term_string\"\n\t                    }\n\t                }\n\t            ],\n\t            \"inLanguage\": \"en-US\"\n\t        },\n\t        {\n\t            \"@type\": \"Organization\",\n\t            \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#organization\",\n\t            \"name\": \"Interactive Brokers\",\n\t            \"alternateName\": \"IBKR\",\n\t            \"url\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/\",\n\t            \"logo\": {\n\t                \"@type\": \"ImageObject\",\n\t                \"inLanguage\": \"en-US\",\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#\\\/schema\\\/logo\\\/image\\\/\",\n\t                \"url\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2024\\\/05\\\/ibkr-campus-logo.jpg\",\n\t                \"contentUrl\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2024\\\/05\\\/ibkr-campus-logo.jpg\",\n\t                \"width\": 669,\n\t                \"height\": 669,\n\t                \"caption\": \"Interactive Brokers\"\n\t            },\n\t            \"image\": {\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#\\\/schema\\\/logo\\\/image\\\/\"\n\t            },\n\t            \"publishingPrinciples\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/about-ibkr-campus\\\/\",\n\t            \"ethicsPolicy\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/cyber-security-notice\\\/\"\n\t        },\n\t        {\n\t            \"@type\": \"Person\",\n\t            \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#\\\/schema\\\/person\\\/7faa788f12ff54d5d598292f5a252fab\",\n\t            \"name\": \"Udisha Alok\",\n\t            \"url\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/author\\\/udisha-alok\\\/\"\n\t        }\n\t    ]\n\t}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Natural Language Processing in Python using spaCy &#8211; Part I","description":"In the quest to find the elusive alpha, data scientists and quant analysts have now shifted their focus on processing the tons of \u2018big data\u2019 churned...","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.interactivebrokers.com\/campus\/wp-json\/wp\/v2\/posts\/156606\/","og_locale":"en_US","og_type":"article","og_title":"Natural Language Processing in Python using spaCy - Part I | IBKR Quant Blog","og_description":"In the quest to find the elusive alpha, data scientists and quant analysts have now shifted their focus on processing the tons of \u2018big data\u2019 churned out there by internet users. Using programs to understand and analyse the human language is called\u00a0natural language processing\u00a0(NLP).","og_url":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/natural-language-processing-in-python-using-spacy-part-i\/","og_site_name":"IBKR Campus US","article_published_time":"2022-09-09T15:33:16+00:00","article_modified_time":"2022-11-21T14:58:04+00:00","og_image":[{"width":1000,"height":563,"url":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2022\/09\/python-blue-button.jpg","type":"image\/jpeg"}],"author":"Udisha Alok","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Udisha Alok","Est. reading time":"5 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"NewsArticle","@id":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/natural-language-processing-in-python-using-spacy-part-i\/#article","isPartOf":{"@id":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/natural-language-processing-in-python-using-spacy-part-i\/"},"author":{"name":"Udisha Alok","@id":"https:\/\/ibkrcampus.com\/campus\/#\/schema\/person\/7faa788f12ff54d5d598292f5a252fab"},"headline":"Natural Language Processing in Python using spaCy &#8211; Part I","datePublished":"2022-09-09T15:33:16+00:00","dateModified":"2022-11-21T14:58:04+00:00","mainEntityOfPage":{"@id":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/natural-language-processing-in-python-using-spacy-part-i\/"},"wordCount":935,"publisher":{"@id":"https:\/\/ibkrcampus.com\/campus\/#organization"},"image":{"@id":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/natural-language-processing-in-python-using-spacy-part-i\/#primaryimage"},"thumbnailUrl":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2022\/09\/python-blue-button.jpg","keywords":["Data Science","Natural Language Processing (NLP)","Natural Language Toolkit (NLTK)","Python","spaCy"],"articleSection":["Data Science","Programming Languages","Python Development","Quant","Quant Asia Pacific","Quant Development","Quant Regions"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/natural-language-processing-in-python-using-spacy-part-i\/","url":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/natural-language-processing-in-python-using-spacy-part-i\/","name":"Natural Language Processing in Python using spaCy - Part I | IBKR Quant Blog","isPartOf":{"@id":"https:\/\/ibkrcampus.com\/campus\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/natural-language-processing-in-python-using-spacy-part-i\/#primaryimage"},"image":{"@id":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/natural-language-processing-in-python-using-spacy-part-i\/#primaryimage"},"thumbnailUrl":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2022\/09\/python-blue-button.jpg","datePublished":"2022-09-09T15:33:16+00:00","dateModified":"2022-11-21T14:58:04+00:00","description":"In the quest to find the elusive alpha, data scientists and quant analysts have now shifted their focus on processing the tons of \u2018big data\u2019 churned out there by internet users. Using programs to understand and analyse the human language is called\u00a0natural language processing\u00a0(NLP).","inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/natural-language-processing-in-python-using-spacy-part-i\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/natural-language-processing-in-python-using-spacy-part-i\/#primaryimage","url":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2022\/09\/python-blue-button.jpg","contentUrl":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2022\/09\/python-blue-button.jpg","width":1000,"height":563,"caption":"Python"},{"@type":"WebSite","@id":"https:\/\/ibkrcampus.com\/campus\/#website","url":"https:\/\/ibkrcampus.com\/campus\/","name":"IBKR Campus US","description":"Financial Education from Interactive Brokers","publisher":{"@id":"https:\/\/ibkrcampus.com\/campus\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/ibkrcampus.com\/campus\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/ibkrcampus.com\/campus\/#organization","name":"Interactive Brokers","alternateName":"IBKR","url":"https:\/\/ibkrcampus.com\/campus\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/ibkrcampus.com\/campus\/#\/schema\/logo\/image\/","url":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/05\/ibkr-campus-logo.jpg","contentUrl":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/05\/ibkr-campus-logo.jpg","width":669,"height":669,"caption":"Interactive Brokers"},"image":{"@id":"https:\/\/ibkrcampus.com\/campus\/#\/schema\/logo\/image\/"},"publishingPrinciples":"https:\/\/www.interactivebrokers.com\/campus\/about-ibkr-campus\/","ethicsPolicy":"https:\/\/www.interactivebrokers.com\/campus\/cyber-security-notice\/"},{"@type":"Person","@id":"https:\/\/ibkrcampus.com\/campus\/#\/schema\/person\/7faa788f12ff54d5d598292f5a252fab","name":"Udisha Alok","url":"https:\/\/www.interactivebrokers.com\/campus\/author\/udisha-alok\/"}]}},"jetpack_featured_media_url":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2022\/09\/python-blue-button.jpg","_links":{"self":[{"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/posts\/156606","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/users\/731"}],"replies":[{"embeddable":true,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/comments?post=156606"}],"version-history":[{"count":0,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/posts\/156606\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/media\/156644"}],"wp:attachment":[{"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/media?parent=156606"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/categories?post=156606"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/tags?post=156606"},{"taxonomy":"contributors-categories","embeddable":true,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/contributors-categories?post=156606"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}