{"id":3913,"date":"2019-05-17T10:23:47","date_gmt":"2019-05-17T14:23:47","guid":{"rendered":"https:\/\/ibkrcampus.com\/?p=3913"},"modified":"2023-04-17T17:08:04","modified_gmt":"2023-04-17T21:08:04","slug":"detecting-bots-on-twitter-using-botometer-part-i","status":"publish","type":"post","link":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/detecting-bots-on-twitter-using-botometer-part-i\/","title":{"rendered":"Detecting Bots On Twitter Using Botometer &#8211; Part I"},"content":{"rendered":"\n<p>In this blog, we will learn what bots are and how they can skew the sentiment analysis used in your trading strategy. We will cover the following topics:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><a href=\"https:\/\/www.quantinsti.com\/blog\/detecting-bots-twitter-botometer#whybots\" target=\"_blank\" rel=\"noreferrer noopener\">Why should we identify a bot?<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/www.quantinsti.com\/blog\/detecting-bots-twitter-botometer#botometer\" target=\"_blank\" rel=\"noreferrer noopener\">Botometer<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/www.quantinsti.com\/blog\/detecting-bots-twitter-botometer#identifybots\" target=\"_blank\" rel=\"noreferrer noopener\">Identifying the bots on Twitter<\/a><\/li>\n<\/ul>\n\n\n\n<p>When we perform trading on the basis of market sentiment, we need to fetch data from news sources such as Twitter, Reuters, Bloomberg and Webhosie, etc. Although reading complete articles and gauging their sentiment can be difficult, estimating the sentiment of a tweet is not that complicated.<\/p>\n\n\n\n<p>But before you estimate the sentiment of a tweet you need to know if the tweet was an automated response of a bot or made by a human.<\/p>\n\n\n\n<p>You may ask why this is relevant?<\/p>\n\n\n\n<p><strong>Why should we identify a bot?<\/strong><\/p>\n\n\n\n<p>It is relevant because you need to know what the bots are doing, which in turn will tell you how the sentiment of a particular stock on Twitter is being manipulated. When we calculate the Twitter sentiment of a particular stock, we identify and remove those tweets made by bot users. This will give the true sentiment sans manipulation. This true sentiment can be a very powerful metric, when used with other technical indicators, to call the tops and bottoms of a trend.<\/p>\n\n\n\n<p><strong>Botometer<\/strong><\/p>\n\n\n\n<p>In python, we use the library called botometer to know if a particular tweet was made by a bot or not.<br>The botometer library uses a machine learning algorithm trained on tens of thousands of labelled data. This algorithm\u2019s output is a probability on a scale of 0 to 1, where 1 indicates that a Twitter account is managed by a bot.<\/p>\n\n\n\n<p>The Botometer API takes the user id as the input and then extracts 1200 features related to that user to compute a score. The Botometer gives separate scores for the following categories:<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>Network features<\/li>\n\n\n\n<li>User features<\/li>\n\n\n\n<li>Friends features<\/li>\n\n\n\n<li>Temporal features<\/li>\n\n\n\n<li>Content features<\/li>\n\n\n\n<li>Sentiment features<\/li>\n<\/ol>\n\n\n\n<p>Let us discuss some of these features.<\/p>\n\n\n\n<p><strong>Network features<\/strong><\/p>\n\n\n\n<p>Network features of a user include information on the retweets, mentions, and hashtags that a user tweeted in the past.<\/p>\n\n\n\n<p>For example, If the user is retweeting only those tweets made by a particular handle, then the user is most likely a bot.<\/p>\n\n\n\n<p><strong>User features<\/strong><\/p>\n\n\n\n<p>This contains user-specific information such as the user name, language, location, account created date, etc., Generally, bots do not contain such information. And if they do, it will be something gibberish.<\/p>\n\n\n\n<p><em>Learn more QuantInsti here&nbsp;<\/em><a href=\"https:\/\/www.quantinsti.com\/\" target=\"_blank\" rel=\"noreferrer noopener\">https:\/\/www.quantinsti.com<\/a><\/p>\n\n\n\n<p><em>To learn more about Python and R, visit QuantInsti&nbsp;<\/em><a rel=\"noreferrer noopener\" href=\"https:\/\/www.quantinsti.com\/category\/quantitative-trading-strategies\" target=\"_blank\"><em>website<\/em><\/a><em>&nbsp;and their educational offerings at their&nbsp;<\/em><a rel=\"noreferrer noopener\" href=\"https:\/\/www.quantinsti.com\/epat\" target=\"_blank\"><em>Executive Programme in Algorithmic Trading (EPAT\u2122)<\/em><\/a><em>.<\/em><\/p>\n","protected":false},"excerpt":{"rendered":"<p>When we perform trading on the basis of market sentiment, we need to fetch data from news sources such as Twitter, Reuters, Bloomberg and Webhosie, etc. Although reading complete articles and gauging their sentiment can be difficult, estimating the sentiment of a tweet is not that complicated.<\/p>\n","protected":false},"author":134,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[339,343,349,338,350,341,344],"tags":[595,494,843],"contributors-categories":[13654],"class_list":{"0":"post-3913","1":"post","2":"type-post","3":"status-publish","4":"format-standard","6":"category-data-science","7":"category-programing-languages","8":"category-python-development","9":"category-ibkr-quant-news","10":"category-quant-asia-pacific","11":"category-quant-development","12":"category-quant-regions","13":"tag-python","14":"tag-quant","15":"tag-social-sentiment","16":"contributors-categories-quantinsti"},"pp_statuses_selecting_workflow":false,"pp_workflow_action":"current","pp_status_selection":"publish","acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v26.9 (Yoast SEO v27.4) - https:\/\/yoast.com\/product\/yoast-seo-premium-wordpress\/ -->\n<title>Detecting Bots On Twitter Using Botometer &#8211; Part I<\/title>\n<meta name=\"description\" content=\"In this blog, we will learn what bots are and how they can skew the sentiment analysis used in your trading strategy.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.interactivebrokers.com\/campus\/wp-json\/wp\/v2\/posts\/3913\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Detecting Bots On Twitter Using Botometer - Part I | IBKR Campus US\" \/>\n<meta property=\"og:description\" content=\"In this blog, we will learn what bots are and how they can skew the sentiment analysis used in your trading strategy.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/detecting-bots-on-twitter-using-botometer-part-i\/\" \/>\n<meta property=\"og:site_name\" content=\"IBKR Campus US\" \/>\n<meta property=\"article:published_time\" content=\"2019-05-17T14:23:47+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2023-04-17T21:08:04+00:00\" \/>\n<meta name=\"author\" content=\"Varun Divakar\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Varun Divakar\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\n\t    \"@context\": \"https:\\\/\\\/schema.org\",\n\t    \"@graph\": [\n\t        {\n\t            \"@type\": \"NewsArticle\",\n\t            \"@id\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/detecting-bots-on-twitter-using-botometer-part-i\\\/#article\",\n\t            \"isPartOf\": {\n\t                \"@id\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/detecting-bots-on-twitter-using-botometer-part-i\\\/\"\n\t            },\n\t            \"author\": {\n\t                \"name\": \"Varun Divakar\",\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#\\\/schema\\\/person\\\/c1ba2ec6d2105bcd9a7668a58f7003f5\"\n\t            },\n\t            \"headline\": \"Detecting Bots On Twitter Using Botometer &#8211; Part I\",\n\t            \"datePublished\": \"2019-05-17T14:23:47+00:00\",\n\t            \"dateModified\": \"2023-04-17T21:08:04+00:00\",\n\t            \"mainEntityOfPage\": {\n\t                \"@id\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/detecting-bots-on-twitter-using-botometer-part-i\\\/\"\n\t            },\n\t            \"wordCount\": 443,\n\t            \"publisher\": {\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#organization\"\n\t            },\n\t            \"keywords\": [\n\t                \"Python\",\n\t                \"Quant\",\n\t                \"Social Sentiment\"\n\t            ],\n\t            \"articleSection\": [\n\t                \"Data Science\",\n\t                \"Programming Languages\",\n\t                \"Python Development\",\n\t                \"Quant\",\n\t                \"Quant Asia Pacific\",\n\t                \"Quant Development\",\n\t                \"Quant Regions\"\n\t            ],\n\t            \"inLanguage\": \"en-US\"\n\t        },\n\t        {\n\t            \"@type\": \"WebPage\",\n\t            \"@id\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/detecting-bots-on-twitter-using-botometer-part-i\\\/\",\n\t            \"url\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/detecting-bots-on-twitter-using-botometer-part-i\\\/\",\n\t            \"name\": \"Detecting Bots On Twitter Using Botometer - Part I | IBKR Campus US\",\n\t            \"isPartOf\": {\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#website\"\n\t            },\n\t            \"datePublished\": \"2019-05-17T14:23:47+00:00\",\n\t            \"dateModified\": \"2023-04-17T21:08:04+00:00\",\n\t            \"description\": \"In this blog, we will learn what bots are and how they can skew the sentiment analysis used in your trading strategy.\",\n\t            \"inLanguage\": \"en-US\",\n\t            \"potentialAction\": [\n\t                {\n\t                    \"@type\": \"ReadAction\",\n\t                    \"target\": [\n\t                        \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/detecting-bots-on-twitter-using-botometer-part-i\\\/\"\n\t                    ]\n\t                }\n\t            ]\n\t        },\n\t        {\n\t            \"@type\": \"WebSite\",\n\t            \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#website\",\n\t            \"url\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/\",\n\t            \"name\": \"IBKR Campus US\",\n\t            \"description\": \"Financial Education from Interactive Brokers\",\n\t            \"publisher\": {\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#organization\"\n\t            },\n\t            \"potentialAction\": [\n\t                {\n\t                    \"@type\": \"SearchAction\",\n\t                    \"target\": {\n\t                        \"@type\": \"EntryPoint\",\n\t                        \"urlTemplate\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/?s={search_term_string}\"\n\t                    },\n\t                    \"query-input\": {\n\t                        \"@type\": \"PropertyValueSpecification\",\n\t                        \"valueRequired\": true,\n\t                        \"valueName\": \"search_term_string\"\n\t                    }\n\t                }\n\t            ],\n\t            \"inLanguage\": \"en-US\"\n\t        },\n\t        {\n\t            \"@type\": \"Organization\",\n\t            \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#organization\",\n\t            \"name\": \"Interactive Brokers\",\n\t            \"alternateName\": \"IBKR\",\n\t            \"url\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/\",\n\t            \"logo\": {\n\t                \"@type\": \"ImageObject\",\n\t                \"inLanguage\": \"en-US\",\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#\\\/schema\\\/logo\\\/image\\\/\",\n\t                \"url\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2024\\\/05\\\/ibkr-campus-logo.jpg\",\n\t                \"contentUrl\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2024\\\/05\\\/ibkr-campus-logo.jpg\",\n\t                \"width\": 669,\n\t                \"height\": 669,\n\t                \"caption\": \"Interactive Brokers\"\n\t            },\n\t            \"image\": {\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#\\\/schema\\\/logo\\\/image\\\/\"\n\t            },\n\t            \"publishingPrinciples\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/about-ibkr-campus\\\/\",\n\t            \"ethicsPolicy\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/cyber-security-notice\\\/\"\n\t        },\n\t        {\n\t            \"@type\": \"Person\",\n\t            \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#\\\/schema\\\/person\\\/c1ba2ec6d2105bcd9a7668a58f7003f5\",\n\t            \"name\": \"Varun Divakar\",\n\t            \"url\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/author\\\/varundivakar\\\/\"\n\t        }\n\t    ]\n\t}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Detecting Bots On Twitter Using Botometer &#8211; Part I","description":"In this blog, we will learn what bots are and how they can skew the sentiment analysis used in your trading strategy.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.interactivebrokers.com\/campus\/wp-json\/wp\/v2\/posts\/3913\/","og_locale":"en_US","og_type":"article","og_title":"Detecting Bots On Twitter Using Botometer - Part I | IBKR Campus US","og_description":"In this blog, we will learn what bots are and how they can skew the sentiment analysis used in your trading strategy.","og_url":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/detecting-bots-on-twitter-using-botometer-part-i\/","og_site_name":"IBKR Campus US","article_published_time":"2019-05-17T14:23:47+00:00","article_modified_time":"2023-04-17T21:08:04+00:00","author":"Varun Divakar","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Varun Divakar","Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"NewsArticle","@id":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/detecting-bots-on-twitter-using-botometer-part-i\/#article","isPartOf":{"@id":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/detecting-bots-on-twitter-using-botometer-part-i\/"},"author":{"name":"Varun Divakar","@id":"https:\/\/ibkrcampus.com\/campus\/#\/schema\/person\/c1ba2ec6d2105bcd9a7668a58f7003f5"},"headline":"Detecting Bots On Twitter Using Botometer &#8211; Part I","datePublished":"2019-05-17T14:23:47+00:00","dateModified":"2023-04-17T21:08:04+00:00","mainEntityOfPage":{"@id":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/detecting-bots-on-twitter-using-botometer-part-i\/"},"wordCount":443,"publisher":{"@id":"https:\/\/ibkrcampus.com\/campus\/#organization"},"keywords":["Python","Quant","Social Sentiment"],"articleSection":["Data Science","Programming Languages","Python Development","Quant","Quant Asia Pacific","Quant Development","Quant Regions"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/detecting-bots-on-twitter-using-botometer-part-i\/","url":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/detecting-bots-on-twitter-using-botometer-part-i\/","name":"Detecting Bots On Twitter Using Botometer - Part I | IBKR Campus US","isPartOf":{"@id":"https:\/\/ibkrcampus.com\/campus\/#website"},"datePublished":"2019-05-17T14:23:47+00:00","dateModified":"2023-04-17T21:08:04+00:00","description":"In this blog, we will learn what bots are and how they can skew the sentiment analysis used in your trading strategy.","inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/detecting-bots-on-twitter-using-botometer-part-i\/"]}]},{"@type":"WebSite","@id":"https:\/\/ibkrcampus.com\/campus\/#website","url":"https:\/\/ibkrcampus.com\/campus\/","name":"IBKR Campus US","description":"Financial Education from Interactive Brokers","publisher":{"@id":"https:\/\/ibkrcampus.com\/campus\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/ibkrcampus.com\/campus\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/ibkrcampus.com\/campus\/#organization","name":"Interactive Brokers","alternateName":"IBKR","url":"https:\/\/ibkrcampus.com\/campus\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/ibkrcampus.com\/campus\/#\/schema\/logo\/image\/","url":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/05\/ibkr-campus-logo.jpg","contentUrl":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/05\/ibkr-campus-logo.jpg","width":669,"height":669,"caption":"Interactive Brokers"},"image":{"@id":"https:\/\/ibkrcampus.com\/campus\/#\/schema\/logo\/image\/"},"publishingPrinciples":"https:\/\/www.interactivebrokers.com\/campus\/about-ibkr-campus\/","ethicsPolicy":"https:\/\/www.interactivebrokers.com\/campus\/cyber-security-notice\/"},{"@type":"Person","@id":"https:\/\/ibkrcampus.com\/campus\/#\/schema\/person\/c1ba2ec6d2105bcd9a7668a58f7003f5","name":"Varun Divakar","url":"https:\/\/www.interactivebrokers.com\/campus\/author\/varundivakar\/"}]}},"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/posts\/3913","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/users\/134"}],"replies":[{"embeddable":true,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/comments?post=3913"}],"version-history":[{"count":0,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/posts\/3913\/revisions"}],"wp:attachment":[{"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/media?parent=3913"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/categories?post=3913"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/tags?post=3913"},{"taxonomy":"contributors-categories","embeddable":true,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/contributors-categories?post=3913"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}