{"id":184104,"date":"2023-01-20T16:12:00","date_gmt":"2023-01-20T21:12:00","guid":{"rendered":"https:\/\/ibkrcampus.com\/traders-insight\/conformal-prediction-a-practical-guide-with-mapie\/"},"modified":"2023-02-09T15:56:17","modified_gmt":"2023-02-09T20:56:17","slug":"conformal-prediction-a-practical-guide-with-mapie","status":"publish","type":"post","link":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/conformal-prediction-a-practical-guide-with-mapie\/","title":{"rendered":"Conformal Prediction \u2013 A Practical Guide with MAPIE"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\"><em>The article &#8220;<a href=\"https:\/\/algotrading101.com\/learn\/conformal-prediction-guide\/\">Conformal Prediction \u2013 A Practical Guide with MAPIE<\/a>&#8221; first appeared on AlgoTrading101 Blog.<\/em><\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong><em>Excerpt<\/em><\/strong><\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"h-what-is-conformal-prediction\">What is Conformal Prediction?<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Conformal Prediction is a set of algorithms that assess the uncertainty of predictions produced by a machine learning model.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"h-what-is-conformal-prediction-used-for\">What is Conformal Prediction used for?<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Conformal Prediction is often used for calibrating one\u2019s machine learning models, estimating their uncertainty, comparing models, increasing the practical performance of the model, and more.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"h-why-should-i-use-conformal-prediction\">Why should I use Conformal Prediction?<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Conformal Prediction is not too computationally expensive<\/li>\n\n\n\n<li>It is easy to implement<\/li>\n\n\n\n<li>On average, it performs better than other calibration methods<\/li>\n\n\n\n<li>Can be applied to any machine learning model or neural network<\/li>\n\n\n\n<li>Can be used in a wide variety of tasks such as regression, classification, outlier detection, and more<\/li>\n\n\n\n<li>Doesn\u2019t depend on the distribution of the data<\/li>\n\n\n\n<li>Tackles uncertainty in predictions<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"h-why-shouldn-t-i-use-conformal-prediction\">Why shouldn\u2019t I use Conformal Prediction?<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Conformal Prediction is a fairly new approach<\/li>\n\n\n\n<li>There aren\u2019t many out-of-the-box libraries that support it<\/li>\n\n\n\n<li>You \u201cditch\u201d the standard way of viewing and interpreting your results (e.g. predictions sets instead of point predictions)<\/li>\n\n\n\n<li>At the time of writing this article, it isn\u2019t widely adopted and there aren\u2019t any books on it<\/li>\n\n\n\n<li>It is still \u201cstuck\u201d more in the academic place than in the practical space<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"h-how-can-conformal-prediction-be-used-in-finance\">How can Conformal Prediction be used in Finance?<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Conformal Prediction can be used in Finance in many ways. It seems to be most promising when it comes to forecasting, addressing repayment uncertainty, modeling algorithms in volatile market settings, and more.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"h-how-can-conformal-prediction-be-used-in-algorithmic-trading\">How can Conformal Prediction be used in Algorithmic Trading?<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Conformal Prediction can help address the uncertainty surrounding trading hypotheses on which your algorithms might be based. Some might even use machine learning outputs as signals for potential trades, here too conformal prediction can help.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"h-what-are-some-conformal-prediction-alternatives\">What are some Conformal Prediction alternatives?<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Conformal Prediction can be replaced with some other methods depending on what you\u2019re doing. Here are some of the top 3 alternatives:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Platt scaling<\/li>\n\n\n\n<li>Isotonic regression<\/li>\n\n\n\n<li>Spline calibration<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"h-understanding-conformal-prediction\">Understanding Conformal Prediction<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">When it comes to understanding Conformal Prediction, everyone has their preferences some like going into the math behind it and research papers, some like looking at code adaptions, and some like reading guides and tutorials like this one.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">I\u2019ll approach this article by explaining it in very crude layman\u2019s terms so that the intuition behind Conformal Prediction can be easily grasped and readily applied to your existing and future projects.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Also, I\u2019ll point you toward resources that feature the other approaches mentioned above in the \u201c<a href=\"https:\/\/algotrading101.com\/learn\/conformal-prediction-guide\/#conformal-prediction-learn-more\">Where can I learn more about Conformal Prediction<\/a>\u201d section.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Conformal Prediction usually \u201cworks\u201d in the following way:<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>Take a trained ML model<\/li>\n\n\n\n<li>Create an additional set from the data called the calibration set (unseen by the model)<\/li>\n\n\n\n<li>Pick an error metric (calibration score) and apply a cutoff point (e.g. 95% quantile cutoff point)<\/li>\n\n\n\n<li>Use the cutoff point to inform the width of your prediction interval<\/li>\n\n\n\n<li>Use this cutoff point to form the prediction sets for new examples<\/li>\n\n\n\n<li>Now, most of your predictions should fall inside the prediction interval<\/li>\n<\/ol>\n\n\n\n<p class=\"wp-block-paragraph\">That\u2019s it! You now can adapt this for different approaches and models. For example, if you had an image classifier trained to classify dolphins the prediction interval wouldn\u2019t hold labels of more uncertain classes that are being kept away by the chosen quantile level.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Also, the more uncertain the model is (e.g. the harder the inputs are), the larger the prediction set will be and vice-versa which is exactly what we\u2019re looking for. This can be further informed by the average set size, set spread, and coverage that we\u2019ll explore further in this article.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Finally, the error metric (also known as the calibration score) is a very important part and engineering choice that informs everything else when interpreting the results of conformal prediction.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Let\u2019s go into coding so that we can grasp this further.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"h-what-is-mapie\">What is MAPIE?<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">MAPIE (Model Agnostic Prediction Interval Estimator) is a Python library that&nbsp;allows you to estimate prediction intervals using any scikit-learn-compatible model for single-output regression or multi-class classification settings. All prediction sets are based on conformal prediction.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"h-how-to-get-started-with-conformal-prediction\">How to get started with Conformal Prediction?<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">To get started with Conformal Prediction, all we need is Python and an IDE such as VS Code, Google Colab, or the like. I\u2019ll go with Google Colab this time and install MAPIE.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>!pip install mapie<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\">How to apply Conformal Prediction with MAPIE in Python?<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">To apply Conformal Prediction with MAPIE in Python, we\u2019ll need to use either the&nbsp;<code>MapieClassification<\/code>&nbsp;or&nbsp;<code>MapieRegression<\/code>&nbsp;algorithm to obtain the scores that we\u2019ll use for implementing conformal prediction.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Let\u2019s start with a classification example:<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">How to apply Conformal Prediction for Classification with MAPIE?<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">To apply Conformal Prediction for classification with MAPIE, we will estimate a prediction set of multiple classes such that the probability of a true label of a new test point is always higher than the target confidence level.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">We\u2019ll use the classifier\u2019s softmax score output as the conformity score on the Iris dataset. The classifier will be the Naive Bayes classifier from the&nbsp;<a href=\"https:\/\/algotrading101.com\/learn\/sklearn-guide\/\"><code>scikit-learn<\/code><\/a>&nbsp;library.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">To execute this properly, we\u2019ll need to follow these steps:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Craft a toy dataset and form train, calibration, and test sets.<\/li>\n\n\n\n<li>Fit the model on the train set.<\/li>\n\n\n\n<li>Set the conformal score to be the softmax output of the true class for each sample in the calibration set&nbsp;<img decoding=\"async\" data-src=\"\/campus\/wp-content\/uploads\/sites\/2\/2023\/02\/algotrading101-mapie-1.jpg\" alt=\" src=\"data:image\/svg+xml;base64,PHN2ZyB3aWR0aD0iMSIgaGVpZ2h0PSIxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPjwvc3ZnPg==\" class=\"lazyload\"><\/li>\n\n\n\n<li>Define&nbsp;<img decoding=\"async\" data-src=\"\/campus\/wp-content\/uploads\/sites\/2\/2023\/02\/algotrading101-mapie-4.jpg\" alt=\" src=\"data:image\/svg+xml;base64,PHN2ZyB3aWR0aD0iMSIgaGVpZ2h0PSIxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPjwvc3ZnPg==\" class=\"lazyload\"> as being the quantile of calibration scores (S1,\u2026,Sn) which is&nbsp;<img decoding=\"async\" data-src=\"\/campus\/wp-content\/uploads\/sites\/2\/2023\/02\/algotrading101-mapie-2.jpg\" alt=\" src=\"data:image\/svg+xml;base64,PHN2ZyB3aWR0aD0iMSIgaGVpZ2h0PSIxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPjwvc3ZnPg==\" class=\"lazyload\"><\/li>\n\n\n\n<li>Form a prediction set for each new test data point (where X<sub>n+1<\/sub>&nbsp;is known but Y<sub>n+1<\/sub>&nbsp;isn\u2019t) so that it includes all the labels with a sufficiently high softmax output.&nbsp;<img decoding=\"async\" data-src=\"\/campus\/wp-content\/uploads\/sites\/2\/2023\/02\/algotrading101-mapie-3.jpg\" alt=\" src=\"data:image\/svg+xml;base64,PHN2ZyB3aWR0aD0iMSIgaGVpZ2h0PSIxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPjwvc3ZnPg==\" class=\"lazyload\"><\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Prior to doing the first two steps, we will import the libraries we need:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>import numpy as np\nimport matplotlib.pyplot as plt\n%matplotlib inline\n\nfrom sklearn.model_selection import train_test_split\nfrom sklearn import naive_bayes\n\nfrom mapie import MapieClassifier\nfrom mapie.metrics import classification_coverage_score<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Now, the first two steps:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code># Create a toy dataset with 2 features and 3 classes (0, 1, 2) with a bit of noise\nn_samples = 1500\nn_features = 2\nn_classes = 3\n\nX = np.random.randn(n_samples, n_features)\ny = np.zeros(n_samples)\n\nfor i in range(n_classes):\n    X&#91;y == i] += np.random.randn(1, n_features) * 1.2\n\ny = np.where(X&#91;:, 0] &gt; 0, 0, 1)\ny = np.where(X&#91;:, 1] &gt; 0, y, y + 1)\n\nX_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.1, random_state=42)\nX_train, X_cal, y_train, y_cal = train_test_split(X_train, y_train, test_size=0.1, random_state=42)\n\n# Train a Gaussian Naive Bayes classifier\nclf = naive_bayes.GaussianNB()\nclf.fit(X_train, y_train)<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\"><em>Visit AlgoTrading101<\/em> for additional insight on this topic and to download the scripts:<em> <a href=\"https:\/\/algotrading101.com\/learn\/conformal-prediction-guide\/\">https:\/\/algotrading101.com\/learn\/conformal-prediction-guide\/<\/a><\/em>.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Conformal Prediction is a set of algorithms that assess the uncertainty of predictions produced by a machine learning model.<\/p>\n","protected":false},"author":815,"featured_media":184109,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":"","jetpack_post_was_ever_published":false},"categories":[339,343,349,338,350,341,351,352,344,2197],"tags":[7257,14477,806,4922,865,852,14478,4659,14479,1225,595,4412],"contributors-categories":[13746],"class_list":["post-184104","post","type-post","status-publish","format-standard","has-post-thumbnail","category-data-science","category-programing-languages","category-python-development","category-ibkr-quant-news","category-quant-asia-pacific","category-quant-development","category-quant-europe","category-quant-north-america","category-quant-regions","category-quant-south-america","tag-algorithmic-trading","tag-conformal-prediction","tag-data-science","tag-econometrics","tag-github","tag-machine-learning","tag-mapie","tag-matplotlib","tag-model-agnostic-prediction-interval-estimator","tag-numpy","tag-python","tag-scikit-learn","contributors-categories-algotrading101"],"pp_statuses_selecting_workflow":false,"pp_workflow_action":"current","pp_status_selection":"publish","acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v26.9 (Yoast SEO v27.8) - https:\/\/yoast.com\/product\/yoast-seo-premium-wordpress\/ -->\n<title>Conformal Prediction \u2013 A Practical Guide with MAPIE<\/title>\n<meta name=\"description\" content=\"Conformal Prediction is a set of algorithms that assess the uncertainty of predictions produced by a machine learning model.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.interactivebrokers.com\/campus\/wp-json\/wp\/v2\/posts\/184104\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Conformal Prediction \u2013 A Practical Guide with MAPIE\" \/>\n<meta property=\"og:description\" content=\"Conformal Prediction is a set of algorithms that assess the uncertainty of predictions produced by a machine learning model.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/conformal-prediction-a-practical-guide-with-mapie\/\" \/>\n<meta property=\"og:site_name\" content=\"IBKR Campus US\" \/>\n<meta property=\"article:published_time\" content=\"2023-01-20T21:12:00+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2023-02-09T20:56:17+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2023\/02\/machine-learning-2.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1000\" \/>\n\t<meta property=\"og:image:height\" content=\"563\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Igor Radovanovic\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:title\" content=\"Conformal Prediction \u2013 A Practical Guide with MAPIE\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Igor Radovanovic\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"5 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\n\t    \"@context\": \"https:\\\/\\\/schema.org\",\n\t    \"@graph\": [\n\t        {\n\t            \"@type\": \"NewsArticle\",\n\t            \"@id\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/conformal-prediction-a-practical-guide-with-mapie\\\/#article\",\n\t            \"isPartOf\": {\n\t                \"@id\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/conformal-prediction-a-practical-guide-with-mapie\\\/\"\n\t            },\n\t            \"author\": {\n\t                \"name\": \"Igor Radovanovic\",\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#\\\/schema\\\/person\\\/b43b33f424bad38d84a7b78eb0193592\"\n\t            },\n\t            \"headline\": \"Conformal Prediction \u2013 A Practical Guide with MAPIE\",\n\t            \"datePublished\": \"2023-01-20T21:12:00+00:00\",\n\t            \"dateModified\": \"2023-02-09T20:56:17+00:00\",\n\t            \"mainEntityOfPage\": {\n\t                \"@id\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/conformal-prediction-a-practical-guide-with-mapie\\\/\"\n\t            },\n\t            \"wordCount\": 915,\n\t            \"publisher\": {\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#organization\"\n\t            },\n\t            \"image\": {\n\t                \"@id\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/conformal-prediction-a-practical-guide-with-mapie\\\/#primaryimage\"\n\t            },\n\t            \"thumbnailUrl\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2023\\\/02\\\/machine-learning-2.jpg\",\n\t            \"keywords\": [\n\t                \"Algorithmic Trading\",\n\t                \"Conformal Prediction\",\n\t                \"Data Science\",\n\t                \"Econometrics\",\n\t                \"GitHub\",\n\t                \"Machine Learning\",\n\t                \"MAPIE\",\n\t                \"Matplotlib\",\n\t                \"Model Agnostic Prediction Interval Estimator\",\n\t                \"NumPy\",\n\t                \"Python\",\n\t                \"Scikit-learn\"\n\t            ],\n\t            \"articleSection\": [\n\t                \"Data Science\",\n\t                \"Programming Languages\",\n\t                \"Python Development\",\n\t                \"Quant\",\n\t                \"Quant Asia Pacific\",\n\t                \"Quant Development\",\n\t                \"Quant Europe\",\n\t                \"Quant North America\",\n\t                \"Quant Regions\",\n\t                \"Quant South America\"\n\t            ],\n\t            \"inLanguage\": \"en-US\"\n\t        },\n\t        {\n\t            \"@type\": \"WebPage\",\n\t            \"@id\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/conformal-prediction-a-practical-guide-with-mapie\\\/\",\n\t            \"url\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/conformal-prediction-a-practical-guide-with-mapie\\\/\",\n\t            \"name\": \"Conformal Prediction \u2013 A Practical Guide with MAPIE\",\n\t            \"isPartOf\": {\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#website\"\n\t            },\n\t            \"primaryImageOfPage\": {\n\t                \"@id\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/conformal-prediction-a-practical-guide-with-mapie\\\/#primaryimage\"\n\t            },\n\t            \"image\": {\n\t                \"@id\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/conformal-prediction-a-practical-guide-with-mapie\\\/#primaryimage\"\n\t            },\n\t            \"thumbnailUrl\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2023\\\/02\\\/machine-learning-2.jpg\",\n\t            \"datePublished\": \"2023-01-20T21:12:00+00:00\",\n\t            \"dateModified\": \"2023-02-09T20:56:17+00:00\",\n\t            \"description\": \"Conformal Prediction is a set of algorithms that assess the uncertainty of predictions produced by a machine learning model.\",\n\t            \"inLanguage\": \"en-US\",\n\t            \"potentialAction\": [\n\t                {\n\t                    \"@type\": \"ReadAction\",\n\t                    \"target\": [\n\t                        \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/conformal-prediction-a-practical-guide-with-mapie\\\/\"\n\t                    ]\n\t                }\n\t            ]\n\t        },\n\t        {\n\t            \"@type\": \"ImageObject\",\n\t            \"inLanguage\": \"en-US\",\n\t            \"@id\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/conformal-prediction-a-practical-guide-with-mapie\\\/#primaryimage\",\n\t            \"url\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2023\\\/02\\\/machine-learning-2.jpg\",\n\t            \"contentUrl\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2023\\\/02\\\/machine-learning-2.jpg\",\n\t            \"width\": 1000,\n\t            \"height\": 563,\n\t            \"caption\": \"Conformal Prediction \u2013 A Practical Guide with MAPIE\"\n\t        },\n\t        {\n\t            \"@type\": \"WebSite\",\n\t            \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#website\",\n\t            \"url\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/\",\n\t            \"name\": \"IBKR Campus US\",\n\t            \"description\": \"Financial Education from Interactive Brokers\",\n\t            \"publisher\": {\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#organization\"\n\t            },\n\t            \"potentialAction\": [\n\t                {\n\t                    \"@type\": \"SearchAction\",\n\t                    \"target\": {\n\t                        \"@type\": \"EntryPoint\",\n\t                        \"urlTemplate\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/?s={search_term_string}\"\n\t                    },\n\t                    \"query-input\": {\n\t                        \"@type\": \"PropertyValueSpecification\",\n\t                        \"valueRequired\": true,\n\t                        \"valueName\": \"search_term_string\"\n\t                    }\n\t                }\n\t            ],\n\t            \"inLanguage\": \"en-US\"\n\t        },\n\t        {\n\t            \"@type\": \"Organization\",\n\t            \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#organization\",\n\t            \"name\": \"Interactive Brokers\",\n\t            \"alternateName\": \"IBKR\",\n\t            \"url\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/\",\n\t            \"logo\": {\n\t                \"@type\": \"ImageObject\",\n\t                \"inLanguage\": \"en-US\",\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#\\\/schema\\\/logo\\\/image\\\/\",\n\t                \"url\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2024\\\/05\\\/ibkr-campus-logo.jpg\",\n\t                \"contentUrl\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2024\\\/05\\\/ibkr-campus-logo.jpg\",\n\t                \"width\": 669,\n\t                \"height\": 669,\n\t                \"caption\": \"Interactive Brokers\"\n\t            },\n\t            \"image\": {\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#\\\/schema\\\/logo\\\/image\\\/\"\n\t            },\n\t            \"publishingPrinciples\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/about-ibkr-campus\\\/\",\n\t            \"ethicsPolicy\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/cyber-security-notice\\\/\"\n\t        },\n\t        {\n\t            \"@type\": \"Person\",\n\t            \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#\\\/schema\\\/person\\\/b43b33f424bad38d84a7b78eb0193592\",\n\t            \"name\": \"Igor Radovanovic\",\n\t            \"url\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/author\\\/igor-radovanovic\\\/\"\n\t        }\n\t    ]\n\t}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Conformal Prediction \u2013 A Practical Guide with MAPIE","description":"Conformal Prediction is a set of algorithms that assess the uncertainty of predictions produced by a machine learning model.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.interactivebrokers.com\/campus\/wp-json\/wp\/v2\/posts\/184104\/","og_locale":"en_US","og_type":"article","og_title":"Conformal Prediction \u2013 A Practical Guide with MAPIE","og_description":"Conformal Prediction is a set of algorithms that assess the uncertainty of predictions produced by a machine learning model.","og_url":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/conformal-prediction-a-practical-guide-with-mapie\/","og_site_name":"IBKR Campus US","article_published_time":"2023-01-20T21:12:00+00:00","article_modified_time":"2023-02-09T20:56:17+00:00","og_image":[{"width":1000,"height":563,"url":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2023\/02\/machine-learning-2.jpg","type":"image\/jpeg"}],"author":"Igor Radovanovic","twitter_card":"summary_large_image","twitter_title":"Conformal Prediction \u2013 A Practical Guide with MAPIE","twitter_misc":{"Written by":"Igor Radovanovic","Est. reading time":"5 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"NewsArticle","@id":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/conformal-prediction-a-practical-guide-with-mapie\/#article","isPartOf":{"@id":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/conformal-prediction-a-practical-guide-with-mapie\/"},"author":{"name":"Igor Radovanovic","@id":"https:\/\/ibkrcampus.com\/campus\/#\/schema\/person\/b43b33f424bad38d84a7b78eb0193592"},"headline":"Conformal Prediction \u2013 A Practical Guide with MAPIE","datePublished":"2023-01-20T21:12:00+00:00","dateModified":"2023-02-09T20:56:17+00:00","mainEntityOfPage":{"@id":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/conformal-prediction-a-practical-guide-with-mapie\/"},"wordCount":915,"publisher":{"@id":"https:\/\/ibkrcampus.com\/campus\/#organization"},"image":{"@id":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/conformal-prediction-a-practical-guide-with-mapie\/#primaryimage"},"thumbnailUrl":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2023\/02\/machine-learning-2.jpg","keywords":["Algorithmic Trading","Conformal Prediction","Data Science","Econometrics","GitHub","Machine Learning","MAPIE","Matplotlib","Model Agnostic Prediction Interval Estimator","NumPy","Python","Scikit-learn"],"articleSection":["Data Science","Programming Languages","Python Development","Quant","Quant Asia Pacific","Quant Development","Quant Europe","Quant North America","Quant Regions","Quant South America"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/conformal-prediction-a-practical-guide-with-mapie\/","url":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/conformal-prediction-a-practical-guide-with-mapie\/","name":"Conformal Prediction \u2013 A Practical Guide with MAPIE","isPartOf":{"@id":"https:\/\/ibkrcampus.com\/campus\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/conformal-prediction-a-practical-guide-with-mapie\/#primaryimage"},"image":{"@id":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/conformal-prediction-a-practical-guide-with-mapie\/#primaryimage"},"thumbnailUrl":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2023\/02\/machine-learning-2.jpg","datePublished":"2023-01-20T21:12:00+00:00","dateModified":"2023-02-09T20:56:17+00:00","description":"Conformal Prediction is a set of algorithms that assess the uncertainty of predictions produced by a machine learning model.","inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/conformal-prediction-a-practical-guide-with-mapie\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/conformal-prediction-a-practical-guide-with-mapie\/#primaryimage","url":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2023\/02\/machine-learning-2.jpg","contentUrl":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2023\/02\/machine-learning-2.jpg","width":1000,"height":563,"caption":"Conformal Prediction \u2013 A Practical Guide with MAPIE"},{"@type":"WebSite","@id":"https:\/\/ibkrcampus.com\/campus\/#website","url":"https:\/\/ibkrcampus.com\/campus\/","name":"IBKR Campus US","description":"Financial Education from Interactive Brokers","publisher":{"@id":"https:\/\/ibkrcampus.com\/campus\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/ibkrcampus.com\/campus\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/ibkrcampus.com\/campus\/#organization","name":"Interactive Brokers","alternateName":"IBKR","url":"https:\/\/ibkrcampus.com\/campus\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/ibkrcampus.com\/campus\/#\/schema\/logo\/image\/","url":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/05\/ibkr-campus-logo.jpg","contentUrl":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/05\/ibkr-campus-logo.jpg","width":669,"height":669,"caption":"Interactive Brokers"},"image":{"@id":"https:\/\/ibkrcampus.com\/campus\/#\/schema\/logo\/image\/"},"publishingPrinciples":"https:\/\/www.interactivebrokers.com\/campus\/about-ibkr-campus\/","ethicsPolicy":"https:\/\/www.interactivebrokers.com\/campus\/cyber-security-notice\/"},{"@type":"Person","@id":"https:\/\/ibkrcampus.com\/campus\/#\/schema\/person\/b43b33f424bad38d84a7b78eb0193592","name":"Igor Radovanovic","url":"https:\/\/www.interactivebrokers.com\/campus\/author\/igor-radovanovic\/"}]}},"jetpack_featured_media_url":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2023\/02\/machine-learning-2.jpg","_links":{"self":[{"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/posts\/184104","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/users\/815"}],"replies":[{"embeddable":true,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/comments?post=184104"}],"version-history":[{"count":0,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/posts\/184104\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/media\/184109"}],"wp:attachment":[{"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/media?parent=184104"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/categories?post=184104"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/tags?post=184104"},{"taxonomy":"contributors-categories","embeddable":true,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/contributors-categories?post=184104"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}