{"id":186240,"date":"2023-03-07T12:42:06","date_gmt":"2023-03-07T17:42:06","guid":{"rendered":"https:\/\/ibkrcampus.com\/?p=186240"},"modified":"2023-03-08T09:37:12","modified_gmt":"2023-03-08T14:37:12","slug":"how-to-get-an-auc-confidence-interval-2","status":"publish","type":"post","link":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/how-to-get-an-auc-confidence-interval-2\/","title":{"rendered":"How to Get an AUC Confidence Interval"},"content":{"rendered":"\n<h2 class=\"wp-block-heading\" id=\"h-background\"><strong>Background<\/strong><\/h2>\n\n\n\n<p>AUC is an important metric in machine learning for classification. It is often used as a measure of a model\u2019s performance. In effect, AUC is a measure between 0 and 1 of a model\u2019s performance that rank-orders predictions from a model. For a detailed explanation of AUC, see&nbsp;<a href=\"https:\/\/stats.stackexchange.com\/questions\/132777\/what-does-auc-stand-for-and-what-is-it\">this link<\/a>.<\/p>\n\n\n\n<p>Since AUC is widely used, being able to get a confidence interval around this metric is valuable to both better demonstrate a model\u2019s performance, as well as to better compare two or more models. For example, if model A has an AUC higher than model B, but the 95% confidence interval around each AUC value overlaps, then the models may not be statistically different in performance. We can get a confidence interval around AUC using&nbsp;<a href=\"https:\/\/theautomatic.net\/category\/r\/\">R\u2019s<\/a>&nbsp;<strong>pROC<\/strong>&nbsp;package, which uses bootstrapping to calculate the interval.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-building-a-simple-model-to-test\"><strong>Building a simple model to test<\/strong><\/h2>\n\n\n\n<p>To demonstrate how to get an AUC confidence interval, let\u2019s build a model using a movies dataset from Kaggle (<a href=\"https:\/\/www.kaggle.com\/carolzhangdc\/imdb-5000-movie-dataset\/\">you can get the data here<\/a>).<\/p>\n\n\n\n<h4 class=\"wp-block-heading\" id=\"h-reading-in-the-data\"><strong>Reading in the data<\/strong><\/h4>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\"># load packages\nlibrary(pROC)\nlibrary(dplyr)\nlibrary(randomForest)\n \n# read in dataset\nmovies &lt;- read.csv(\"movie_metadata.csv\")\n \n# remove records with missing budget \/ gross data\nmovies &lt;- movies %>% filter(!is.na(budget) &amp; !is.na(gross))<\/pre>\n\n\n\n<h4 class=\"wp-block-heading\"><strong>Split into train \/ test<\/strong><\/h4>\n\n\n\n<p>Next, let\u2019s randomly select 70% of the records to be in the training set and leave the rest for testing.<\/p>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\"># get random sample of rows\nset.seed(0)\ntrain_rows &lt;- sample(1:nrow(movies), .7 * nrow(movies))\n \n# split data into train \/ test\ntrain_data &lt;- movies[train_rows,]\ntest_data &lt;- movies[-train_rows,]\n \n# select only fields we need\ntrain_need &lt;- train_data %>% select(gross, duration, director_facebook_likes, budget, imdb_score, content_rating, movie_title)\ntest_need &lt;- test_data %>% select(gross, duration, director_facebook_likes, budget, imdb_score, content_rating, movie_title)<\/pre>\n\n\n\n<h4 class=\"wp-block-heading\"><strong>Create the label<\/strong><\/h4>\n\n\n\n<p>Lastly, we need to create our label i.e. what we\u2019re trying to predict. Here, we\u2019re going to predict if a movie\u2019s gross beats its budget (1 if so, 0 if not).<\/p>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">train_need$beat_budget &lt;- as.factor(ifelse(train_need$gross > train_need$budget, 1, 0))\ntest_need$beat_budget &lt;- as.factor(ifelse(test_need$gross > test_need$budget, 1, 0))<\/pre>\n\n\n\n<h4 class=\"wp-block-heading\"><strong>Train a random forest<\/strong><\/h4>\n\n\n\n<p>Now, let\u2019s train a simple random forest model with just 50 trees.<\/p>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">train_need &lt;- train_need[complete.cases(train_need),]\n \n# train a random forest\nforest &lt;- randomForest(beat_budget ~ duration + director_facebook_likes + budget + imdb_score + content_rating,\n                       train_need, ntree = 50)<\/pre>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>Getting an AUC confidence interval<\/strong><\/h2>\n\n\n\n<p>Next, let\u2019s use our model to get predictions on the test set.<\/p>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">test_pred &lt;- predict(forest, test_need, type = \"prob\")[,2]<\/pre>\n\n\n\n<p>And now, we\u2019re reading to get our confidence interval! We can do that in just one line of code using the&nbsp;<em>ci.auc<\/em>&nbsp;function from&nbsp;<strong>pROC<\/strong>. By default, this function uses 2000 bootstraps to calculate a 95% confidence interval. This means our 95% confidence interval for the AUC on the test set is between 0.6198 and 0.6822, as can be seen below.<\/p>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">ci.auc(test_need$beat_budget, test_pred) \n# 95% CI: 0.6198-0.6822 (DeLong)<\/pre>\n\n\n\n<p>We can adjust the confidence interval using the&nbsp;<em>conf.level<\/em>&nbsp;parameter:<\/p>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">ci.auc(test_need$beat_budget, test_pred, conf.level = 0.9) \n# 90% CI: 0.6248-0.6772 (DeLong)<\/pre>\n\n\n\n<p><em>Originally posted on <a href=\"https:\/\/theautomatic.net\/2019\/08\/20\/how-to-get-an-auc-confidence-interval\/\">TheAutomatic.net<\/a> Blog.<\/em><\/p>\n","protected":false},"excerpt":{"rendered":"<p>AUC is an important metric in machine learning for classification. It is often used as a measure of a model\u2019s performance.<\/p>\n","protected":false},"author":388,"featured_media":183462,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[339,343,338,341,352,344,342],"tags":[806,2535,14870,487,12111,6591],"contributors-categories":[13695],"class_list":{"0":"post-186240","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-data-science","8":"category-programing-languages","9":"category-ibkr-quant-news","10":"category-quant-development","11":"category-quant-north-america","12":"category-quant-regions","13":"category-r-development","14":"tag-data-science","15":"tag-dplyr","16":"tag-proc","17":"tag-r","18":"tag-randomforest","19":"tag-rstats","20":"contributors-categories-theautomatic-net"},"pp_statuses_selecting_workflow":false,"pp_workflow_action":"current","pp_status_selection":"publish","acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v26.9 (Yoast SEO v27.3) - https:\/\/yoast.com\/product\/yoast-seo-premium-wordpress\/ -->\n<title>How to Get an AUC Confidence Interval | IBKR Quant<\/title>\n<meta name=\"description\" content=\"AUC is an important metric in machine learning for classification. It is often used as a measure of a model\u2019s performance.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.interactivebrokers.com\/campus\/wp-json\/wp\/v2\/posts\/186240\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"How to Get an AUC Confidence Interval | IBKR Campus US\" \/>\n<meta property=\"og:description\" content=\"AUC is an important metric in machine learning for classification. It is often used as a measure of a model\u2019s performance.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/how-to-get-an-auc-confidence-interval-2\/\" \/>\n<meta property=\"og:site_name\" content=\"IBKR Campus US\" \/>\n<meta property=\"article:published_time\" content=\"2023-03-07T17:42:06+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2023-03-08T14:37:12+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2023\/02\/abstract-quant-blue-purple-nodes.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1000\" \/>\n\t<meta property=\"og:image:height\" content=\"563\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Andrew Treadway\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Andrew Treadway\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\n\t    \"@context\": \"https:\\\/\\\/schema.org\",\n\t    \"@graph\": [\n\t        {\n\t            \"@type\": \"NewsArticle\",\n\t            \"@id\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/how-to-get-an-auc-confidence-interval-2\\\/#article\",\n\t            \"isPartOf\": {\n\t                \"@id\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/how-to-get-an-auc-confidence-interval-2\\\/\"\n\t            },\n\t            \"author\": {\n\t                \"name\": \"Andrew Treadway\",\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#\\\/schema\\\/person\\\/d4018570a16fb867f1c08412fc9c64bc\"\n\t            },\n\t            \"headline\": \"How to Get an AUC Confidence Interval\",\n\t            \"datePublished\": \"2023-03-07T17:42:06+00:00\",\n\t            \"dateModified\": \"2023-03-08T14:37:12+00:00\",\n\t            \"mainEntityOfPage\": {\n\t                \"@id\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/how-to-get-an-auc-confidence-interval-2\\\/\"\n\t            },\n\t            \"wordCount\": 358,\n\t            \"publisher\": {\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#organization\"\n\t            },\n\t            \"image\": {\n\t                \"@id\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/how-to-get-an-auc-confidence-interval-2\\\/#primaryimage\"\n\t            },\n\t            \"thumbnailUrl\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2023\\\/02\\\/abstract-quant-blue-purple-nodes.jpg\",\n\t            \"keywords\": [\n\t                \"Data Science\",\n\t                \"dplyr\",\n\t                \"pROC\",\n\t                \"R\",\n\t                \"randomForest\",\n\t                \"rstats\"\n\t            ],\n\t            \"articleSection\": [\n\t                \"Data Science\",\n\t                \"Programming Languages\",\n\t                \"Quant\",\n\t                \"Quant Development\",\n\t                \"Quant North America\",\n\t                \"Quant Regions\",\n\t                \"R Development\"\n\t            ],\n\t            \"inLanguage\": \"en-US\"\n\t        },\n\t        {\n\t            \"@type\": \"WebPage\",\n\t            \"@id\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/how-to-get-an-auc-confidence-interval-2\\\/\",\n\t            \"url\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/how-to-get-an-auc-confidence-interval-2\\\/\",\n\t            \"name\": \"How to Get an AUC Confidence Interval | IBKR Campus US\",\n\t            \"isPartOf\": {\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#website\"\n\t            },\n\t            \"primaryImageOfPage\": {\n\t                \"@id\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/how-to-get-an-auc-confidence-interval-2\\\/#primaryimage\"\n\t            },\n\t            \"image\": {\n\t                \"@id\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/how-to-get-an-auc-confidence-interval-2\\\/#primaryimage\"\n\t            },\n\t            \"thumbnailUrl\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2023\\\/02\\\/abstract-quant-blue-purple-nodes.jpg\",\n\t            \"datePublished\": \"2023-03-07T17:42:06+00:00\",\n\t            \"dateModified\": \"2023-03-08T14:37:12+00:00\",\n\t            \"description\": \"AUC is an important metric in machine learning for classification. It is often used as a measure of a model\u2019s performance.\",\n\t            \"inLanguage\": \"en-US\",\n\t            \"potentialAction\": [\n\t                {\n\t                    \"@type\": \"ReadAction\",\n\t                    \"target\": [\n\t                        \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/how-to-get-an-auc-confidence-interval-2\\\/\"\n\t                    ]\n\t                }\n\t            ]\n\t        },\n\t        {\n\t            \"@type\": \"ImageObject\",\n\t            \"inLanguage\": \"en-US\",\n\t            \"@id\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/ibkr-quant-news\\\/how-to-get-an-auc-confidence-interval-2\\\/#primaryimage\",\n\t            \"url\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2023\\\/02\\\/abstract-quant-blue-purple-nodes.jpg\",\n\t            \"contentUrl\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2023\\\/02\\\/abstract-quant-blue-purple-nodes.jpg\",\n\t            \"width\": 1000,\n\t            \"height\": 563,\n\t            \"caption\": \"Hull-White 2-factor Model: 3) Simulation\"\n\t        },\n\t        {\n\t            \"@type\": \"WebSite\",\n\t            \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#website\",\n\t            \"url\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/\",\n\t            \"name\": \"IBKR Campus US\",\n\t            \"description\": \"Financial Education from Interactive Brokers\",\n\t            \"publisher\": {\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#organization\"\n\t            },\n\t            \"potentialAction\": [\n\t                {\n\t                    \"@type\": \"SearchAction\",\n\t                    \"target\": {\n\t                        \"@type\": \"EntryPoint\",\n\t                        \"urlTemplate\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/?s={search_term_string}\"\n\t                    },\n\t                    \"query-input\": {\n\t                        \"@type\": \"PropertyValueSpecification\",\n\t                        \"valueRequired\": true,\n\t                        \"valueName\": \"search_term_string\"\n\t                    }\n\t                }\n\t            ],\n\t            \"inLanguage\": \"en-US\"\n\t        },\n\t        {\n\t            \"@type\": \"Organization\",\n\t            \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#organization\",\n\t            \"name\": \"Interactive Brokers\",\n\t            \"alternateName\": \"IBKR\",\n\t            \"url\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/\",\n\t            \"logo\": {\n\t                \"@type\": \"ImageObject\",\n\t                \"inLanguage\": \"en-US\",\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#\\\/schema\\\/logo\\\/image\\\/\",\n\t                \"url\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2024\\\/05\\\/ibkr-campus-logo.jpg\",\n\t                \"contentUrl\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2024\\\/05\\\/ibkr-campus-logo.jpg\",\n\t                \"width\": 669,\n\t                \"height\": 669,\n\t                \"caption\": \"Interactive Brokers\"\n\t            },\n\t            \"image\": {\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#\\\/schema\\\/logo\\\/image\\\/\"\n\t            },\n\t            \"publishingPrinciples\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/about-ibkr-campus\\\/\",\n\t            \"ethicsPolicy\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/cyber-security-notice\\\/\"\n\t        },\n\t        {\n\t            \"@type\": \"Person\",\n\t            \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#\\\/schema\\\/person\\\/d4018570a16fb867f1c08412fc9c64bc\",\n\t            \"name\": \"Andrew Treadway\",\n\t            \"description\": \"Andrew Treadway currently works as a Senior Data Scientist, and has experience doing analytics, software automation, and ETL. He completed a master\u2019s degree in computer science \\\/ machine learning, and an undergraduate degree in pure mathematics. Connect with him on LinkedIn: https:\\\/\\\/www.linkedin.com\\\/in\\\/andrew-treadway-a3b19b103\\\/In addition to TheAutomatic.net blog, he also teaches in-person courses on Python and R through my NYC meetup: more details.\",\n\t            \"sameAs\": [\n\t                \"https:\\\/\\\/theautomatic.net\\\/about-me\\\/\"\n\t            ],\n\t            \"url\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/author\\\/andrewtreadway\\\/\"\n\t        }\n\t    ]\n\t}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"How to Get an AUC Confidence Interval | IBKR Quant","description":"AUC is an important metric in machine learning for classification. It is often used as a measure of a model\u2019s performance.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.interactivebrokers.com\/campus\/wp-json\/wp\/v2\/posts\/186240\/","og_locale":"en_US","og_type":"article","og_title":"How to Get an AUC Confidence Interval | IBKR Campus US","og_description":"AUC is an important metric in machine learning for classification. It is often used as a measure of a model\u2019s performance.","og_url":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/how-to-get-an-auc-confidence-interval-2\/","og_site_name":"IBKR Campus US","article_published_time":"2023-03-07T17:42:06+00:00","article_modified_time":"2023-03-08T14:37:12+00:00","og_image":[{"width":1000,"height":563,"url":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2023\/02\/abstract-quant-blue-purple-nodes.jpg","type":"image\/jpeg"}],"author":"Andrew Treadway","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Andrew Treadway","Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"NewsArticle","@id":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/how-to-get-an-auc-confidence-interval-2\/#article","isPartOf":{"@id":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/how-to-get-an-auc-confidence-interval-2\/"},"author":{"name":"Andrew Treadway","@id":"https:\/\/ibkrcampus.com\/campus\/#\/schema\/person\/d4018570a16fb867f1c08412fc9c64bc"},"headline":"How to Get an AUC Confidence Interval","datePublished":"2023-03-07T17:42:06+00:00","dateModified":"2023-03-08T14:37:12+00:00","mainEntityOfPage":{"@id":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/how-to-get-an-auc-confidence-interval-2\/"},"wordCount":358,"publisher":{"@id":"https:\/\/ibkrcampus.com\/campus\/#organization"},"image":{"@id":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/how-to-get-an-auc-confidence-interval-2\/#primaryimage"},"thumbnailUrl":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2023\/02\/abstract-quant-blue-purple-nodes.jpg","keywords":["Data Science","dplyr","pROC","R","randomForest","rstats"],"articleSection":["Data Science","Programming Languages","Quant","Quant Development","Quant North America","Quant Regions","R Development"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/how-to-get-an-auc-confidence-interval-2\/","url":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/how-to-get-an-auc-confidence-interval-2\/","name":"How to Get an AUC Confidence Interval | IBKR Campus US","isPartOf":{"@id":"https:\/\/ibkrcampus.com\/campus\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/how-to-get-an-auc-confidence-interval-2\/#primaryimage"},"image":{"@id":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/how-to-get-an-auc-confidence-interval-2\/#primaryimage"},"thumbnailUrl":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2023\/02\/abstract-quant-blue-purple-nodes.jpg","datePublished":"2023-03-07T17:42:06+00:00","dateModified":"2023-03-08T14:37:12+00:00","description":"AUC is an important metric in machine learning for classification. It is often used as a measure of a model\u2019s performance.","inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/how-to-get-an-auc-confidence-interval-2\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/how-to-get-an-auc-confidence-interval-2\/#primaryimage","url":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2023\/02\/abstract-quant-blue-purple-nodes.jpg","contentUrl":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2023\/02\/abstract-quant-blue-purple-nodes.jpg","width":1000,"height":563,"caption":"Hull-White 2-factor Model: 3) Simulation"},{"@type":"WebSite","@id":"https:\/\/ibkrcampus.com\/campus\/#website","url":"https:\/\/ibkrcampus.com\/campus\/","name":"IBKR Campus US","description":"Financial Education from Interactive Brokers","publisher":{"@id":"https:\/\/ibkrcampus.com\/campus\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/ibkrcampus.com\/campus\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/ibkrcampus.com\/campus\/#organization","name":"Interactive Brokers","alternateName":"IBKR","url":"https:\/\/ibkrcampus.com\/campus\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/ibkrcampus.com\/campus\/#\/schema\/logo\/image\/","url":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/05\/ibkr-campus-logo.jpg","contentUrl":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/05\/ibkr-campus-logo.jpg","width":669,"height":669,"caption":"Interactive Brokers"},"image":{"@id":"https:\/\/ibkrcampus.com\/campus\/#\/schema\/logo\/image\/"},"publishingPrinciples":"https:\/\/www.interactivebrokers.com\/campus\/about-ibkr-campus\/","ethicsPolicy":"https:\/\/www.interactivebrokers.com\/campus\/cyber-security-notice\/"},{"@type":"Person","@id":"https:\/\/ibkrcampus.com\/campus\/#\/schema\/person\/d4018570a16fb867f1c08412fc9c64bc","name":"Andrew Treadway","description":"Andrew Treadway currently works as a Senior Data Scientist, and has experience doing analytics, software automation, and ETL. He completed a master\u2019s degree in computer science \/ machine learning, and an undergraduate degree in pure mathematics. Connect with him on LinkedIn: https:\/\/www.linkedin.com\/in\/andrew-treadway-a3b19b103\/In addition to TheAutomatic.net blog, he also teaches in-person courses on Python and R through my NYC meetup: more details.","sameAs":["https:\/\/theautomatic.net\/about-me\/"],"url":"https:\/\/www.interactivebrokers.com\/campus\/author\/andrewtreadway\/"}]}},"jetpack_featured_media_url":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2023\/02\/abstract-quant-blue-purple-nodes.jpg","_links":{"self":[{"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/posts\/186240","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/users\/388"}],"replies":[{"embeddable":true,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/comments?post=186240"}],"version-history":[{"count":0,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/posts\/186240\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/media\/183462"}],"wp:attachment":[{"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/media?parent=186240"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/categories?post=186240"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/tags?post=186240"},{"taxonomy":"contributors-categories","embeddable":true,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/contributors-categories?post=186240"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}