{"id":235866,"date":"2025-12-11T10:17:18","date_gmt":"2025-12-11T15:17:18","guid":{"rendered":"https:\/\/ibkrcampus.com\/campus\/?p=235866"},"modified":"2025-12-11T10:17:29","modified_gmt":"2025-12-11T15:17:29","slug":"building-blocks-of-bias-variance-tradeoff-for-trading-the-financial-markets","status":"publish","type":"post","link":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/building-blocks-of-bias-variance-tradeoff-for-trading-the-financial-markets\/","title":{"rendered":"Building Blocks of Bias-Variance Tradeoff for Trading the Financial Markets"},"content":{"rendered":"\n<p><em>The article &#8220;Building Blocks of Bias-Variance Tradeoff for Trading the Financial Markets&#8221; was originally published on <a href=\"https:\/\/blog.quantinsti.com\/bias-variance-machine-learning-trading\/\">QuantInsti<\/a> blog.<\/em><\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"prerequisites\"><strong>Prerequisites<\/strong><\/h3>\n\n\n\n<p>To fully grasp the bias-variance tradeoff and its role in trading, it is essential first to build a strong foundation in mathematics, machine learning, and programming.<\/p>\n\n\n\n<p>Start with the fundamental mathematical concepts necessary for algorithmic trading by reading&nbsp;<a href=\"https:\/\/blog.quantinsti.com\/algorithmic-trading-maths\/\">Stock Market Math: Essential Concepts for Algorithmic Trading<\/a>. This will help you develop a strong understanding of algebra, arithmetic, and probability\u2014critical elements in statistical modelling.<\/p>\n\n\n\n<p>Since the bias-variance tradeoff is closely linked to regression models, go through&nbsp;<a href=\"https:\/\/blog.quantinsti.com\/linear-regression\/\">Exploring Linear Regression Analysis in Finance and Trading<\/a>&nbsp;to understand how regression-based predictive models work. To further strengthen your understanding,&nbsp;<a href=\"https:\/\/blog.quantinsti.com\/linear-regression-assumptions-limitations\/\">Linear Regression: Assumptions and Limitations<\/a>&nbsp;explains common pitfalls in linear regression, which are directly related to bias and variance issues in model performance.<\/p>\n\n\n\n<p>Since this blog focuses on a machine learning concept, it&#8217;s crucial to start with the basics.&nbsp;<a href=\"https:\/\/blog.quantinsti.com\/machine-learning-basics\/\">Machine Learning Basics: Components, Application, Resources, and More<\/a>&nbsp;introduces the fundamental aspects of ML, followed by&nbsp;<a href=\"https:\/\/blog.quantinsti.com\/trading-using-machine-learning-python\/\">Machine Learning for Algorithmic Trading in Python: A Complete Guide<\/a>, which demonstrates how ML models are applied in financial markets.If you&#8217;re new to Python, start with&nbsp;<a href=\"https:\/\/blog.quantinsti.com\/python-programming\/\">Basics of Python Programming<\/a>. Additionally, the&nbsp;<a href=\"https:\/\/quantra.quantinsti.com\/course\/python-trading-basic\/\">Python for Trading: Basic<\/a>&nbsp;free course provides a structured approach to learning Python for financial data analysis and trading strategies.<\/p>\n\n\n\n<p>This blog covers:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Generating Random Values with a Uniform Distribution and Fitting Them to a Second-Order Equation<\/li>\n\n\n\n<li>Adding a Noise Component<\/li>\n\n\n\n<li>Splitting into Testing and Training Sets<\/li>\n\n\n\n<li>Fitting Four Different Models to the Data for Illustrating Bias and Variance<\/li>\n\n\n\n<li>Formal Discussion on Bias and Variance<\/li>\n\n\n\n<li>Change in Training and Testing Accuracy with Model Complexity<\/li>\n\n\n\n<li>Mathematical Treatment of Different Error Terms and Decomposition<\/li>\n\n\n\n<li>Values of the MSE, Bias, Variance, and Irreducible Error for the Simulated Data, and Their Intuition<\/li>\n<\/ul>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\" \/>\n\n\n\n<p><strong>Ice Breaker<\/strong><\/p>\n\n\n\n<p>Machine learning model creation is a tightrope walk. You create an easy model, and you end up with an underfit. Increase the complexity, and you end up with an overfitted model. What to do then? Well, that\u2019s the agenda for this blog post. This is the first part of a two-blog series on bias-variance tradeoff and its use in market trading. We\u2019ll explore the fundamental concepts in this first part and discuss the application in the second part.<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\" \/>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"generating-random-values-with-a-uniform-distribution-and-fitting-them-to-a-second-order-equation\">Generating Random Values with a Uniform Distribution and Fitting Them to a Second-Order Equation<\/h2>\n\n\n\n<p>Let\u2019s start with a simple illustration of underfitting and overfitting. Let\u2019s take an equation and plot it. The equation is:<\/p>\n\n\n\n<p class=\"has-text-align-center\">y = 2X<sup>2<\/sup> + 3X + 4<\/p>\n\n\n\n<p>When plotted, this is how it looks like:<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img decoding=\"async\" width=\"720\" height=\"512\" data-src=\"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/Underfitting-and-overfitting.jpg\" alt=\"Building Blocks of Bias-Variance Tradeoff\" class=\"wp-image-235873 lazyload\" data-srcset=\"https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/Underfitting-and-overfitting.jpg 720w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/Underfitting-and-overfitting-700x498.jpg 700w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/Underfitting-and-overfitting-300x213.jpg 300w\" data-sizes=\"(max-width: 720px) 100vw, 720px\" src=\"data:image\/svg+xml;base64,PHN2ZyB3aWR0aD0iMSIgaGVpZ2h0PSIxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPjwvc3ZnPg==\" style=\"--smush-placeholder-width: 720px; aspect-ratio: 720\/512;\" \/><\/figure>\n\n\n\n<p><em>Figure 1: Plot of the second-order polynomial<\/em><\/p>\n\n\n\n<p>Here\u2019s the Python code for plotting the equation:<\/p>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"python\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">import numpy as np\nimport matplotlib.pyplot as plt\n\n# Simulating a second-order polynomial with random input values\nnp.random.seed(42) # For reproducibility\nX = np.random.uniform(-5, 5, 30)\ny = 2 * X**2 + 3 * X + 4\nX = X.reshape(-1, 1)\n\n# Plotting the data\nplt.scatter(X,y, color='blue')<\/pre>\n\n\n\n<p><a href=\"https:\/\/gist.github.com\/quantra-go-algo\/9a421f5af6b80b7309c921315cf4f2a3#file-plotting_equation-py\" target=\"_blank\" rel=\"noreferrer noopener\">Plotting_equation.py&nbsp;<\/a>hosted with \u2764 by&nbsp;<a href=\"https:\/\/github.com\/\" target=\"_blank\" rel=\"noreferrer noopener\">GitHub<\/a><\/p>\n\n\n\n<p>I have assigned random values to X, which range from -5 to 5 and belong to a uniform distribution. Suppose we are given only this scatter plot (not the equation). With some basic math knowledge, we could identify it as a second-order polynomial. We can even do this visually.<\/p>\n\n\n\n<p>But in real settings, things aren\u2019t that straightforward. Any data we gather or analyze will not form such a clear pattern, and there will be a random component. We term this random component as noise. To know more about noise, you can go through&nbsp;<a href=\"https:\/\/blog.quantinsti.com\/random-walk\/\">this<\/a>&nbsp;blog article and also&nbsp;<a href=\"https:\/\/blog.quantinsti.com\/itos-lemma-trading-concepts-guide\/\">this<\/a>&nbsp;one.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"adding-a-noise-component\">Adding a Noise Component<\/h2>\n\n\n\n<p>When we add a noise component to the above equation, this is how it looks like:<\/p>\n\n\n\n<p class=\"has-text-align-center\">y = 2X<sup>2<\/sup> + 3X + 4 + noise<\/p>\n\n\n\n<p>What would its plot look like? Here you go:<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img decoding=\"async\" width=\"720\" height=\"544\" data-src=\"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/Second-order-polynomial-with-noise.jpg\" alt=\"Building Blocks of Bias-Variance Tradeoff\" class=\"wp-image-235874 lazyload\" data-srcset=\"https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/Second-order-polynomial-with-noise.jpg 720w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/Second-order-polynomial-with-noise-700x529.jpg 700w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/Second-order-polynomial-with-noise-300x227.jpg 300w\" data-sizes=\"(max-width: 720px) 100vw, 720px\" src=\"data:image\/svg+xml;base64,PHN2ZyB3aWR0aD0iMSIgaGVpZ2h0PSIxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPjwvc3ZnPg==\" style=\"--smush-placeholder-width: 720px; aspect-ratio: 720\/544;\" \/><\/figure>\n\n\n\n<p><em>Figure 2: Plot of the second-order polynomial with noise<\/em><\/p>\n\n\n\n<p>Do you think it\u2019s as easily interpretable as the previous one? Maybe, since we only have 30 data points, the curve still looks somewhat second-orderish! But we\u2019ll need a deeper analysis when we have many data points, and the underlying equation also starts getting more complex.<\/p>\n\n\n\n<p>Here\u2019s the code for generating the above data points and the plot from Figure 2:<\/p>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"python\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\"># Simulating a second-order polynomial with random input values and noise component\nnp.random.seed(42) # For reproducibility\nX = np.random.uniform(-5, 5, 30)\ny = 2 * X**2 + 3 * X + 4 + np.random.normal(0, 10, X.shape[0])\nX = X.reshape(-1, 1)\n\n# Plotting the data\nplt.scatter(X,y, color='blue')<\/pre>\n\n\n\n<p><a href=\"https:\/\/gist.github.com\/quantra-go-algo\/bc25a76db2ce17063563f7248997f166#file-second_order_polynomial-py\" target=\"_blank\" rel=\"noreferrer noopener\">Second_order_polynomial.py&nbsp;<\/a>hosted with \u2764 by&nbsp;<a href=\"https:\/\/github.com\/\" target=\"_blank\" rel=\"noreferrer noopener\">GitHub<\/a><\/p>\n\n\n\n<p>Looking closely, you\u2019ll realize that the noise component above belongs to a normal distribution, with mean = 0 and standard deviation = 10.<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\" \/>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"splitting-into-testing-and-training-sets\">Splitting into Testing and Training Sets<\/h2>\n\n\n\n<p>Let\u2019s now discuss the meaty part. We shall split the above data into train and test sets, with sizes of 20 and 10, respectively. If you aren\u2019t conversant with these basic machine-learning concepts, I recommend skimming through this free book:&nbsp;<a href=\"https:\/\/www.quantinsti.com\/machine-learning-trading-book\">ML for Trading<\/a>.<\/p>\n\n\n\n<p>This is what the data looks like after splitting:<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img decoding=\"async\" width=\"720\" height=\"524\" data-src=\"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/Second-order-polynomial-after-splitting-into-train-and-test-data-1.png\" alt=\"Building Blocks of Bias-Variance Tradeoff\" class=\"wp-image-235875 lazyload\" data-srcset=\"https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/Second-order-polynomial-after-splitting-into-train-and-test-data-1.png 720w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/Second-order-polynomial-after-splitting-into-train-and-test-data-1-700x509.png 700w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/Second-order-polynomial-after-splitting-into-train-and-test-data-1-300x218.png 300w\" data-sizes=\"(max-width: 720px) 100vw, 720px\" src=\"data:image\/svg+xml;base64,PHN2ZyB3aWR0aD0iMSIgaGVpZ2h0PSIxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPjwvc3ZnPg==\" style=\"--smush-placeholder-width: 720px; aspect-ratio: 720\/524;\" \/><\/figure>\n\n\n\n<p><em>Figure 3: Plot of the second-order polynomial after splitting into train and test data<\/em><\/p>\n\n\n\n<p>Here\u2019s the code for the split and the above plot:<\/p>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"python\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">from sklearn.model_selection import train_test_split\n\n# Splitting into training and testing sets\nX_train, X_test, y_train, y_test = train_test_split(X, y, test_size=10, random_state=42)\n\n# Plotting the data\nplt.scatter(X_train, y_train, color='blue')\nplt.scatter(X_test, y_test, color='red')<\/pre>\n\n\n\n<p><a href=\"https:\/\/gist.github.com\/quantra-go-algo\/e1434483313b298a43e655d1f768484f#file-split_train_test-py\" target=\"_blank\" rel=\"noreferrer noopener\">Split_train_test.py&nbsp;<\/a>hosted with \u2764 by&nbsp;<a href=\"https:\/\/github.com\/\" target=\"_blank\" rel=\"noreferrer noopener\">GitHub<\/a><\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"fitting-four-different-models-to-the-data-for-illustrating-bias-and-variance\">Fitting Four Different Models to the Data for Illustrating Bias and Variance<\/h2>\n\n\n\n<p>After splitting the data, we\u2019ll train four different models with polynomials of order 1, 2, 3, and 10, respectively, and check their accuracies. We\u2019ll do this by using linear regression. We\u2019ll import the \u201cPolynomialFeatures\u201d and \u201cLinearRegression\u201d functionalities from different sub-modules of the scikit-learn library. Let\u2019s see what the four models look like after we fit them on the data, with their respective accuracies:<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img decoding=\"async\" width=\"720\" height=\"576\" data-src=\"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/Underfit-model-Degree-1.jpg\" alt=\"Building Blocks of Bias-Variance Tradeoff\" class=\"wp-image-235878 lazyload\" data-srcset=\"https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/Underfit-model-Degree-1.jpg 720w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/Underfit-model-Degree-1-700x560-1.jpg 700w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/Underfit-model-Degree-1-300x240-1.jpg 300w\" data-sizes=\"(max-width: 720px) 100vw, 720px\" src=\"data:image\/svg+xml;base64,PHN2ZyB3aWR0aD0iMSIgaGVpZ2h0PSIxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPjwvc3ZnPg==\" style=\"--smush-placeholder-width: 720px; aspect-ratio: 720\/576;\" \/><\/figure>\n\n\n\n<p><em>Figure 4a: Underfit model with high bias<\/em><\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img decoding=\"async\" width=\"720\" height=\"574\" data-src=\"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/Proper-fit-model-Degree-2.jpg\" alt=\"equation model\" class=\"wp-image-235879 lazyload\" data-srcset=\"https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/Proper-fit-model-Degree-2.jpg 720w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/Proper-fit-model-Degree-2-700x558.jpg 700w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/Proper-fit-model-Degree-2-300x239.jpg 300w\" data-sizes=\"(max-width: 720px) 100vw, 720px\" src=\"data:image\/svg+xml;base64,PHN2ZyB3aWR0aD0iMSIgaGVpZ2h0PSIxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPjwvc3ZnPg==\" style=\"--smush-placeholder-width: 720px; aspect-ratio: 720\/574;\" \/><\/figure>\n\n\n\n<p><em>Figure 4b: Properly fit model with low bias and low variance (second order)<\/em><\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img decoding=\"async\" width=\"720\" height=\"728\" data-src=\"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/Proper-fit-model-Degree-3.jpg\" alt=\"Building Blocks of Bias-Variance Tradeoff\" class=\"wp-image-235881 lazyload\" data-srcset=\"https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/Proper-fit-model-Degree-3.jpg 720w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/Proper-fit-model-Degree-3-700x708-1.jpg 700w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/Proper-fit-model-Degree-3-300x303-1.jpg 300w\" data-sizes=\"(max-width: 720px) 100vw, 720px\" src=\"data:image\/svg+xml;base64,PHN2ZyB3aWR0aD0iMSIgaGVpZ2h0PSIxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPjwvc3ZnPg==\" style=\"--smush-placeholder-width: 720px; aspect-ratio: 720\/728;\" \/><\/figure>\n\n\n\n<p><em>Figure 4c: Properly fit model with low bias and low variance (third order)<\/em><\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img decoding=\"async\" width=\"720\" height=\"552\" data-src=\"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/Overfit-model-Degree-10.jpg\" alt=\"Building Blocks of Bias-Variance\" class=\"wp-image-235907 lazyload\" data-srcset=\"https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/Overfit-model-Degree-10.jpg 720w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/Overfit-model-Degree-10-700x537-1.jpg 700w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/Overfit-model-Degree-10-300x230-1.jpg 300w\" data-sizes=\"(max-width: 720px) 100vw, 720px\" src=\"data:image\/svg+xml;base64,PHN2ZyB3aWR0aD0iMSIgaGVpZ2h0PSIxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPjwvc3ZnPg==\" style=\"--smush-placeholder-width: 720px; aspect-ratio: 720\/552;\" \/><\/figure>\n\n\n\n<p><em>Figure 4d: Overfit model with high variance<\/em><\/p>\n\n\n\n<p>The above four plots (Figure 4a to Figure 4d) should give you a clear picture of what it looks like when a machine learning model underfits, properly fits, and overfits on the training data. You might wonder why I\u2019m showing you two plots (and thus two different models) for a proper fit. Don\u2019t worry; I\u2019ll discuss this in a couple of minutes.<\/p>\n\n\n\n<p>For now, here\u2019s the code for training the four models and for plotting them:<\/p>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"python\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">from sklearn.preprocessing import PolynomialFeatures\nfrom sklearn.linear_model import LinearRegression\nfrom sklearn.metrics import mean_squared_error\n\n# Generating smooth data for plotting\nX_plot = np.linspace(-5, 5, 1000).reshape(-1, 1)\n\n# Function to train and evaluate models\ndef train_model(degree, X_train, y_train, X_test, y_test):\n    poly = PolynomialFeatures(degree)\n    X_train_poly = poly.fit_transform(X_train)\n    X_test_poly = poly.transform(X_test)\n    model = LinearRegression().fit(X_train_poly, y_train)\n\n    # Training and testing errors\n    y_train_pred = model.predict(X_train_poly)\n    train_error = mean_squared_error(y_train, y_train_pred)\n    y_test_pred = model.predict(X_test_poly)\n    test_error = mean_squared_error(y_test, y_test_pred)\n\n    # Predicting smooth curve for plotting\n    X_plot_poly = poly.transform(X_plot)\n    y_plot = model.predict(X_plot_poly)\n\n    return train_error, test_error, y_plot\n\n# Training models\ndegrees = [1, 2, 3, 10]\nresults = {}\nfor degree in degrees:\n    train_error, test_error, y_plot = train_model(degree, X_train, y_train, X_test, y_test)\n    results[degree] = (train_error, test_error, y_plot)\n\n# Plotting results\nplt.figure(figsize=(18, 5))\n\n# Underfit (degree 1)\nplt.subplot(1, 4, 1)\nplt.scatter(X_train, y_train, color='blue', label='Training Data')\nplt.scatter(X_test, y_test, color='red', label='Testing Data')\nplt.plot(X_plot, results[1][2], color='black', label='Underfit Model')\nplt.title(f'Underfit Model (Degree 1)\\nTrain MSE: {results[1][0]:.2f}, Test MSE: {results[1][1]:.2f}')\nplt.legend()\n\n# Proper Fit (degree 2)\nplt.subplot(1, 4, 2)\nplt.scatter(X_train, y_train, color='blue', label='Training Data')\nplt.scatter(X_test, y_test, color='red', label='Testing Data')\nplt.plot(X_plot, results[2][2], color='green', label='Proper Fit Model')\nplt.title(f'Proper Fit Model (Degree 2)\\nTrain MSE: {results[2][0]:.2f}, Test MSE: {results[2][1]:.2f}')\nplt.legend()\n\n# Proper Fit (degree 3)\nplt.subplot(1, 4, 3)\nplt.scatter(X_train, y_train, color='blue', label='Training Data')\nplt.scatter(X_test, y_test, color='red', label='Testing Data')\nplt.plot(X_plot, results[3][2], color='brown', label='Proper Fit Model')\nplt.title(f'Proper Fit Model (Degree 3)\\nTrain MSE: {results[3][0]:.2f}, Test MSE: {results[3][1]:.2f}')\nplt.legend()\n\n# Overfit (degree 10)\nplt.subplot(1, 4, 4)\nplt.scatter(X_train, y_train, color='blue', label='Training Data')\nplt.scatter(X_test, y_test, color='red', label='Testing Data')\nplt.plot(X_plot, results[10][2], color='purple', label='Overfit Model')\nplt.title(f'Overfit Model (Degree 10)\\nTrain MSE: {results[10][0]:.2f}, Test MSE: {results[10][1]:.2f}')\nplt.legend()\n\nplt.tight_layout()\nplt.show()<\/pre>\n\n\n\n<p><a href=\"https:\/\/gist.github.com\/quantra-go-algo\/b205c4b7077ab665d8a6083d6a0850c8#file-training_the_four_models-py\" target=\"_blank\" rel=\"noreferrer noopener\">Training_the_four_models.py&nbsp;<\/a>hosted with \u2764 by&nbsp;<a href=\"https:\/\/github.com\/\" target=\"_blank\" rel=\"noreferrer noopener\">GitHub<\/a><\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"formal-discussion-on-bias-and-variance\">Formal Discussion on Bias and Variance<\/h2>\n\n\n\n<p>Let\u2019s understand underfitting and overfitting better now.<\/p>\n\n\n\n<p>Underfitting is also termed as \u201cbias\u201d. An underfit model doesn\u2019t align with many points in the training data and carries on its own path. It doesn\u2019t allow the data points to modify itself much. You can think of an underfit model as a person with a mind that\u2019s mostly, if not wholly, closed to the ideas, methods, and opinions of others and always carries a mental bias toward things. An underfit model, or a model with high bias, is simplistic and can\u2019t capture much essence or inherent information in the training data. Thus, it can not generalize well to the testing (unseen) data. Both the training and testing accuracies of such a model are low.<\/p>\n\n\n\n<p>Overfitting is referred to as \u201cvariance\u201d. In this case, the model aligns itself with most, if not all, data points in the training set. You can think of a model which overfits as a fickle-minded person who always sways on the opinions and decisions of others, and does not have any conviction of hertheir own. An overfit model, or a model with a high variance, tries to capture every minute detail of the training data, including the noise, so much so that it can\u2019t generalize to the testing (unseen) data. In the case of a model with an overfit, the training accuracy is high, but the testing accuracy is low.<\/p>\n\n\n\n<p>In the case of a properly fit model, we get low errors on both the training and testing data. For the given example, since we already know the equation to be a second-order polynomial, we should expect a second-order polynomial to yield the minimum testing and training errors. However, as you can see from the above results, the third-order polynomial model gives fewer errors on the training and the testing data. What\u2019s the reason for this? Remember the noise term? Yup, that\u2019s the primary reason for this discrepancy. What\u2019s the secondary reason, then? The low number of data points!<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\" \/>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"change-in-training-and-testing-accuracy-with-model-complexity\">Change in Training and Testing Accuracy with Model Complexity<\/h2>\n\n\n\n<p>Let\u2019s plot the training and testing accuracies of all four models:<\/p>\n\n\n\n<figure class=\"wp-block-image size-full is-resized\"><img decoding=\"async\" width=\"720\" height=\"569\" data-src=\"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/Training-and-testing-errors-for-all-the-four-models.png\" alt=\"Building Blocks of Bias-Variance Tradeoff\" class=\"wp-image-235883 lazyload\" style=\"--smush-placeholder-width: 720px; aspect-ratio: 720\/569;width:950px;height:auto\" data-srcset=\"https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/Training-and-testing-errors-for-all-the-four-models.png 720w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/Training-and-testing-errors-for-all-the-four-models-700x553.png 700w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/Training-and-testing-errors-for-all-the-four-models-300x237.png 300w\" data-sizes=\"(max-width: 720px) 100vw, 720px\" src=\"data:image\/svg+xml;base64,PHN2ZyB3aWR0aD0iMSIgaGVpZ2h0PSIxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPjwvc3ZnPg==\" \/><\/figure>\n\n\n\n<p><em>Figure 5: Training and testing errors for all the four models<\/em><\/p>\n\n\n\n<p>From Figure 5 above, we can infer the following:<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>The training and testing errors are pretty high for the underfit model.<\/li>\n\n\n\n<li>The training and testing errors are around the lowest for the proper fit model\/s.<\/li>\n\n\n\n<li>The training error is low (even lower than the proper fit models), but the testing error is high.<\/li>\n\n\n\n<li>The testing error is higher than the training error in all cases.<\/li>\n<\/ol>\n\n\n\n<p>Oh, and yes, here\u2019s the code for the above calculation and the plots:<\/p>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"python\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\"># Calculating training and testing errors for all models\ntrain_errors = [results[degree][0] for degree in degrees]\ntest_errors = [results[degree][1] for degree in degrees]\n\n# Plotting the training and testing errors\nplt.figure(figsize=(8, 6))\nplt.plot(degrees, train_errors, marker='o', label='Training Error')\nplt.plot(degrees, test_errors, marker='o', label='Testing Error')\nplt.xlabel('Model Complexity (Polynomial Degree)')\nplt.ylabel('Mean Squared Error')\nplt.title('Training and Testing Errors for Different Model Complexities')\nplt.xticks(degrees)  # Ensure all degrees are shown on x-axis\nplt.legend()\nplt.grid(True)\nplt.show()<\/pre>\n\n\n\n<p><a href=\"https:\/\/gist.github.com\/quantra-go-algo\/45c51341513aacbb0f6e86bd39782a81#file-calculating_training_testing_errors-py\" target=\"_blank\" rel=\"noreferrer noopener\">Calculating_training_testing_errors.py&nbsp;<\/a>hosted with \u2764 by&nbsp;<a href=\"https:\/\/github.com\/\" target=\"_blank\" rel=\"noreferrer noopener\">GitHub<\/a><\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"mathematical-treatment-of-different-error-terms-and-decomposition\">Mathematical Treatment of Different Error Terms and Decomposition<\/h2>\n\n\n\n<p>Now, with that out of the way, let\u2019s proceed towards more mathematical definitions of bias and variance.<\/p>\n\n\n\n<p>Let\u2019s begin with the loss function. In machine learning parlance, the loss function is the function that we desire to minimize. Only after we get the least possible value of the loss function can we say that we have trained or fit the model well.<\/p>\n\n\n\n<p>The mean square error (MSE) is one such loss function. If you\u2019re familiar with the MSE, you\u2019ll know that the lower the MSE, the more accurate the model.<\/p>\n\n\n\n<p>The equation for the MSE is:<\/p>\n\n\n\n<p>Thus,<\/p>\n\n\n\n<p class=\"has-text-align-center\">MSE = Bias<sup>2<\/sup> + Variance<\/p>\n\n\n\n<p>From the above equation, it is apparent that to reduce the error, we need to reduce either or both from bias and variance. However, since lowering either of these leads to a rise in the other, we need to develop a combination of both, which yields the minimum value for the MSE. So, if we do that and are lucky, can we end up with an MSE value of 0? Well, not quite! Apart from the bias and variance terms, there&#8217;s another term that we need to add here. Owing to the inherent nature of any observed\/recorded data, there is some noise in it, which comprises that part of the error we can&#8217;t reduce. We term this part as the irreducible error. Thus, the equation for MSE becomes:<\/p>\n\n\n\n<p class=\"has-text-align-center\">MSE = Bias<sup>2<\/sup> + Variance + Irreducible Error<\/p>\n\n\n\n<p>Let&#8217;s develop an intuition using the same simulated dataset as before.<\/p>\n\n\n\n<p>We shall tweak the equation for MSE:<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" width=\"1100\" height=\"77\" data-src=\"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/building-blocks-quantinsti-7-1100x77.png\" alt=\"equation\" class=\"wp-image-235916 lazyload\" data-srcset=\"https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/building-blocks-quantinsti-7-1100x77.png 1100w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/building-blocks-quantinsti-7-700x49.png 700w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/building-blocks-quantinsti-7-300x21.png 300w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/building-blocks-quantinsti-7-768x54.png 768w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/building-blocks-quantinsti-7.png 1381w\" data-sizes=\"(max-width: 1100px) 100vw, 1100px\" src=\"data:image\/svg+xml;base64,PHN2ZyB3aWR0aD0iMSIgaGVpZ2h0PSIxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPjwvc3ZnPg==\" style=\"--smush-placeholder-width: 1100px; aspect-ratio: 1100\/77;\" \/><\/figure>\n\n\n\n<p class=\"has-text-align-left\">How and why did we do this? To get into the details, refer to\u00a0<a href=\"https:\/\/ieeexplore.ieee.org\/document\/6797087\" target=\"_blank\" rel=\"noreferrer noopener\">Neural Networks and the Bias\/Variance Dilemma&#8221; by Stuart German<\/a>.<\/p>\n\n\n\n<p>Basis the above paper, the equation for the MSE is:<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" width=\"1100\" height=\"66\" data-src=\"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/building-blocks-quantinsti-2-1100x66.png\" alt=\"equation for the MSE\" class=\"wp-image-235910 lazyload\" data-srcset=\"https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/building-blocks-quantinsti-2-1100x66.png 1100w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/building-blocks-quantinsti-2-700x42.png 700w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/building-blocks-quantinsti-2-300x18.png 300w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/building-blocks-quantinsti-2-768x46.png 768w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/building-blocks-quantinsti-2.png 1414w\" data-sizes=\"(max-width: 1100px) 100vw, 1100px\" src=\"data:image\/svg+xml;base64,PHN2ZyB3aWR0aD0iMSIgaGVpZ2h0PSIxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPjwvc3ZnPg==\" style=\"--smush-placeholder-width: 1100px; aspect-ratio: 1100\/66;\" \/><\/figure>\n\n\n\n<p>where,<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" width=\"1100\" height=\"302\" data-src=\"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/building-blocks-quantinsti-3-1100x302.png\" alt=\"equation\" class=\"wp-image-235912 lazyload\" data-srcset=\"https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/building-blocks-quantinsti-3-1100x302.png 1100w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/building-blocks-quantinsti-3-700x192.png 700w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/building-blocks-quantinsti-3-300x82.png 300w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/building-blocks-quantinsti-3-768x211.png 768w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/building-blocks-quantinsti-3.png 1431w\" data-sizes=\"(max-width: 1100px) 100vw, 1100px\" src=\"data:image\/svg+xml;base64,PHN2ZyB3aWR0aD0iMSIgaGVpZ2h0PSIxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPjwvc3ZnPg==\" style=\"--smush-placeholder-width: 1100px; aspect-ratio: 1100\/302;\" \/><\/figure>\n\n\n\n<p>Thus, the bias for each data point is the difference between the mean of all predicted values and the mean of all observed values. Quite intuitively, the lesser this difference, the lesser the bias, and the more accurate the model. But is it really so? Let&#8217;s not forget that we increase the variance when we obtain a fit with a low bias.. How do we define the variance in mathematical terms? Here&#8217;s the equation:<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" width=\"1100\" height=\"98\" data-src=\"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/building-blocks-quantinsti-4-1100x98.png\" alt=\"variance in mathematical terms\" class=\"wp-image-235913 lazyload\" data-srcset=\"https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/building-blocks-quantinsti-4-1100x98.png 1100w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/building-blocks-quantinsti-4-700x62.png 700w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/building-blocks-quantinsti-4-300x27.png 300w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/building-blocks-quantinsti-4-768x68.png 768w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/building-blocks-quantinsti-4.png 1414w\" data-sizes=\"(max-width: 1100px) 100vw, 1100px\" src=\"data:image\/svg+xml;base64,PHN2ZyB3aWR0aD0iMSIgaGVpZ2h0PSIxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPjwvc3ZnPg==\" style=\"--smush-placeholder-width: 1100px; aspect-ratio: 1100\/98;\" \/><\/figure>\n\n\n\n<p>The MSE comprises the above-defined bias and variance terms. However, since the MSE and variance terms are essentially squares of the differences between different y values, we must do the same to the bias term to ensure dimensional homogeneity.<\/p>\n\n\n\n<p>Thus,<\/p>\n\n\n\n<p class=\"has-text-align-center\">MSE = Bias<sup>2<\/sup> + Variance<\/p>\n\n\n\n<p>From the above equation, it is apparent that to reduce the error, we need to reduce either or both from bias and variance. However, since lowering either of these leads to a rise in the other, we need to develop a combination of both, which yields the minimum value for the MSE. So, if we do that and are lucky, can we end up with an MSE value of 0? Well, not quite! Apart from the bias and variance terms, there&#8217;s another term that we need to add here. Owing to the inherent nature of any observed\/recorded data, there is some noise in it, which comprises that part of the error we can&#8217;t reduce. We term this part as the irreducible error. Thus, the equation for MSE becomes:<\/p>\n\n\n\n<p class=\"has-text-align-center\">MSE = Bias<sup>2<\/sup> + Variance + IrreducibleError<\/p>\n\n\n\n<p>Let&#8217;s develop an intuition using the same simulated dataset as before.<\/p>\n\n\n\n<p>We shall tweak the equation for MSE:<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" width=\"1100\" height=\"94\" data-src=\"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/building-blocks-quantinsti-8-1100x94.png\" alt=\"equation\" class=\"wp-image-235918 lazyload\" data-srcset=\"https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/building-blocks-quantinsti-8-1100x94.png 1100w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/building-blocks-quantinsti-8-700x60.png 700w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/building-blocks-quantinsti-8-300x26.png 300w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/building-blocks-quantinsti-8-768x66.png 768w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/building-blocks-quantinsti-8.png 1405w\" data-sizes=\"(max-width: 1100px) 100vw, 1100px\" src=\"data:image\/svg+xml;base64,PHN2ZyB3aWR0aD0iMSIgaGVpZ2h0PSIxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPjwvc3ZnPg==\" style=\"--smush-placeholder-width: 1100px; aspect-ratio: 1100\/94;\" \/><\/figure>\n\n\n\n<p>How and why did we do this? To get into the details, refer to&nbsp;<a href=\"https:\/\/ieeexplore.ieee.org\/document\/6797087\" target=\"_blank\" rel=\"noreferrer noopener\">Neural Networks and the Bias\/Variance Dilemma&#8221; by Stuart German<\/a>.<\/p>\n\n\n\n<p>Basis the above paper, the equation for the MSE is:<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" width=\"1100\" height=\"57\" data-src=\"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/building-blocks-quantinsti-9-1100x57.png\" alt=\"equation\" class=\"wp-image-235919 lazyload\" data-srcset=\"https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/building-blocks-quantinsti-9-1100x57.png 1100w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/building-blocks-quantinsti-9-700x36.png 700w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/building-blocks-quantinsti-9-300x16.png 300w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/building-blocks-quantinsti-9-768x40.png 768w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/building-blocks-quantinsti-9.png 1402w\" data-sizes=\"(max-width: 1100px) 100vw, 1100px\" src=\"data:image\/svg+xml;base64,PHN2ZyB3aWR0aD0iMSIgaGVpZ2h0PSIxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPjwvc3ZnPg==\" style=\"--smush-placeholder-width: 1100px; aspect-ratio: 1100\/57;\" \/><\/figure>\n\n\n\n<p>where,<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" width=\"1100\" height=\"324\" data-src=\"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/building-blocks-quantinsti-10-1100x324.png\" alt=\"equation\" class=\"wp-image-235920 lazyload\" data-srcset=\"https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/building-blocks-quantinsti-10-1100x324.png 1100w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/building-blocks-quantinsti-10-700x206.png 700w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/building-blocks-quantinsti-10-300x88.png 300w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/building-blocks-quantinsti-10-768x226.png 768w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/building-blocks-quantinsti-10.png 1441w\" data-sizes=\"(max-width: 1100px) 100vw, 1100px\" src=\"data:image\/svg+xml;base64,PHN2ZyB3aWR0aD0iMSIgaGVpZ2h0PSIxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPjwvc3ZnPg==\" style=\"--smush-placeholder-width: 1100px; aspect-ratio: 1100\/324;\" \/><\/figure>\n\n\n\n<p class=\"has-text-align-left\">We won\u2019t discuss anything more here since it is outside the scope of this blog article. You can refer to the paper cited above for a deeper understanding.<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\" \/>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"values-of-the-mse-bias-variance-and-irreducible-error-for-the-simulated-data-and-their-intuition\">Values of the MSE, Bias, Variance, and Irreducible Error for the Simulated Data, and Their Intuition<\/h2>\n\n\n\n<p>We\u2019ll also calculate the bias term and the variance term along with the MSE (using the tweaked formula mentioned above).. This is what the values look like for the testing data:<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img decoding=\"async\" width=\"720\" height=\"247\" data-src=\"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/Testing-errors-for-simulated-data.jpg\" alt=\"Building Blocks of Bias-Variance Tradeoff\" class=\"wp-image-235893 lazyload\" data-srcset=\"https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/Testing-errors-for-simulated-data.jpg 720w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/Testing-errors-for-simulated-data-700x240.jpg 700w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/Testing-errors-for-simulated-data-300x103.jpg 300w\" data-sizes=\"(max-width: 720px) 100vw, 720px\" src=\"data:image\/svg+xml;base64,PHN2ZyB3aWR0aD0iMSIgaGVpZ2h0PSIxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPjwvc3ZnPg==\" style=\"--smush-placeholder-width: 720px; aspect-ratio: 720\/247;\" \/><\/figure>\n\n\n\n<p><em>Table 1: Testing errors for simulated data<\/em><\/p>\n\n\n\n<p>We can make the following observations from Table 1 above:<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>The test errors are minimal for models with orders 2 and 3.<\/li>\n\n\n\n<li>The model with order 0 has the maximum bias term.<\/li>\n\n\n\n<li>The model with order 10 has the maximum variance term.<\/li>\n\n\n\n<li>The irreducible error is either 0 or negligible.<\/li>\n<\/ol>\n\n\n\n<p>Here are the corresponding values for the training dataset:<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img decoding=\"async\" width=\"720\" height=\"228\" data-src=\"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/Training-errors-for-simulated-data.jpg\" alt=\"Building Blocks of Bias-Variance Tradeoff\" class=\"wp-image-235895 lazyload\" data-srcset=\"https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/Training-errors-for-simulated-data.jpg 720w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/Training-errors-for-simulated-data-700x222-1.jpg 700w, https:\/\/ibkrcampus.com\/campus\/wp-content\/uploads\/sites\/2\/2025\/12\/Training-errors-for-simulated-data-300x95-1.jpg 300w\" data-sizes=\"(max-width: 720px) 100vw, 720px\" src=\"data:image\/svg+xml;base64,PHN2ZyB3aWR0aD0iMSIgaGVpZ2h0PSIxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPjwvc3ZnPg==\" style=\"--smush-placeholder-width: 720px; aspect-ratio: 720\/228;\" \/><\/figure>\n\n\n\n<p><em>Table 2: Training errors for simulated data<\/em><\/p>\n\n\n\n<p>We can make the following observations from Table 2 above:<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>The training errors are higher for the three higher order models than the testing errors.<\/li>\n\n\n\n<li>The bias term is negligible for all four models.<\/li>\n\n\n\n<li>The error increases with increasing model complexity.<\/li>\n<\/ol>\n\n\n\n<p>The above three discrepancies can be attributed to our data comprising only 20 train and 10 test data points. How, despite this data sampling, did we not get discrepancies in the test data error calculations? Well, for one, the test data remains unseen by the model, and the model tried to predict values based on what it learned during the training. Secondly, there is an inherent randomness when we work with such small samples, and we may have landed on the luckier side of things with the testing data. Thirdly, we did get a discrepancy, with the irreducible errors being almost 0 in the testing sample. Like I mentioned above, there is always an irreducible error owing to the inherent nature of any data. However, we got no such error since we used data that was simulated by using equations and did not use actual observed data.<\/p>\n\n\n\n<p>The point of the above discussion is not to inspect the values that we got but to derive an intuition of the bias and variance terms. Hope you have a clear picture of these terms now. There\u2019s another term called \u2018decomposition\u2019. It simply refers to the fact that we can \u2018decompose\u2019 the total error of any model into its error owing to bias, error owing to variance, and the inherent irreducible error.<\/p>\n\n\n\n<p>Here\u2019s the code for getting the above tables:<\/p>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"python\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\"># Function to train and evaluate models, and demonstrate the bias-variance decomposition\ndef train_model_decomposition(degree, X_train, y_train, X_test, y_test):\n    poly = PolynomialFeatures(degree)\n    X_train_poly = poly.fit_transform(X_train)\n    X_test_poly = poly.transform(X_test)\n    model = LinearRegression().fit(X_train_poly, y_train)\n\n    # Training and testing errors\n    y_train_pred = model.predict(X_train_poly)\n    train_error = np.mean((y_train_pred - np.mean(y_train))**2)\n    train_bias = (np.mean(y_train_pred) - np.mean(y_train))**2\n    train_variance = np.mean((y_train_pred - np.mean(y_train_pred))**2)\n    y_test_pred = model.predict(X_test_poly)\n    test_error = np.mean((y_test_pred - np.mean(y_test))**2)\n    test_bias = (np.mean(y_test_pred) - np.mean(y_test))**2\n    test_variance = np.mean((y_test_pred - np.mean(y_test_pred))**2)\n\n    # Predicting smooth curve for plotting\n    X_plot_poly = poly.transform(X_plot)\n    y_plot = model.predict(X_plot_poly)\n\n    return train_error, train_bias, train_variance, test_error, test_bias, test_variance, y_plot\n\n# Training models\ndegrees_decomposition = [1, 2, 3, 10]\nresults_decomposition = {}\nfor degree in degrees_decomposition:\n    train_error, train_bias, train_variance, test_error, test_bias, test_variance, y_plot = train_model_decomposition(degree, X_train, y_train, X_test, y_test)\n    results_decomposition[degree] = (train_error, train_bias, train_variance, test_error, test_bias, test_variance, y_plot)<\/pre>\n\n\n\n<p><a href=\"https:\/\/gist.github.com\/quantra-go-algo\/a6e1878f2fde83c2c158a2eeae6f3997#file-training_models_bvt-py\" target=\"_blank\" rel=\"noreferrer noopener\">Training_models_BVT.py&nbsp;<\/a>hosted with \u2764 by&nbsp;<a href=\"https:\/\/github.com\/\" target=\"_blank\" rel=\"noreferrer noopener\">GitHub<\/a><\/p>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"python\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">import pandas as pd\n\n# Creating a dataframe for the testing error decomposition\ndf_test = pd.DataFrame({\n    \"Model_Order\": [1, 2, 3, 10],\n    \"Test Error\": [results_decomposition[1][3], results_decomposition[2][3], results_decomposition[3][3], results_decomposition[10][3]],\n    \"Test Bias\": [results_decomposition[1][4], results_decomposition[2][4], results_decomposition[3][4], results_decomposition[10][4]],\n    \"Test Variance\": [results_decomposition[1][5], results_decomposition[2][5], results_decomposition[3][5], results_decomposition[10][5]]\n})\n\n# Calculating the irreducible testing error\ndf_test['Irreducible Error'] = df_test['Test Error'] - df_test['Test Bias'] - df_test['Test Variance']\ndf_test\n\n# Creating a dataframe for the training error decomposition\ndf_train = pd.DataFrame({\n    \"Model_Order\": [1, 2, 3, 10],\n    \"Train Error\": [results_decomposition[1][0], results_decomposition[2][0], results_decomposition[3][0], results_decomposition[10][0]],\n    \"Train Bias\": [results_decomposition[1][1], results_decomposition[2][1], results_decomposition[3][1], results_decomposition[10][1]],\n    \"Train Variance\": [results_decomposition[1][2], results_decomposition[2][2], results_decomposition[3][2], results_decomposition[10][2]]\n})\n\n# Calculating the irreducible training error\ndf_train['Irreducible Error'] = df_train['Train Error'] - df_train['Train Bias'] - df_train['Train Variance']\ndf_train<\/pre>\n\n\n\n<p><a href=\"https:\/\/gist.github.com\/quantra-go-algo\/0290874172fd3c3b2d85e062bd0856d3#file-error_calculation-py\" target=\"_blank\" rel=\"noreferrer noopener\">Error_calculation.py&nbsp;<\/a>hosted with \u2764 by&nbsp;<a href=\"https:\/\/github.com\/\" target=\"_blank\" rel=\"noreferrer noopener\">GitHub<\/a><\/p>\n\n\n\n<p><strong>Till Next Time<\/strong><\/p>\n\n\n\n<p>Phew! That was a lot! We should stop here for now. In the second part, we\u2019ll explore how to predict market prices and build trading strategies by employing bias-variance decomposition.<\/p>\n\n\n\n<p><em>Visit <a href=\"https:\/\/blog.quantinsti.com\/bias-variance-machine-learning-trading\/\">QuantInsti<\/a> to download the Bias Variance Decomposition &#8211; Python notebook and for additional insights on this topic.<\/em><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Machine Learning model creation is a tightrope walk. You create an easy model, and you end up with an underfit. Increase the complexity, and you end up with an overfitted model.<\/p>\n","protected":false},"author":1553,"featured_media":197903,"comment_status":"open","ping_status":"closed","sticky":true,"template":"","format":"standard","meta":{"_acf_changed":true,"footnotes":""},"categories":[339,349,338,341],"tags":[7257,20923,20928,7560,4404,852,20929,4659,20927,20926,20925,1225,12203,595,20924,4412,6810,11120],"contributors-categories":[13654],"class_list":{"0":"post-235866","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-data-science","8":"category-python-development","9":"category-ibkr-quant-news","10":"category-quant-development","11":"tag-algorithmic-trading","12":"tag-bias-variance-tradeoff","13":"tag-data-simulation","14":"tag-data-visualization","15":"tag-linear-regression","16":"tag-machine-learning","17":"tag-machine-learning-models-in-finance","18":"tag-matplotlib","19":"tag-mean-squared-error-mse","20":"tag-model-overfitting","21":"tag-model-underfitting","22":"tag-numpy","23":"tag-polynomial-regression","24":"tag-python","25":"tag-regression-models","26":"tag-scikit-learn","27":"tag-sklearn","28":"tag-statistical-modeling","29":"contributors-categories-quantinsti"},"pp_statuses_selecting_workflow":false,"pp_workflow_action":"current","pp_status_selection":"publish","acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v26.9 (Yoast SEO v27.4) - https:\/\/yoast.com\/product\/yoast-seo-premium-wordpress\/ -->\n<title>Building Blocks of Bias-Variance Tradeoff for Trading the Financial Markets<\/title>\n<meta name=\"description\" content=\"Machine Learning model creation is a tightrope walk. You create an easy model, and you end up with an underfit..\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.interactivebrokers.com\/campus\/wp-json\/wp\/v2\/posts\/235866\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Building Blocks of Bias-Variance Tradeoff for Trading the Financial Markets\" \/>\n<meta property=\"og:description\" content=\"Machine learning model creation is a tightrope walk. You create an easy model, and you end up with an underfit. Increase the complexity, and you end up with an overfitted model.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/building-blocks-of-bias-variance-tradeoff-for-trading-the-financial-markets\/\" \/>\n<meta property=\"og:site_name\" content=\"IBKR Campus US\" \/>\n<meta property=\"article:published_time\" content=\"2025-12-11T15:17:18+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2025-12-11T15:17:29+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2023\/10\/digital-transformation-blue-background.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1000\" \/>\n\t<meta property=\"og:image:height\" content=\"563\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Mahavir A. Bhattacharya\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Mahavir A. Bhattacharya\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"15 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\n\t    \"@context\": \"https:\\\/\\\/schema.org\",\n\t    \"@graph\": [\n\t        {\n\t            \"@type\": \"NewsArticle\",\n\t            \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/ibkr-quant-news\\\/building-blocks-of-bias-variance-tradeoff-for-trading-the-financial-markets\\\/#article\",\n\t            \"isPartOf\": {\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/ibkr-quant-news\\\/building-blocks-of-bias-variance-tradeoff-for-trading-the-financial-markets\\\/\"\n\t            },\n\t            \"author\": {\n\t                \"name\": \"Mahavir A. Bhattacharya\",\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#\\\/schema\\\/person\\\/338f97d31ffff7f18b4c14903d39d1d3\"\n\t            },\n\t            \"headline\": \"Building Blocks of Bias-Variance Tradeoff for Trading the Financial Markets\",\n\t            \"datePublished\": \"2025-12-11T15:17:18+00:00\",\n\t            \"dateModified\": \"2025-12-11T15:17:29+00:00\",\n\t            \"mainEntityOfPage\": {\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/ibkr-quant-news\\\/building-blocks-of-bias-variance-tradeoff-for-trading-the-financial-markets\\\/\"\n\t            },\n\t            \"wordCount\": 2568,\n\t            \"commentCount\": 0,\n\t            \"publisher\": {\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#organization\"\n\t            },\n\t            \"image\": {\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/ibkr-quant-news\\\/building-blocks-of-bias-variance-tradeoff-for-trading-the-financial-markets\\\/#primaryimage\"\n\t            },\n\t            \"thumbnailUrl\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2023\\\/10\\\/digital-transformation-blue-background.jpg\",\n\t            \"keywords\": [\n\t                \"Algorithmic Trading\",\n\t                \"Bias-Variance Tradeoff\",\n\t                \"Data Simulation\",\n\t                \"Data Visualization\",\n\t                \"Linear Regression\",\n\t                \"Machine Learning\",\n\t                \"Machine Learning Models in Finance\",\n\t                \"Matplotlib\",\n\t                \"Mean Squared Error (MSE)\",\n\t                \"Model Overfitting\",\n\t                \"Model Underfitting\",\n\t                \"NumPy\",\n\t                \"Polynomial regression\",\n\t                \"Python\",\n\t                \"Regression Models\",\n\t                \"Scikit-learn\",\n\t                \"sklearn\",\n\t                \"Statistical Modeling\"\n\t            ],\n\t            \"articleSection\": [\n\t                \"Data Science\",\n\t                \"Python Development\",\n\t                \"Quant\",\n\t                \"Quant Development\"\n\t            ],\n\t            \"inLanguage\": \"en-US\",\n\t            \"potentialAction\": [\n\t                {\n\t                    \"@type\": \"CommentAction\",\n\t                    \"name\": \"Comment\",\n\t                    \"target\": [\n\t                        \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/ibkr-quant-news\\\/building-blocks-of-bias-variance-tradeoff-for-trading-the-financial-markets\\\/#respond\"\n\t                    ]\n\t                }\n\t            ]\n\t        },\n\t        {\n\t            \"@type\": \"WebPage\",\n\t            \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/ibkr-quant-news\\\/building-blocks-of-bias-variance-tradeoff-for-trading-the-financial-markets\\\/\",\n\t            \"url\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/ibkr-quant-news\\\/building-blocks-of-bias-variance-tradeoff-for-trading-the-financial-markets\\\/\",\n\t            \"name\": \"Building Blocks of Bias-Variance Tradeoff for Trading the Financial Markets | IBKR Campus US\",\n\t            \"isPartOf\": {\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#website\"\n\t            },\n\t            \"primaryImageOfPage\": {\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/ibkr-quant-news\\\/building-blocks-of-bias-variance-tradeoff-for-trading-the-financial-markets\\\/#primaryimage\"\n\t            },\n\t            \"image\": {\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/ibkr-quant-news\\\/building-blocks-of-bias-variance-tradeoff-for-trading-the-financial-markets\\\/#primaryimage\"\n\t            },\n\t            \"thumbnailUrl\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2023\\\/10\\\/digital-transformation-blue-background.jpg\",\n\t            \"datePublished\": \"2025-12-11T15:17:18+00:00\",\n\t            \"dateModified\": \"2025-12-11T15:17:29+00:00\",\n\t            \"description\": \"Machine Learning model creation is a tightrope walk. You create an easy model, and you end up with an underfit..\",\n\t            \"inLanguage\": \"en-US\",\n\t            \"potentialAction\": [\n\t                {\n\t                    \"@type\": \"ReadAction\",\n\t                    \"target\": [\n\t                        \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/ibkr-quant-news\\\/building-blocks-of-bias-variance-tradeoff-for-trading-the-financial-markets\\\/\"\n\t                    ]\n\t                }\n\t            ]\n\t        },\n\t        {\n\t            \"@type\": \"ImageObject\",\n\t            \"inLanguage\": \"en-US\",\n\t            \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/ibkr-quant-news\\\/building-blocks-of-bias-variance-tradeoff-for-trading-the-financial-markets\\\/#primaryimage\",\n\t            \"url\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2023\\\/10\\\/digital-transformation-blue-background.jpg\",\n\t            \"contentUrl\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2023\\\/10\\\/digital-transformation-blue-background.jpg\",\n\t            \"width\": 1000,\n\t            \"height\": 563,\n\t            \"caption\": \"Quant\"\n\t        },\n\t        {\n\t            \"@type\": \"WebSite\",\n\t            \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#website\",\n\t            \"url\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/\",\n\t            \"name\": \"IBKR Campus US\",\n\t            \"description\": \"Financial Education from Interactive Brokers\",\n\t            \"publisher\": {\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#organization\"\n\t            },\n\t            \"potentialAction\": [\n\t                {\n\t                    \"@type\": \"SearchAction\",\n\t                    \"target\": {\n\t                        \"@type\": \"EntryPoint\",\n\t                        \"urlTemplate\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/?s={search_term_string}\"\n\t                    },\n\t                    \"query-input\": {\n\t                        \"@type\": \"PropertyValueSpecification\",\n\t                        \"valueRequired\": true,\n\t                        \"valueName\": \"search_term_string\"\n\t                    }\n\t                }\n\t            ],\n\t            \"inLanguage\": \"en-US\"\n\t        },\n\t        {\n\t            \"@type\": \"Organization\",\n\t            \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#organization\",\n\t            \"name\": \"Interactive Brokers\",\n\t            \"alternateName\": \"IBKR\",\n\t            \"url\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/\",\n\t            \"logo\": {\n\t                \"@type\": \"ImageObject\",\n\t                \"inLanguage\": \"en-US\",\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#\\\/schema\\\/logo\\\/image\\\/\",\n\t                \"url\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2024\\\/05\\\/ibkr-campus-logo.jpg\",\n\t                \"contentUrl\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2024\\\/05\\\/ibkr-campus-logo.jpg\",\n\t                \"width\": 669,\n\t                \"height\": 669,\n\t                \"caption\": \"Interactive Brokers\"\n\t            },\n\t            \"image\": {\n\t                \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#\\\/schema\\\/logo\\\/image\\\/\"\n\t            },\n\t            \"publishingPrinciples\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/about-ibkr-campus\\\/\",\n\t            \"ethicsPolicy\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/cyber-security-notice\\\/\"\n\t        },\n\t        {\n\t            \"@type\": \"Person\",\n\t            \"@id\": \"https:\\\/\\\/ibkrcampus.com\\\/campus\\\/#\\\/schema\\\/person\\\/338f97d31ffff7f18b4c14903d39d1d3\",\n\t            \"name\": \"Mahavir A. Bhattacharya\",\n\t            \"url\": \"https:\\\/\\\/www.interactivebrokers.com\\\/campus\\\/author\\\/mahavirbhattacharya\\\/\"\n\t        }\n\t    ]\n\t}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Building Blocks of Bias-Variance Tradeoff for Trading the Financial Markets","description":"Machine Learning model creation is a tightrope walk. You create an easy model, and you end up with an underfit..","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.interactivebrokers.com\/campus\/wp-json\/wp\/v2\/posts\/235866\/","og_locale":"en_US","og_type":"article","og_title":"Building Blocks of Bias-Variance Tradeoff for Trading the Financial Markets","og_description":"Machine learning model creation is a tightrope walk. You create an easy model, and you end up with an underfit. Increase the complexity, and you end up with an overfitted model.","og_url":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/building-blocks-of-bias-variance-tradeoff-for-trading-the-financial-markets\/","og_site_name":"IBKR Campus US","article_published_time":"2025-12-11T15:17:18+00:00","article_modified_time":"2025-12-11T15:17:29+00:00","og_image":[{"width":1000,"height":563,"url":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2023\/10\/digital-transformation-blue-background.jpg","type":"image\/jpeg"}],"author":"Mahavir A. Bhattacharya","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Mahavir A. Bhattacharya","Est. reading time":"15 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"NewsArticle","@id":"https:\/\/ibkrcampus.com\/campus\/ibkr-quant-news\/building-blocks-of-bias-variance-tradeoff-for-trading-the-financial-markets\/#article","isPartOf":{"@id":"https:\/\/ibkrcampus.com\/campus\/ibkr-quant-news\/building-blocks-of-bias-variance-tradeoff-for-trading-the-financial-markets\/"},"author":{"name":"Mahavir A. Bhattacharya","@id":"https:\/\/ibkrcampus.com\/campus\/#\/schema\/person\/338f97d31ffff7f18b4c14903d39d1d3"},"headline":"Building Blocks of Bias-Variance Tradeoff for Trading the Financial Markets","datePublished":"2025-12-11T15:17:18+00:00","dateModified":"2025-12-11T15:17:29+00:00","mainEntityOfPage":{"@id":"https:\/\/ibkrcampus.com\/campus\/ibkr-quant-news\/building-blocks-of-bias-variance-tradeoff-for-trading-the-financial-markets\/"},"wordCount":2568,"commentCount":0,"publisher":{"@id":"https:\/\/ibkrcampus.com\/campus\/#organization"},"image":{"@id":"https:\/\/ibkrcampus.com\/campus\/ibkr-quant-news\/building-blocks-of-bias-variance-tradeoff-for-trading-the-financial-markets\/#primaryimage"},"thumbnailUrl":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2023\/10\/digital-transformation-blue-background.jpg","keywords":["Algorithmic Trading","Bias-Variance Tradeoff","Data Simulation","Data Visualization","Linear Regression","Machine Learning","Machine Learning Models in Finance","Matplotlib","Mean Squared Error (MSE)","Model Overfitting","Model Underfitting","NumPy","Polynomial regression","Python","Regression Models","Scikit-learn","sklearn","Statistical Modeling"],"articleSection":["Data Science","Python Development","Quant","Quant Development"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/ibkrcampus.com\/campus\/ibkr-quant-news\/building-blocks-of-bias-variance-tradeoff-for-trading-the-financial-markets\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/ibkrcampus.com\/campus\/ibkr-quant-news\/building-blocks-of-bias-variance-tradeoff-for-trading-the-financial-markets\/","url":"https:\/\/ibkrcampus.com\/campus\/ibkr-quant-news\/building-blocks-of-bias-variance-tradeoff-for-trading-the-financial-markets\/","name":"Building Blocks of Bias-Variance Tradeoff for Trading the Financial Markets | IBKR Campus US","isPartOf":{"@id":"https:\/\/ibkrcampus.com\/campus\/#website"},"primaryImageOfPage":{"@id":"https:\/\/ibkrcampus.com\/campus\/ibkr-quant-news\/building-blocks-of-bias-variance-tradeoff-for-trading-the-financial-markets\/#primaryimage"},"image":{"@id":"https:\/\/ibkrcampus.com\/campus\/ibkr-quant-news\/building-blocks-of-bias-variance-tradeoff-for-trading-the-financial-markets\/#primaryimage"},"thumbnailUrl":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2023\/10\/digital-transformation-blue-background.jpg","datePublished":"2025-12-11T15:17:18+00:00","dateModified":"2025-12-11T15:17:29+00:00","description":"Machine Learning model creation is a tightrope walk. You create an easy model, and you end up with an underfit..","inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/ibkrcampus.com\/campus\/ibkr-quant-news\/building-blocks-of-bias-variance-tradeoff-for-trading-the-financial-markets\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/ibkrcampus.com\/campus\/ibkr-quant-news\/building-blocks-of-bias-variance-tradeoff-for-trading-the-financial-markets\/#primaryimage","url":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2023\/10\/digital-transformation-blue-background.jpg","contentUrl":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2023\/10\/digital-transformation-blue-background.jpg","width":1000,"height":563,"caption":"Quant"},{"@type":"WebSite","@id":"https:\/\/ibkrcampus.com\/campus\/#website","url":"https:\/\/ibkrcampus.com\/campus\/","name":"IBKR Campus US","description":"Financial Education from Interactive Brokers","publisher":{"@id":"https:\/\/ibkrcampus.com\/campus\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/ibkrcampus.com\/campus\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/ibkrcampus.com\/campus\/#organization","name":"Interactive Brokers","alternateName":"IBKR","url":"https:\/\/ibkrcampus.com\/campus\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/ibkrcampus.com\/campus\/#\/schema\/logo\/image\/","url":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/05\/ibkr-campus-logo.jpg","contentUrl":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2024\/05\/ibkr-campus-logo.jpg","width":669,"height":669,"caption":"Interactive Brokers"},"image":{"@id":"https:\/\/ibkrcampus.com\/campus\/#\/schema\/logo\/image\/"},"publishingPrinciples":"https:\/\/www.interactivebrokers.com\/campus\/about-ibkr-campus\/","ethicsPolicy":"https:\/\/www.interactivebrokers.com\/campus\/cyber-security-notice\/"},{"@type":"Person","@id":"https:\/\/ibkrcampus.com\/campus\/#\/schema\/person\/338f97d31ffff7f18b4c14903d39d1d3","name":"Mahavir A. Bhattacharya","url":"https:\/\/www.interactivebrokers.com\/campus\/author\/mahavirbhattacharya\/"}]}},"jetpack_featured_media_url":"https:\/\/www.interactivebrokers.com\/campus\/wp-content\/uploads\/sites\/2\/2023\/10\/digital-transformation-blue-background.jpg","_links":{"self":[{"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/posts\/235866","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/users\/1553"}],"replies":[{"embeddable":true,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/comments?post=235866"}],"version-history":[{"count":0,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/posts\/235866\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/media\/197903"}],"wp:attachment":[{"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/media?parent=235866"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/categories?post=235866"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/tags?post=235866"},{"taxonomy":"contributors-categories","embeddable":true,"href":"https:\/\/ibkrcampus.com\/campus\/wp-json\/wp\/v2\/contributors-categories?post=235866"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}