{"id":207712,"date":"2024-06-07T11:02:53","date_gmt":"2024-06-07T15:02:53","guid":{"rendered":"https:\/\/ibkrcampus.com\/?p=207712"},"modified":"2024-08-13T16:19:27","modified_gmt":"2024-08-13T20:19:27","slug":"understanding-the-lstm-model","status":"publish","type":"post","link":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/understanding-the-lstm-model\/","title":{"rendered":"Understanding the LSTM Model"},"content":{"rendered":"\n

This post gives a brief introduction to the basic structure of LSTM (Long-Short Term Memory) model with an intuitive figures and explanations. Let’s interpret “gate”<\/strong> as “rate”<\/strong> or “weight”<\/strong> for grasping the concept of LSTM faster since we are not engineers but finance-related practitioners.<\/p>\n\n\n\n

Understanding the LSTM model<\/h2>\n\n\n\n

I will explain the LSTM model and then talk about the RNN model by eliminating the LSTM gates one by one. Now let’s understand the structure of LSTM model.<\/p>\n\n\n\n

Activation function : Sigmoid versus Tanh<\/strong><\/p>\n\n\n\n

LSTM uses two types of activation functions: sigmoid (S) and hyperbolic tangent or tanh (T). The sigmoid<\/strong> determines the ratio or rate of information<\/strong> and tanh<\/strong> normalizes information into the range between -1 and 1<\/strong>. These two activation function have the following responses respectively.<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n
\"\"<\/figure>\n\n\n\n

Input, Output (State) and Memory<\/h3>\n\n\n\n

LSTM is a function so that it has input (xt\u22121<\/sub><\/strong><\/em>, ht\u22121<\/sub><\/em><\/strong>) and output (ht<\/sub><\/strong><\/em>). It is a variant of RNN model and has two temporal dependence mechanism: short-term memory (\u210e????<\/sub><\/em><\/strong>) and long-term memory (Ct<\/sub><\/em><\/strong>).<\/p>\n\n\n\n

These variables are summarized as follows.<\/p>\n\n\n\n