IBKR Quant Blog


1 2 3 4 5 2 63


Quant

Top 10 Machine Learning Algorithms For Beginners - Part VIII


Recurrent Neural Networks (RNN)

 

Did you know Siri and Google Assistant use RNN in their programming? RNNs are essentially a type of neural network, which have a memory attached to each node. which makes it easy to process sequential data, i.e. one data unit is dependent on the previous one.
 
A way to explain the advantage of RNN over a normal neural network is that we are supposed to process a word character by character. If the word is “trading”, a normal neural network node would forget the character “t” by the time it moves to “d”, whereas a recurrent neural network will remember the character as it has its own memory.
 

Quant

Learn more QuantInsti here https://www.quantinsti.com

To learn more about Python and R, visit QuantInsti website and their educational offerings at their Executive Programme in Algorithmic Trading (EPAT™).

Trading on margin is only for sophisticated investors with high risk tolerance. You may lose more than your initial investment.


This material is from QuantInsti and is being posted with QuantInsti’s permission. The views expressed in this material are solely those of the author and/or QuantInsti and IBKR is not endorsing or recommending any investment or trading discussed in the material. This material is not and should not be construed as an offer to sell or the solicitation of an offer to buy any security. To the extent that this material discusses general market activity, industry or sector trends or other broad based economic or political conditions, it should not be construed as research or investment advice. To the extent that it includes references to specific securities, commodities, currencies, or other instruments, those references do not constitute a recommendation to buy, sell or hold such security. This material does not and is not intended to take into account the particular financial conditions, investment objectives or requirements of individual customers. Before acting on this material, you should consider whether it is suitable for your particular circumstances and, as necessary, seek professional advice.


23787




Quant

Detecting Bots On Twitter Using Botometer - Part I


In this blog, we will learn what bots are and how they can skew the sentiment analysis used in your trading strategy. We will cover the following topics:

When we perform trading on the basis of market sentiment, we need to fetch data from news sources such as Twitter, Reuters, Bloomberg and Webhosie, etc. Although reading complete articles and gauging their sentiment can be difficult, estimating the sentiment of a tweet is not that complicated.

But before you estimate the sentiment of a tweet you need to know if the tweet was an automated response of a bot or made by a human.

You may ask why this is relevant?

Why should we identify a bot?

It is relevant because you need to know what the bots are doing, which in turn will tell you how the sentiment of a particular stock on Twitter is being manipulated. When we calculate the Twitter sentiment of a particular stock, we identify and remove those tweets made by bot users. This will give the true sentiment sans manipulation. This true sentiment can be a very powerful metric, when used with other technical indicators, to call the tops and bottoms of a trend.

Botometer

In python, we use the library called botometer to know if a particular tweet was made by a bot or not.
The botometer library uses a machine learning algorithm trained on tens of thousands of labelled data. This algorithm’s output is a probability on a scale of 0 to 1, where 1 indicates that a Twitter account is managed by a bot.

The Botometer API takes the user id as the input and then extracts 1200 features related to that user to compute a score. The Botometer gives separate scores for the following categories:

  1. Network features
  2. User features
  3. Friends features
  4. Temporal features
  5. Content features
  6. Sentiment features

Let us discuss some of these features.

Network features

Network features of a user include information on the retweets, mentions, and hashtags that a user tweeted in the past.

For example, If the user is retweeting only those tweets made by a particular handle, then the user is most likely a bot.

User features

This contains user-specific information such as the user name, language, location, account created date, etc., Generally, bots do not contain such information. And if they do, it will be something gibberish.

 

 

Learn more QuantInsti here https://www.quantinsti.com

To learn more about Python and R, visit QuantInsti website and their educational offerings at their Executive Programme in Algorithmic Trading (EPAT™).

This material is from QuantInsti and is being posted with QuantInsti’s permission. The views expressed in this material are solely those of the author and/or QuantInsti and IBKR is not endorsing or recommending any investment or trading discussed in the material. This material is not and should not be construed as an offer to sell or the solicitation of an offer to buy any security. To the extent that this material discusses general market activity, industry or sector trends or other broad based economic or political conditions, it should not be construed as research or investment advice. To the extent that it includes references to specific securities, commodities, currencies, or other instruments, those references do not constitute a recommendation to buy, sell or hold such security. This material does not and is not intended to take into account the particular financial conditions, investment objectives or requirements of individual customers. Before acting on this material, you should consider whether it is suitable for your particular circumstances and, as necessary, seek professional advice.


23825




Quant

Towards Better Keras Modeling - Part III


 

See the first installments of this series here

 In today’s post, The Alpha Scientist walks us through how to set up Talos and Numerox.

 

Looking inside of one of these dataframes, you'll note a few things:

  1. The last several columns are named 'bernie', 'elizabeth' etc... These are code names for different outcome variables the data can be used to generate. The key point is that each of these outcomes is independent of each other (not a multi-class problem, instead several one-class problems).
  2. The features are labeled as only x1 to x50, not with names that provide any hint to what they represent. This is by design - part of numerai's approach to abstracting real financial market data into 100% data science, 0% market knowledge.
  3. The features have already been scaled and standardized. Allegedly, the data has also been carefully cleansed as well.
    Next, we'll define a simple function to split each dataframe into features (X), labels (y), and two other series regions and eras, which we will discard (they are not available on the out of sample data, so we definitely don't want to train models using them!)

 

We'll choose one target variable to use as our target label. Here, I'll choose elizabeth.

 

Quant

Now that we have data, I'll train a very basic logistic regression model to serve as baseline:

 

Quant

 

Quant

 

I've calculated the log-loss values and class accuracies for training and validation sets, as well as for a naive strategy of always guessing 50%. You'll note that the model has learned something but not all that much.

This is the unfortunate reality of using machine learning to form likely outcomes in the financial markets. Edge exists, but it is very, very slight. There is also an upside in this reality.

Achieving a model, which truly offers 51% (or better yet, 52/53/54%) accuracy out of a sample can potentially be extraordinarily profitable, given highly liquid markets and careful attention to transaction costs. And the reward for moving from 51/49 advantage to 52/48 is a doubling in potential profit.
_ actually more than double, when considering transaction costs_

If we wanted to improve upon this result with a "representation" model, there would be any number of tactics to employ. However, since the purpose of this post is to explore hyperparameter optimization of keras models, I won't bother.

 

Enter, Talos...

At this point, you may want to take a minute or two to read the talos docs and example notebooks.

TL;DR: essentially a talos workflow, involves (1) creating a dict of parameter values to evaluate, (2) defining your keras model within a buildmodel as you may already do, but with a few small modifications in format, and (3) running a "Scan" method.

 

 

In the next installment, the Alpha Scientist will continue with the demonstration of Talos.

----------------

About The Alpha Scientist

I'm Chad, aka The Alpha Scientist. I've created The Alpha Scientist blog to explore the intersection of my two professional passions: locating "alpha" in market inefficiencies and applying data science methods. If you've found this post useful, please follow @data2alpha on Twitter and forward to a friend or colleague who may also find this topic interesting. https://alphascientist.com/

 

This material is from The Alpha Scientist and is being posted with The Alpha Scientist’s permission. The views expressed in this material are solely those of the author and/or The Alpha Scientist and IBKR is not endorsing or recommending any investment or trading discussed in the material. This material is not and should not be construed as an offer to sell or the solicitation of an offer to buy any security. To the extent that this material discusses general market activity, industry or sector trends or other broad based economic or political conditions, it should not be construed as research or investment advice. To the extent that it includes references to specific securities, commodities, currencies, or other instruments, those references do not constitute a recommendation to buy, sell or hold such security. This material does not and is not intended to take into account the particular financial conditions, investment objectives or requirements of individual customers. Before acting on this material, you should consider whether it is suitable for your particular circumstances and, as necessary, seek professional advice.


23824




Quant

RSI Indicator - Part II


In in the first installment, Rekhit discussed Steps 1-2: Closing Price, Changes in Closing Price and Gain and Loss.

Step 4: Average Gain and Loss

In the RSI indicator, to smoothen the price movement, we take an average of the gains (and losses) for a certain period.

While we call it an average, a little explanation would be needed. For the first 14 periods, it is a simple average of the values.

To explain it, we will look at the average gain column.

Thus, in the table, the first 14 values would be from (25-04) to (14-05) which is, (0.00 + 4.79 + 8.60 + 0.00 + 6.02 + 1.23 + 0.00 + 9.64 + 8.68 + 0.00 + 4.88 + 0.00 + 0.00 + 0.00)/14 = 3.13.

Now, since we are placing more emphasis on the recent values, for the next set of values, we use the following formula,

[(Previous avg. gain)*13)+ current gain)]/14.

Thus, for (15-05), we will calculate the average gain as [(3.13*13)+0.00]/14 = 2.91.

Similarly, we will calculate the average Loss, too.

Based on these formulae, the table is updated for the columns “Avg Gain (5)” and “Avg Loss (6)”.

Step 5: Calculate RS

Now, to make matters simple, we add a column called “RS”, which is simply (Avg Gain)/(Avg Loss). Thus, for 14-05, RS = (Avg Gain)/(Avg Loss) = 3.13/2.52 = 1.24.

In this manner, the table for the column “RS (7)” is updated. In the next step, we finally work out the RSI values.

 Step 6: Calculation of RSI

RSI = [100 - (100/{1+ RS})].

For example, for (14-05),

RSI = [100 – (100/{1+ RS})] = [100 – (100/{1+ 1.24})] = 55.37.

In this manner, the table is updated.

This is how we get the value of RSI. The RSI indicator graph is always created with respect to the closing price. For example, if we take into account the values of RSI in the above table, the graph should be as follows:
 

Quant

While the formula for RSI mentioned here is generally accepted, we can have certain variations on the formula based on the weightage given to the Average gain or loss values. It is a known fact that as the number of data points increase, the RSI Indicator gets better at gauging the market. Certain sources recommend at least 250 data points before you start making trading decisions on them.

One of the reasons why the RSI indicator is popular is because its value is always between 0 and 100. This makes it easy to read and form opinions about the asset. Furthermore, it is generally accepted that a value below 30 indicates “oversold,” while a value over 70 indicates “overbought”.

However, it is not as easy as it looks, and if we try to enter and exit the market based on this factor alone, we would not be able to trade effectively, as there are numerous occasions where this advice can fail. You can see in the graph below that the closing price continued to rise even after entering the overbought zone from March 19 to April 25.


 

Quant

 

 

In the next article, Rekhit will discuss few strategies based on the RSI indicator.

 

Learn more QuantInsti here https://www.quantinsti.com

To learn more about Python and R, visit QuantInsti website and their educational offerings at their Executive Programme in Algorithmic Trading (EPAT™).

This material is from QuantInsti and is being posted with QuantInsti’s permission. The views expressed in this material are solely those of the author and/or QuantInsti and IBKR is not endorsing or recommending any investment or trading discussed in the material. This material is not and should not be construed as an offer to sell or the solicitation of an offer to buy any security. To the extent that this material discusses general market activity, industry or sector trends or other broad based economic or political conditions, it should not be construed as research or investment advice. To the extent that it includes references to specific securities, commodities, currencies, or other instruments, those references do not constitute a recommendation to buy, sell or hold such security. This material does not and is not intended to take into account the particular financial conditions, investment objectives or requirements of individual customers. Before acting on this material, you should consider whether it is suitable for your particular circumstances and, as necessary, seek professional advice.


23820




Quant

What is the TWS API?


In this initial lesson we introduce Trader Workstation (TWS), IBKR's flagship desktop trading application, and the TWS Application Programming Interface (API). An overview is provided of what the TWS API is, what it can provide, and where it can be found. We discuss the hardware and software requirements for this course on programming with the TWS API, with experienced programmers as the intended audience. We end by discussing, and answering, the most frequently-asked questions about the API. For the entire course - go to Traders' Academy: ibkr.com/ibta

 

 

 

 

 

Trading on margin is only for sophisticated investors with high risk tolerance. You may lose more than your initial investment.

The analysis in this material is provided for information only and is not and should not be construed as an offer to sell or the solicitation of an offer to buy any security. To the extent that this material discusses general market activity, industry or sector trends or other broad-based economic or political conditions, it should not be construed as research or investment advice. To the extent that it includes references to specific securities, commodities, currencies, or other instruments, those references do not constitute a recommendation by IBKR to buy, sell or hold such investments. This material does not and is not intended to take into account the particular financial conditions, investment objectives or requirements of individual customers. Before acting on this material, you should consider whether it is suitable for your particular circumstances and, as necessary, seek professional advice.


23941




1 2 3 4 5 2 63

Disclosures

We appreciate your feedback. If you have any questions or comments about IBKR Quant Blog please contact ibkrquant@ibkr.com.

The material (including articles and commentary) provided on IBKR Quant Blog is offered for informational purposes only. The posted material is NOT a recommendation by Interactive Brokers (IB) that you or your clients should contract for the services of or invest with any of the independent advisors or hedge funds or others who may post on IBKR Quant Blog or invest with any advisors or hedge funds. The advisors, hedge funds and other analysts who may post on IBKR Quant Blog are independent of IB and IB does not make any representations or warranties concerning the past or future performance of these advisors, hedge funds and others or the accuracy of the information they provide. Interactive Brokers does not conduct a "suitability review" to make sure the trading of any advisor or hedge fund or other party is suitable for you.

Securities or other financial instruments mentioned in the material posted are not suitable for all investors. The material posted does not take into account your particular investment objectives, financial situations or needs and is not intended as a recommendation to you of any particular securities, financial instruments or strategies. Before making any investment or trade, you should consider whether it is suitable for your particular circumstances and, as necessary, seek professional advice. Past performance is no guarantee of future results.

Any information provided by third parties has been obtained from sources believed to be reliable and accurate; however, IB does not warrant its accuracy and assumes no responsibility for any errors or omissions.

Any information posted by employees of IB or an affiliated company is based upon information that is believed to be reliable. However, neither IB nor its affiliates warrant its completeness, accuracy or adequacy. IB does not make any representations or warranties concerning the past or future performance of any financial instrument. By posting material on IB Quant Blog, IB is not representing that any particular financial instrument or trading strategy is appropriate for you.