Asset Classes

Free investment financial education

Language

Multilingual content from IBKR

Close Navigation
Learn more about IBKR accounts

TWS Python API Market Parameters and Scanners

Lesson 10 of 11
Duration 9:23
Level Intermediate
Close Navigation

Welcome. In this lesson, we will be discussing how to request market scanner parameters, and how to request the market scanner itself.

To begin, we will need to understand which parameters we want to request. This is not technically required, though it would be suggested to be sent before your first ever request. This list is also updated periodically, so you may also wish to make a new call for this list to review it from time to time.

Access Market Scanner Parameters

For this initial program, I will use my simple TestApp class, and in my nextValidId method, I will make a simple call to self.reqScannerParameters(). Next, I can define my scannerParameters method, and I will only receive self and xml therein.

Within my scannerParameters function, I will use some file manipulation to save these details, instead of our usual print. I will save my XML file to my python samples directory, TWS API}samplesPythonTestbed.

I will set that directory to a variable string labelled dir. Then I can type open(dir,’w’).write(xml) to write my whole xml string to that file. Then I can create a quick print statement to say, “Scanner Parameters Received.”

Then I can end my file with my standard app, connect, and run combination I have used in some of our other videos.

I would like to stress that I am saving these details to a file instead of my usual print structure because there is a massive amount of data returned here. And at least in my case, my terminal cannot print all of this data. As a point of reference, this xml file is approximately 2 megabytes.

Now if we run this, we will find the file saved in our Testbed directory. I can open this file up with Visual Studio Code and see all the values present. This list is quite large, so we would encourage you to explore this to find exactly what you are looking for.

If I scroll down a bit, we can see STK.NASDAQ for example, or STK.BATS.

In addition to being able to refine my search for specific value requirements, we can use these location codes to specify the exchanges we would like to operate with. This isn’t unique for just US exchanges.

We can refine this for any country. If I search the document, I will be able to see we have scanners for Hong Kong, Korea, Belgium and more.

Moving in towards the actual filters made available to us, we can see further refinement still. We are able to see values for Volume, or perhaps going through usdVolume so we may see the total value as opposed to share quantity.

I can search nearly any tag from the Trader Workstation, and those values should be made available here. We can search for priceAbove or a variety of 52 Week calculations for stocks, options, and so on.

With the scanner parameters in place, we are set to request the market scanner.  I will be using the same values here that I was using before, with my class, nextValidId. I will once again be import TagValue using “from ibapi.tag_value import *”.

With my basic outline created, I can start building my scanner. I want to receive top stock info for the US. I will start by setting sub to ScannerSubscription() to sub as a variable. Then I will set sub.instrument = “STK”, which uses the “Instruments” tag in the XML file. Per my scanner parameters document, I can include sub.locationCode = “STK.US.MAJOR” from the locationCode tag.  This will give me all major exchanges in the US, including NYSE, Nasdaq, Amex and more.

I will use the scan code “TOP_OPEN_PERC_GAIN” which is based on the scanCode tag in that scanner parameters document.

I will now create a quick variable labelled scan_options as an empty list. And now I can create another list for filter_options. In the filter options list, I will add a few filters I want to specify for my request.

First, I will add TagValue(“volumeAbove”, “10000”) so I can receive only stocks with a volume over 10000.

Then I can add something like TagValue(“marketCapBelow”, “1000”). This would further refine my list to contracts with a lower market cap still.

And finally, I will set TagValue(“priceAbove”, “1’). This will filter out penny stocks, or anything below 1 USD.

All of these values are listed as a filter option within our scanner parameters xml file we downloaded before.

There is no limit to how much or little refinement you can choose to include here, so feel free to experiment. With that said, not every inquiry will result in a response however, as logic will still apply in this scenario.

For example, if I make a call for “Curr Year Earnings Per Share Above” I can set any earnings threshold I like. If I enter a value of 100, while my program will certainly let me make the request, but I must also understand that a company will not make $100 per share for themselves. And so, I can balance my values to return something more realistic like $10.

With that said, let’s go ahead and create our market scanner request. This requires our self.reqScannerSubscription request, and we will include a request ID, our sub variable for our Scanner Subscription object. Then we can add our scan_options list, and then our filter_options list.

We can quickly mov through our EWrapper object.

As you might expect, this is simply def scannerData. This includes variables for self, reqId, and then the contracts rank, contract details, and then distance, benchmark, projection, and legsStr. And then, as usual, I will print out all of these values. Our variables after contractDetails are values which may or may not return depending on your scanner request. You will not always receive legsStr for example, as you may not receive values for combos in your requests.

Now, moving on, we have our scannerDataEnd method, that simply includes self and the request ID. Typically I note that this is an optional field, though that is not necessary the case for scanner subscriptions. If the subscription is not closed, then you will receive an error regarding a duplicate scanner id on future requests.. And so, I will add in here a quick print that we are at the scanner data end. Then I can add self.cancelScannerSubscription and only include the request ID. After my cancellation, in my case I will make my typical call to self.disconnect().

If I run our code. I can see my usual error codes returned. Now I can also see each rank in my request, starting from rank 0 all the way to rank 49. This will only provide data with respect to the position and contract information.

This is a very basic list to get you started. Keep in mind that you can use market scanners with any other security, like options or futures, but you will need to be sure your Tag Values match up accordingly.

In a future video, we will dive into these issues more in depth to see how we can request market data from the market scanner. This concludes our video on Market Scanners in the TWS API. Thank you for watching, and we look forward to having you join us for more TWS Python API lessons.

Market Scanners – API User Guide & Examples

Scanner Parameters

from ibapi.client import *
from ibapi.wrapper import *

port = 7497

class TestApp(EClient, EWrapper):
    def __init__(self):
        EClient.__init__(self, self)

    def nextValidId(self, orderId: int):
        self.reqScannerParameters()

    def scannerParameters(self, xml):
        dir = "C:\\IBKR\\TWS API\\samples\\Python\\Testbed\\Traders Academy\\scanner.xml"
        open(dir, 'w').write(xml)
        print("Scanner parameters received!")



app = TestApp()
app.connect("127.0.0.1", port, 1001)
app.run()

Scanner Subscription

from ibapi.client import *
from ibapi.wrapper import *
from ibapi.tag_value import *

port = 7497

class TestApp(EClient, EWrapper):
    def __init__(self):
        EClient.__init__(self, self)

    def nextValidId(self, orderId: int):
        sub = ScannerSubscription()
        sub.instrument = "STK"
        sub.locationCode = "STK.US.MAJOR"
        sub.scanCode = "TOP_OPEN_PERC_GAIN"

        scan_options = []
        filter_options = [
            TagValue("volumeAbove","10000"),
            TagValue("marketCapBelow1e6", "1000"),
            TagValue("priceAbove", '1')
        ]

        self.reqScannerSubscription(orderId, sub, scan_options, filter_options)

    def scannerData(self, reqId, rank, contractDetails, distance, benchmark, projection, legsStr):
        print(f"scannerData. reqId: {reqId}, rank: {rank}, contractDetails: {contractDetails}, distance: {distance}, benchmark: {benchmark}, projection: {projection}, legsStr: {legsStr}.")

    def scannerDataEnd(self, reqId):
        print("ScannerDataEnd!")
        self.cancelScannerSubscription(reqId)
        self.disconnect()


app = TestApp()
app.connect("127.0.0.1", port, 1001)
app.run()

Join The Conversation

If you have a general question, it may already be covered in our FAQs. If you have an account-specific question or concern, please reach out to Client Services.

17 thoughts on “TWS Python API Market Parameters and Scanners”

  • istvan

    Hello,

    I am using the Scanner Subscription example code, but receiving the below error:
    ERROR -1 2104 Market data farm connection is OK:hfarm
    ERROR -1 2106 HMDS data farm connection is OK:hkhmds
    ERROR -1 2106 HMDS data farm connection is OK:ushmds
    ERROR -1 2157 Sec-def data farm connection is broken:secdefhk
    ERROR 1 165 Historical Market Data Service query message:no items retrieved
    ScannerDataEnd!

    Could you please help what could be the root cause of problem and how to fix it?

    I’m using the latest IBKR GW with the latest API version on Mac.

    • Interactive Brokers

      Hello, thank you for reaching out. Reviewing your error message, we can see the message “ERROR 1 165 Historical Market Data Service query message:no items retrieved” which is indicative that no symbols correspond to your request. Reviewing the code in further detail, it appears that the scanCode “TOP_OPEN_PERC_GAIN” is not valid before regular trading hours. One way to resolve this would be changing your scanCode to “TOP_PERC_GAIN” instead. Otherwise, you could test the request again during regular trading hours to confirm this behavior. If you have any additional questions or concerns, please feel free to contact Client Services: https://www.interactivebrokers.com/sso/resolver?action=NEW_TICKET

      Our API Experts would be happy to guide you!

  • PB

    Hi, the results give us a list for only 50 tickers, is there any way we can get more if the criteria is met by more than 50 tickers?

  • Charles Brauer

    Please help with obtaining Earnings Date from Wall Street Horizon Api. I have a subscription to WSH, but I’m having trouble with my Python script. Here is what I have tried:
    import threading
    import time
    import xml.etree.ElementTree as ET
    from ibapi.client import EClient
    from ibapi.wrapper import EWrapper
    from ibapi.contract import Contract
    from ibapi.common import TickerId

    class TWS_API(EWrapper, EClient):
    def __init__(self):
    EClient.__init__(self, self)
    self.earnings_data = None

    def error(self, reqId: TickerId, errorCode: int, errorString: str):
    print(f”Error: reqId: {reqId}, errorCode: {errorCode}, errorString: {errorString}”)

    def wshMetaData(self, reqId: int, dataJson: str):
    super().wshMetaData(reqId, dataJson)
    print(“WshMetaData.”, “ReqId:”, reqId, “Data JSON:”, dataJson)

    def run_loop():
    app.run()

    def main():
    app.connect(“127.0.0.1”, 7496, clientId=7)

    contract = Contract()
    contract.symbol = “AAPL”
    contract.secType = “STK”
    contract.exchange = “SMART”
    contract.currency = “USD”

    app.reqWshMetaData(1100);

    event.wait(timeout=3)

    if app.earnings_data:
    xml_data = app.earnings_data
    root = ET.fromstring(xml_data)

    earnings_date = None
    time_of_day = None

    for item in root.iter(“Ratio”):
    if item.get(“FieldName”) == “Earnings_Date”:
    earnings_date = item.get(“Value”)
    elif item.get(“FieldName”) == “Time_of_Day”:
    time_of_day = item.get(“Value”)

    if earnings_date and time_of_day:
    print(f”Earnings Date: {earnings_date}”)
    print(f”Time of Day: {time_of_day}”)
    else:
    print(“Earnings date information not found.”)
    else:
    print(“Failed to retrieve earnings data.”)

    app.disconnect()

    if __name__ == “__main__”:
    app = TWS_API()
    event = threading.Event()

    api_thread = threading.Thread(target=run_loop)
    api_thread.start()

    main_thread = threading.Thread(target=main)
    main_thread.start()

    api_thread.join()
    main_thread.join()

    I don’t get any xml data from this script.
    Please help.

    • Interactive Brokers

      Hello, thank you for reaching out. The wall street horizons event filters should be returned by reqWshMetaData. One of the returned filters should be “wshe_ed” which is the filter parameter for Wall Street Horizons Earnings Dates. Passing this as a filter into reqWshEventData will result in the earnings dates of AAPL in your instance. If you are in need of additional support, please create a web ticket for this inquiry; we have a category specifically for “API.” One of our API experts will be happy to guide you! https://www.interactivebrokers.com/sso/resolver?action=NEW_TICKET

  • Charles Brauer, cbrauer@cypresspoint.com

    Hello,

    The problem I’m trying to solve is how to obtain a stock’s next earning announcement date and time.

    I have subscribed to Wall Street Horizons data in my Interactive Brokers account.

    I obtained the Wall Street Horizon Event Types and Fields via Trader Workstation API document.

    This document describes specific event types. I am particularly interested in:

    wshe_ed Earnings Date
    ————– ———————-
    earnings_date Earnings date for the quarter/fiscal year
    time_of_day BEFORE MARKET, DURING MARKET, AFTER MARKET or UNSPECIFIED

    I have downloaded the XML for the contract details.

    The following section in the XML shows the information from Wall Street Horizons (WSH):

    WSH
    Upcoming Earnings (WSH)
    WSH_NEXT_EARNINGS
    STK,STOCK.NA,STOCK.EU,STOCK.ME,STOCK.HK
    false

    742
    Upcoming Earnings
    true
    m
    DATA

    0
    BA
    false
    DATA

    7
    Last

    742
    Upc Ern

    true
    2147483647
    500
    false
    unrestricted

    Here is what I have done so far with my Python script:

    from ibapi.client import *
    from ibapi.wrapper import *
    from ibapi.scanner import *
    from ibapi.tag_value import *

    class TestApp(EClient, EWrapper):
    def __init__(self):
    EClient.__init__(self, self)

    def nextValidId(self, orderId: int):
    # Define the scanner subscription parameters
    subscription = ScannerSubscription()
    subscription.instrument = ‘STK’
    subscription.locationCode = ‘WSH’
    subscription.scanCode = “wshe_ed”

    scan_options = []
    filter_options = [
    TagValue(“earnings_date”, 20240401),
    TagValue(“earnings_time”)
    ]

    self.reqScannerSubscription(orderId, subscription, scan_options, filter_options)

    def scannerData(self, reqId: int, rank: int, contractDetails):
    # Process the scanner data
    print(f”Received data for {contractDetails.contract.symbol}”)
    print(f”Earnings Date/Time: {contractDetails.contractDetails.nextEarningsDate}”)

    def scannerDataEnd(self, reqId: int):
    print(“Scanner data received.”)

    # Initialize and run the TestApp
    app = TestApp()
    app.connect(“127.0.0.1”, 7496, clientId=1001)
    app.run()

    When I run the script, I get:

    ERROR -1 2104 Market data farm connection is OK:usfarm.nj
    ERROR -1 2104 Market data farm connection is OK:hfarm
    ERROR -1 2104 Market data farm connection is OK:jfarm
    ERROR -1 2104 Market data farm connection is OK:usfuture
    ERROR -1 2104 Market data farm connection is OK:cashfarm
    ERROR -1 2104 Market data farm connection is OK:eufarmnj
    ERROR -1 2104 Market data farm connection is OK:usfarm
    ERROR -1 2106 HMDS data farm connection is OK:hkhmds
    ERROR -1 2106 HMDS data farm connection is OK:cashhmds
    ERROR -1 2106 HMDS data farm connection is OK:fundfarm
    ERROR -1 2106 HMDS data farm connection is OK:ushmds
    ERROR -1 2158 Sec-def data farm connection is OK:secdefil
    ERROR 1 162 Historical Market Data Service error message:Scanner type with code wshe_ed is disabled.

    What am I doing wrong, and I can I fix this Python script?

    Charles

    • Interactive Brokers

      Hello, thank you for asking. There are a few visible issues with the request provided. While you are welcome to use the WSH_NEXT_EARNINGS scanCode for Stocks, the locationCode of WSH is not valid. I would also note that the filters used in wall street horizons event requests are not applicable for market scanner requests. As a result, the filter values referring to the WSHE filters would also not be valid in these requests. A simple variant on this request would look something like this:

      sub = ScannerSubscription()

      sub.instrument = “STK”

      sub.locationCode = “STK.US.MAJOR”

      sub.scanCode = “WSH_NEXT_EARNINGS”

      scan_options = []

      filter_options = []

      If you are in need of additional support, please create a web ticket for this inquiry; we have a category specifically for “API.” One of our API experts will be happy to guide you! https://www.interactivebrokers.com/sso/resolver?action=NEW_TICKET

  • KSK

    Hi,
    At the end of the video it says that there will be a future video that explains how to get the data from a scanner for further processing. Are there still plans to release such a video and, if so, is there an ETA for the release?

    Thank you.

  • Charles Brauer, cbrauer@cypresspoint.com

    Hi Guys,

    Thanks for the reply.
    On the Wall Street Horizons web site there is a sample Python script. It is:

    ib = IB()
    ib.connect(‘127.0.0.1’, 7496, clientId=1234)

    # Get the conId of an instrument (IBM in this case):
    ibm = Stock(‘IBM’, ‘SMART’, ‘USD’)
    ib.qualifyContracts(ibm)
    print(ibm.conId) # is 8314

    # Get the list of available filters and event types:
    meta = ib.getWshMetaData()
    print(meta)

    # For IBM (with conId=8314) query the:
    # – Earnings Dates (wshe_ed)
    # – Board of Directors meetings (wshe_bod)
    data = WshEventData(
    filter = ”'{
    “country”: “All”,
    “watchlist”: [“8314”],
    “limit_region”: 10,
    “limit”: 10,
    “wshe_ed”: “true”,
    “wshe_bod”: “true”
    }”’)
    events = ib.getWshEventData(data)
    print(events)

    This script doesn’t work for me. I get:

    8314
    Error 10276, reqId 4: News feed is not allowed.
    Error 10276, reqId 5: News feed is not allowed.
    Press any key to continue . . .

    And yes, I am subscribed to Wall Street Horizons in my Trader Workstation account.

    As you can see, I am struggling with getting a simple example of how to get an Earnings Date from Wall Street Horizons subscription.

    Do you guys have a simple Python script that works with today’s release of Python and the TWS Api?

    Any code examples will be greatly appreciated.

    Charles

    • Interactive Brokers

      Hello again. Please note that Interactive Brokers does not support IB_Insync. If you are still experiencing issues, please contact Client Services: http://spr.ly/IBKR_ClientServicesCampus

  • Anonymous

    When I run your example I get the following error:

    Error code 162. Historical Market Data Service error message:Scanner filter TagValue(tag is disabled.

    What does this error mean?

    • Interactive Brokers

      Hello, thank you for asking. This means that one of the filters they submitted through the Eclient.reqScanerSubsubscription() method’s “scannerSubscriptionOptions” or “scannerSubscriptionFilterOptions” parameters. If you opt to use a different filter, the issue should be resolved.
      If you are in need of additional support, please create a web ticket for this inquiry; we have a category specifically for “API.” One of our API experts will be happy to guide you! https://spr.ly/IBKR_ClientServicesForum

  • Anthony

    Is there a way to increase scanner frequency? I can only receive scanner data every 35 seconds, which is very slow for active trading. Thanks

  • rami

    Hello How can I filter an ETF and get only stocks thanks

    • Interactive Brokers

      Hello, thank you for reaching out. Following the same structure as our Stock example above, we would simply need to add the additional value to our filter_options list: TagValue(“stkTypes”,”exc:ETF”). The tag value listed will now force our list to exclude ETFs. That being said, numerous other Stock Type filters are listed in our scanner parameters xml file that I would encourage users to review. There are further options for inclusion or exclusion, as well as specifying ETFs, CEFs, and more. This can most easily be found by searching for the STKTYPE values within the scanner parameters file. We hope this helps answer your question!

Leave a Reply

Disclosure: Interactive Brokers

The analysis in this material is provided for information only and is not and should not be construed as an offer to sell or the solicitation of an offer to buy any security. To the extent that this material discusses general market activity, industry or sector trends or other broad-based economic or political conditions, it should not be construed as research or investment advice. To the extent that it includes references to specific securities, commodities, currencies, or other instruments, those references do not constitute a recommendation by IBKR to buy, sell or hold such investments. This material does not and is not intended to take into account the particular financial conditions, investment objectives or requirements of individual customers. Before acting on this material, you should consider whether it is suitable for your particular circumstances and, as necessary, seek professional advice.

The views and opinions expressed herein are those of the author and do not necessarily reflect the views of Interactive Brokers, its affiliates, or its employees.

Disclosure: API Examples Discussed

Throughout the lesson, please keep in mind that the examples discussed are purely for technical demonstration purposes, and do not constitute trading advice. Also, it is important to remember that placing trades in a paper account is recommended before any live trading.

Disclosure: Displaying Symbols on Video

Any stock, options or futures symbols displayed are for illustrative purposes only and are not intended to portray recommendations.

IBKR Campus Newsletters

This website uses cookies to collect usage information in order to offer a better browsing experience. By browsing this site or by clicking on the "ACCEPT COOKIES" button you accept our Cookie Policy.