Confused about live market data providers and packages.

Discussion in 'Data Sets and Feeds' started by howdoyouturnthison, Mar 6, 2021.

  1. I have been exploring different data packages for a while and I learned that simply level 2 means top of the book (best bid/offer) for a specific exchange. In order to see other “worse” bids and offers for each exchange, one has to get a specific “full book” data package. Here is where things get unclear:

    I went over a dozen of data providers and very few offered full book for NASDAQ and NYSE. Usually, they only have NASDAQ OpenView and standard level 2 for everything else or not even that. Why is that so?

    For example, Das Trader’s “Premium Elite” package has the following market data: Totalview, Options Level 2, OTC Markets Level 2, ARCA book, IEX Deep, e-Mini Level 1, TSX Level 1, Imbalance, Fundamentals and Forex data.
    Notice that TotalView is not OpenView so I believe this doesn’t even include full book, just level 2 top of the book. On the other hand, ARCA book and IEX Deep are actual full books I believe.

    Another example is iqfeed.net. They provide NASDAQ Level II (OpenView) but there is no full book package for NYSE, NYSE American, Cboe, IEX…

    I read somewhere that orders placed on other exchanges are only included in NASDAQ OpenView if they are placed through NASDAQ exchange. I am unsure what that means. Depending on which ECN/MM/Exchange I select to send my order to, which live market data package will it show up in? Because stocks can trade on any exchange, would that mean that only if I send my order to NASDAQ it will show in their full book but not others, unless it is at NBBO and will be then visible in level 1 top of book?

    Does “Time and Sales” table that I get with any trading platform and basic data feed packages include trades that happen across all 3 tapes / SIP’s (NYSE, Regional, NASDAQ)? Or is that again dependent on how broad in scope my data subscription is?

    How about trades that happen “off-exchange” (TRF)? I read that they send their trade data to exchanges which then publish them. Does this publishing of trade on "time and sales" happens live with a small latency due to geographical distances or is it consolidated more sporadically over time? I guess they have to send in their trades because how else could we otherwise verify that they operate within the NBBO spread. But if this is not done live, it is useless for day traders. How about alternative trading systems (ATS) that we also call dark pools and are a subsection of TRF?

    [​IMG]

    If you know of any book that covers these technicals in-depth, I would appreciate a recommendation. I would hate to think of trading as a black box. I got most of this information from nasdaq.com articles and random notes scattered across the internet forums and pages. I ordered 1 book (Financial Markets and Institutions, Mishkin, Frederic) that I hope will help with some of the higher-level view about the market but I fear it won’t go into technical details of how the market is actually interconnected. If I made mistakes in this writeup, do point them out. Thank you for reading.
     
    Last edited: Mar 6, 2021
  2. qlai

    qlai

    This is a lot to digest. What kind of trading are you looking to do?
     
  3. Occam

    Occam

    Off-exchange TRF data does come via the SIP feeds.

    Nasdaq Level 2 technically means top-of-book for each NSDQ market maker. This has become far less relevant over time. Most traders seem to confuse Level 2 with full book, so you can take some solace in that you are already far more informed than most ;) But I know of no single source for this type of information, unfortunately. US equity market structure is comically over-complex and there seems to be no movement towards simplification -- in fact quite the opposite.

    To get the true full book for US equities, you need the full book product for every exchange, the number of which is growing. If you are a point-and-click trader, you may get this from some vendors; to get the full feed on an API basis, suppliers like Activ or QuantLab provide some approximation to this for a substantial price. Nanex offers the SIP's plus a few other things (top of book for each exchange plus Level 2) at a reasonable price. If you really want it it all, you need to get full feeds from every exchange and it is probably a low 8 figures budget situation.
     
    howdoyouturnthison and qlai like this.
  4. I am only interested in stocks so that narrows the data packages down a bit. This is what I currently got from IBKR:
    • Cboe BZX Depth
    • Cboe One Add-On Bundle
    • IEX Depth of Book
    • NASDAQ (Network C/UTP)
    • NASDAQ BX TotalView
    • NASDAQ TotalView-OpenView
    • NYSE (Network A/CTA) - Trader Workstation
    • NYSE American, BATS, ARCA, IEX, and Regional Exchanges (Network B)
    • NYSE ArcaBook
    • NYSE OpenBook
    • US Securities Snapshot and Futures Value Bundle

    Ignoring the level 1’s, these basically are full books for NASDAQ, NYSE, Cboe, IEX, Arca, CBOE…
    Because markets are closed, I can’t doublecheck in “Market Depth Trader” in TWS to see if I see more than top of the book for others, some of which I didn’t even know I would get and didn’t know about: BEX, BATS, MEMX, AMEX, BYX, EDGX, PSX, PHX, PEARL, ISE and a few more I didn't notice bids so far (a couple of days).

    All this for under 100$ seems like a great deal. Only downside is that the data comes in 250ms intervals and is aggregated but I don’t think this will affect me and if it will, it won’t be the critical reason for success or failure.

    There are two SIP operators running the three tapes:
    • SIAC operates all Tape A and B stocks, and is based out of NYSE’s data center in Mahwah, New Jersey
    • Nasdaq operates all Tape C stocks and is based in out of Nasdaq’s data center in Carteret, New Jersey
    Does that mean every data feed also gives me the combined SIP’s that form my “time and sales” feed? And you say this also includes off-exchange trades live feeds? That’s good to know that at least actual sales are visible if the liquidity is partially in the dark.

    I found a page that looks like it gives some insight into where the volume happens and which exchange/MM/ECN full book might be nice to have.

    [​IMG]

    One thing about it that is unclear to me is, if NASDAQ operates only tape C, how do I explain then the Tape A and B inputs for NASDAQ field in 2nd row?

    Even if I get every possible data package in existence, I still can’t see the book, much less full book of 41% volume on TRF. From a NYSE article, for cheap stocks that percentage goes even higher.
     
    Last edited: Mar 7, 2021
    honest-trader likes this.
  5. jharmon

    jharmon

    You're conflating a listing on Nasdaq with trading on Nasdaq.

    Trades on Nasdaq-LISTED securities are reported on tape C. Trades on NYSE-LISTED securities are report on Tape A. Trades on for listings on any other exchange (NYSE American, NYSE Arca, Cboe, IEX etc.) securities are shown on Tape B.

    For example, Nasdaq:TSLA is available for trading on NYSE Arca. Those trades are are reported on tape C only.
    NYSE:GE is avaiable for trading on Nasdaq. Those trades are are reported on tape A only.
     
  6. Since I started this thread, I gained a bit more insight into the data providers market and am currently in contact with several of them, trying to evaluate the pricing and data of each before eventually deciding on one as my main.

    Here is a list from less to more expensive (I assume, because I still need to receive confirmation on some price quotes):

    https://developer.tradier.com
    https://www.activetick.com
    https://polygon.io
    https://ginses.com
    https://www.iqfeed.net
    https://www.nxcoredata.com
    https://www.refinitiv.com

    Order may be different for you, depending which securities interest you. I also don't know yet how they compare data quality-wise. I don't believe any of those provide real time streaming of full book for all exchanges but I wouldn’t be surprised if that gets implemented in the future.

    Here is an interesting article I found linked in another section of this forum that explains well the structure of US stock market and participants: https://polygon.io/blog/understanding-the-sips/

    It further links to pages of both SIP's where we can check the documentation that describes it in great detail:
    https://utpplan.com/technical
    https://www.ctaplan.com/tech-specs

    Here is a good image I borrowed from article I linked above that made a nice visaulisation:
    [​IMG]
    Note: From what it says in the article, I would expect FINRA lines leading into UTP SIP icon. Not sure why it is a separate icon.


    Another thing that boggled my mind for a long time and was finally clarified is about odd lots (orders of fewer than 100 shares):

    The Participants of the Nasdaq Unlisted Trading Privileges (UTP) Plan and the Consolidated Quotation (CQ) Plan operate the SIPs that are responsible for disseminating the national best bid and national best offer (NBBO) for quotations in NMS stocks. Pursuant to Regulation NMS, the NBBO as disseminated by the SIPs is calculated using bids and offers to buy or sell one or more round lots of an NMS stock1. Round lots are defined by the exchanges and generally refer to quotes to buy or sell 100 shares of a given security or a larger number of shares divisible by 100. Odd lots, or orders for fewer than 100 shares, are not included in the NBBO and are not currently distributed by the SIPs.
     
    Last edited: Mar 17, 2021
    qlai likes this.
  7. jharmon

    jharmon

    The FINRA Trade Reporting Facillities do indeed report dark pool trades to both CTA and UTP depending upon where the security is listed.

    I don't trade any Pink Sheets/OTC stuff, but I don't think OTC Markets is a trading venue - it's only a quotation venue, just to confuse matters further.

    Polygon needs to fix their diagram.
     
    howdoyouturnthison likes this.