Databento launches its free public time service for algorithmic trading

Discussion in 'Announcements' started by Databento, Oct 23, 2023.

  1. Databento

    Databento Sponsor

    I'm pleased to announce that Databento's public time service is now available.

    Getting started
    This is a free public time service that's available to everyone over internet and public cloud. Anyone can synchronize their clocks against our service by pointing their NTP client at ntp.databento.com.

    History
    We were inspired to provide this service because our feeds so fast that many customers were reporting negative latencies when differencing local timestamps against our outbound timestamps. We found this was mostly an artifact of time synchronization error between our clock source and the customers' in such situations. This made it hard for latency benchmarking, and called for a way for customers to get a more accurate time and higher precision time sync against Databento's internal clocks.

    Architecture - How it works
    Our time service is composed of a pool of stratum 1 NTP servers at the CyrusOne Aurora I and Equinix NY4 data centers, making it the world's first public NTP service that is explicitly housed in financial data centers and optimized specifically for systematic trading users.

    Since the margin of error in time synchronization is largely a function of distance and hops to the time source, Databento's time service helps users obtain more accurate timestamps than common time providers like Google and Cloudflare, whose time servers are usually situated in general-purpose data centers that are further from popular colocation and proximity hosting sites used by financial firms.

    Another important reason to use Databento's time service is the nature of its clock source. The time servers for our service are synchronized against the same PTP time source used for supplemental receive and outbound timestamps in our data, within 200 nanosecond tolerance:

    Code:
     MS Name/IP address         Stratum Poll Reach LastRx Last sample
        ===============================================================================
        #* PTP                           0   0   377     1     +9ns[  +11ns] +/-  159ns
    In turn, our PTP source is based on a pair of grandmaster clocks engineered for electronic trading and high-frequency trading firms, with <25 nanosecond root-mean-squared error against UTC. Other public NTP services do not provide any guarantees about their clock sources—even if you were to obtain a time sync to their NTP servers with sub-millisecond tolerance, you may still be off from UTC by tens of milliseconds.

    Who should use this?
    Databento's time service is ideal for users using proximity hosting services and public cloud providers like Azure, AWS, and GCP, and for customers whose servers are located in the Chicago and New York/New Jersey metro regions.

    While more accurate time synchronization practices exist, the next marginal improvement for these users will generally require dedicated hosting or cross-connects, and run as much as $1,075 per month as a managed service.

    About Databento
    Databento provides APIs that make it simpler and faster to get market data. To learn more about Databento, go to databento.com.
     
    Last edited: Oct 23, 2023
    DarkTemplar, cruisecontrol and Occam like this.
  2. Databento

    Databento Sponsor

    This is explained in the post. The NTP pools you've shared are not optimized for electronic trading.

    Here's an example comparing our time service to other popular NTP servers in Azure IL cloud—the sync is much more precise than Cloudflare's/Google's, which are in turn usually better than most of the ntp.org pools. ntp.org pools are just based on DNS load balancing to a set of volunteer servers with very few restrictions or standards.

    Code:
    MS Name/IP address         Stratum Poll Reach LastRx Last sample
    ===============================================================================
    ^- ntp-01.ny4.databento.com      1   6    37    48  -1601us[-1548us] +/-   13ms
    ^- ntp-02.ny4.databento.com      1   6    37    47  -1566us[-1512us] +/-   14ms
    ^* ntp-01.dc3.databento.com      1   6    37    47  -2045us[-1991us] +/- 1812us <--
    ^- time.cloudflare.com           3   6    37    47  +1624us[+1678us] +/-   11ms
    ^- time.cloudflare.com           3   6    37    48  +1800us[+1853us] +/-   11ms
    ^? time.cloudflare.com           0   6     0     -     +0ns[   +0ns] +/-    0ns
    ^? time.cloudflare.com           0   6     0     -     +0ns[   +0ns] +/-    0ns
    ^- time1.facebook.com            1   6    37    47   -600us[ -546us] +/-   48ms
    ^? time5.facebook.com            0   6     0     -     +0ns[   +0ns] +/-    0ns
    ^- time4.google.com              1   6    37    46  -2152us[-2098us] +/- 6008us
    ^- time1.google.com              1   6    37    47   +496us[ +549us] +/-   11ms
    ^- time3.google.com              1   6    37    48  -2313us[-2260us] +/- 6094us
    And yes, we're offering this for free in spite of the much higher precision and more sophisticated equipment than the typical NTP server.
     
    DarkTemplar likes this.
  3. You might be right for customers that are very close to your sever, that's why you mentioned that in your post.

    You conveniently chose data for a client that would be very close to your server, now provide an example for someone that is at the other side of the world, that would be much different. That different, in fact, that they would be much better using a server close to them instead of yours.

    Which points to the issue that all servers have -> latency.

    That's why there are different servers around the world so you can choose whichever works better for your needs.

    As I said, this is not news. Anyone can spin an NTP server and offer it to the public.
     
  4. Databento

    Databento Sponsor

    Azure's IL site is ideal to prove my point exactly because it's the nearest public cloud location to Chicago (<50 mi, <1.6 ms half RTT). Chicago is popular for proximity hosting if you're running an algo trading strategy, and as described in the announcement, that's what we optimized for. And Azure, being publicly accessible, makes it easy to replicate my results.

    Picking a location on the other side of the world, say Indonesia, wouldn't make sense for this demonstration because you wouldn't host or colocate your servers in Indonesia to trade CME.

    Yes, I agree there's many free NTP servers online. Could you actually share one that has better accuracy than ours within 50 miles of Secaucus NY4-7 and Aurora I? The us.pool.ntp.org pool (872 servers) you shared actually yields much worse than the Cloudflare/Google time servers that I picked for comparison:

    Code:
    MS Name/IP address         Stratum Poll Reach LastRx Last sample
    ===============================================================================
    ^* ntp-01.dc3.databento.com      1   6    17    29    +70us[ +104us] +/- 1683us
    ^- h134-215-114-62.mdtnwi.b>     2   6    17    28   +494us[ +494us] +/-   15ms
    ^- time.cloudflare.com           3   6    17    28   +261us[ +261us] +/-   11ms
    ^- lithium.constant.com          2   6    17    28   -216us[ -216us] +/-   42ms
    ^- t1.time.gq1.yahoo.com         2   6    17    28   -241us[ -241us] +/-   26ms
    NTP servers are not born equal. It's actually expensive and time-consuming to get it to the level of accuracy we've achieved here. Facebook's engineering team has a good write-up on this here.

    That said, it's okay if you still prefer using another NTP service after my explanations. (I recommend Cloudflare if not us, as they are consistently ahead of other providers at most locations we tested.) At the end of the day, we're offering this for free so people can decide for themselves.
     
    Last edited: Oct 24, 2023
    DarkTemplar likes this.
  5. This is fantastic! I've added you to my NTP cluster. Lowest ping of any server in my list currently.
     
    Databento likes this.
  6. newwurldmn

    newwurldmn

    Why do I need you to tell me the time when I have a windows clock in the bottom right corner of my screen? I hope you aren't seriously charging money for this.
     
  7. 2rosy

    2rosy

    Does CME still provide a ntp ptp servers
     
    Databento likes this.
  8. Databento

    Databento Sponsor

    Thank you!

    This only has a discernible benefit to systematic trading users. To the naked eye, +/-20 ms and +/-2 ms tolerance is going to be impossible to tell apart. We recommend just using Cloudflare or your OS default time servers (e.g. time.windows.com, time.apple.com) for your home desktop.

    It's completely free.

    We think that having an accurate way for users to measure the latency of our feeds is more valuable than whatever we can get out of commercializing this.

    Yes, they still do, but these are not publicly available, you need to be in CME's extranet. They don't offer a managed PTP service, but you can pay for GPS antenna access on the roof of the data center.
     
  9. newwurldmn

    newwurldmn

    I was kidding. While I don’t trade in short time frames, I think this is a good service.
     
    #10     Oct 25, 2023
    Databento likes this.