Fully automated futures trading

Discussion in 'Journals' started by globalarbtrader, Feb 11, 2015.

  1. Todays trades

             code contractid     filled_datetime  filledtrade  filledprice
    2765  AUSSTIR     201606 2015-02-13 08:07:32            1        97.86
    2766    SP500     201503 2015-02-13 14:18:10            1      2087.50
    Slippage in GBP, for entire trade
             code  gbpt_slippage_process  gbpt_slippage_bidask  gbpt_slippage_execution  gbpt_slippage_all_trading  gbpt_slippage_total
    2766    SP500                 -16.36                  4.09                     0.00                       4.09               -12.27
    2765  AUSSTIR                    NaN                 12.19                   -24.37                     -12.19                  NaN
    The reason there are is no slippage for processing is because I'm rolling into the next contract, for which I don't yet have a sample price (until I've completely rolled out I calculate my signals from the old contract).

    Might be a good time to talk to you about rolling. This is the kind of report I get 3 times a day.

           code                              msg  near_expiry_days  position price_contract  relative_volume rollstatus                 suggest
    25      V2X           Roll positions ongoing                 4        -1         201503         0.974434      ALLOW                CONTINUE
    28      VIX           Roll positions ongoing                 4        -1         201503         0.218656      ALLOW                CONTINUE
    40   GAS_US           Roll positions ongoing                11        -1         201504         0.282547      ALLOW                CONTINUE
    41  AUSSTIR           Roll positions ongoing                16         1         201603         0.672121      ALLOW                CONTINUE
    42  EDOLLAR           Roll positions ongoing                 3         2         201806         1.008589      ALLOW                CONTINUE
    20      AEX  Roll near on price, not liquid.                 6         1         201502         0.003839        NOT  START ROLL...NOTLIQUID
    21      CAC  Roll near on price, not liquid.                 6         2         201502         0.011452        NOT  START ROLL...NOTLIQUID
    msg: Current rolling status. Roll positions ongoing means I will do closing trades in the near contract, and opening trades in the far one (a natural one). If that doesn't work out and I still have positions when the time comes I will do a forced roll

    near_expiry_days: Number of days to when I'd ideally get out. The ideal contract depends on the market. So for example for STIR to avoid the very low volatility at the front end I like to stay about 3.5 years out for Eurodollar and about a year for Aussie STIR.


    relative_volume: 1.0 means the next and current contract have the same price

    suggest: What the report thinks I should do. For the first 5 instruments, nothing. For the last two, nothing yet because there is no liquidity yet (and there won't be until Wednesday or Thursday).

    PROFIT: £2295

    It's quite dull reading this thread this isn't it? Automated trading is dull, if done correctly. Tens of thousands of lines of code to make sure things stay as boring as possible.
    #21     Feb 13, 2015
  2. Got a question: Based on the picture you posted on your first post, how much was the beginning Account Balance? Was it 500k?
    #22     Feb 14, 2015
  3. No, 300K.
    #23     Feb 14, 2015
  4. OK makes sense: Another question. You use many different instruments to trade. What is the advantage for you to trade many instruments instead of focusing on just one single instrument?
    #24     Feb 14, 2015
  5. Hello globalarbtrader, first of all thanks for your reply concerning the order placement approach.
    After reading your post about roll trades I have some passages that I did not understand.

    So as far as I understood the rolling dates for you contracts are not fixed but they depend on the relative values of prices and volume for front and back month.

    However I did not understand clearly this passage:

    What do you mean exactly by saying "I like to stay about 3.5 years out for Eurodollar".

    Furthermore, for a correct calculation of the roll dates, information about exchange holidays should be known. Is it there a clean way you might suggest to get the exchange holiday calendar. I know that they are available on the exchange website, but that would require scrapping the values.

    Thank you again for your posts, they are really helpful for a beginner like me.
    #25     Feb 15, 2015
  6. Diversification. The back-tested Sharpe Ratio on a single instrument probably averages about 0.5. On a system of 45 instruments, if they are properly diversified, the back-test comes out closer to 2.0.

    "I like to stay about 3.5 years out for Eurodollar".

    What this means right now I'd like to be trading the September 2018 contract, i.e. the 14th quarterly contract in the future (ignoring the monthly serials in the first year). In a years time I'd like to be trading September 2019. The nearer contracts don't have enough volatility. The later ones don't have enough liquidity.

    IB provides a contact object which includes the expiry date of the contract. The caveat is that in some contracts, like bond futures, you actually want to be out by the first notice date which is a couple of weeks before. I use this expiry calendar https://www.interactivebrokers.com/calendar/2015/

    I don't see why I need the full exchange holiday calendar to calculate roll dates - can you explain why you think this is required?
    #26     Feb 16, 2015
    panganp likes this.
  7. I thought that the holiday calendar should be needed in order to be able to know how many "active trading days" are left before the expiration date or first notice date.
    #27     Feb 16, 2015
    garachen likes this.
  8. I guess I would need that if the decision to go from 'passively rolling' (trade contracts such that we're increasing in the new contract, and reducing in the old one) to 'force rolling' was automatic. But it's not, it's manual. The decision is quite complicated, market specific, and involves a lot of knowledge that the system doesn't have, so it's not worth coding it up for the few minutes a month it would save me.
    #28     Feb 16, 2015
  9. Trades from the last couple of days:

    Trades take 1
            code contractid     filled_datetime  filledtrade  filledprice
    2770  GAS_US     201504 2015-02-16 16:47:23            1        2.886
    2771  AUSSTIR     201606 2015-02-17 02:37:15            1        97.85
    2774     BUND     201503 2015-02-17 08:41:21            1       159.16
    2772      V2X     201503 2015-02-17 08:37:06            1        23.90
    2773      V2X     201504 2015-02-17 08:37:06           -1        23.25
            code  gbpt_slippage_process  gbpt_slippage_bidask  gbpt_slippage_execution  gbpt_slippage_all_trading  gbpt_slippage_total
    2770  GAS_US                 111.25                  6.54                    -6.54                          0               111.25
    2774     BUND                 -66.64                  3.70                      7.4                      11.11               -55.53
    2772      V2X                  53.68                  1.85                      0.0                       1.85                55.53
    2771  AUSSTIR                    NaN                  6.08                      0.0                       6.08                  NaN
    2773      V2X                    NaN                 -1.85                      3.7                       1.85                  NaN
    You can see that I did a spread trade to roll V2X, which I managed to do at a good level.

    Yesterdays LOSS: £192
    Todays LOSS £3,511
    #29     Feb 17, 2015
  10. I've been away for a few days, but naturally my system has continued to trade. I'm confident enough in the checks and balances I have in place to do this, even without email access. If I'm going away for more than a week then I might have to do some work in advance, for example to avoid missing a roll. If my risk looks quite high I might even prevent the system from opening new positions whilst I am away.

    First P&L. Three good days. I should go on holiday more often.

    18th Feb: £6,214
    19th Feb: £5,592
    20th Feb: £816

    My expected risk is £6,277 per day, so none of these are large days. One more day like Wednesday or Thursday and I will be up to my high water mark.

    Here are my trades:

    2783      AEX     201503 2015-02-18 10:42:32            1       466.65
    2784      AEX     201503 2015-02-18 10:47:19            1       466.75
    2785      AEX     201502 2015-02-18 10:51:47           -1       466.35
    2777      ASX     201503 2015-02-18 02:31:48            1      5857.00
    2778      CAC     201502 2015-02-18 08:01:38           -2      4780.00
    2779      CAC     201503 2015-02-18 08:01:38            2      4781.00
    2786  LEANHOG     201506 2015-02-18 14:34:07           -1        78.05
    2780      VIX     201503 2015-02-18 10:04:53            1        18.20
    2782      VIX     201504 2015-02-18 12:55:34           -1        18.80
    2787  AUSSTIR     201606 2015-02-19 02:00:30            1        97.91
    2788  AUSSTIR     201606 2015-02-20 05:17:01            1        97.91
    2789      CAC     201503 2015-02-20 11:30:30            1      4825.00
    You can see there are quite a few roll trades there.

    And the slippage:

    2778      CAC                -547.95                  3.70                   -14.81                     -11.11              -559.06
    2786  LEANHOG                 -19.51                 13.01                    -0.00                      13.01                -6.50
    2780      VIX                   0.00                 16.26                     0.00                      16.26                16.26
    2784      AEX                    NaN                  7.40                    0.00                      7.40                  NaN
    2785      AEX                  77.75                  3.70                    -0.00                       3.70                81.45
    2777      ASX                -259.69                  6.33                   430.70                     437.04               177.35
    2779      CAC                    NaN                 -3.70                    14.81                      11.11                  NaN
    2782      VIX                    NaN                 16.26                   -32.52                     -16.26                  NaN
    2783      AEX                    NaN                  7.40                   -14.81                      -7.40                  NaN
    2787  AUSSTIR                    NaN                  6.12                        0                       6.12                  NaN
    2789      CAC                  -5.54                  1.85                    -3.69                      -1.85                -7.39
    2788  AUSSTIR                    NaN                  6.11                   -12.21                      -6.11                  NaN
    Sticking out like a sore thumb is the ASX trade, which merits further investigation. A sample mid price of 5843 was taken at 02:30:30, UK time. 70 seconds later after the signal was processed and when we went to the market to trade the offer was 5823, and the bid was 5822. We joined the bid at 5822, but a few seconds later the price updated to 5856 - 5857. The algo immediately cut its losses and took the offer, with the fill coming in 8 seconds after we submitted the trade at 02:31:48.

    Looking closely at the bars the price at the time of the order of 5822 - 5823 was incorrect, and at least a couple of hours old. So the fill was a good one, and the slippage much smaller than the figure above suggests.

    It's worth having a discussion about how one should spot bad prices, and what one should do with them. The basic ways to spot bad prices (apart from obvious SNAFU's like zeros or Nan's) are a large change from a previous price, a large bid-offer suggesting an illiquid market, or delays on timestamps. Its easy to exclude stale prices and to set a maximum bid-offer at which you'd accept a price.

    Its a bit harder to know where to set the threshold for a large change. After all we don't want to be prompted every time the price moves a 'normal' amount. Even if you set the trigger where it goes off once a month, given I trade 45 markets, that implies I'd get about two prompts every trading day, which is too many.

    The 30 odd tick movement above wasn't big enough to trigger any concern, and no action was taken.

    What should we do with a bad price? One option is to live with it, as I did here. If you have enough robustness in your system then that isn't a problem. If you're trading slowly enough then the impact of the last tick on your signal should be minimal. In simple terms if you're using say a 50 day moving average then a bad price will only affect 2%, as long as the price isn't way off, say by a factor of 10. Of course if you're high frequency trading then a 30 tick error over 8 seconds could make a huge difference.

    We can warn on bad prices. This might make sense if a price moves more than expected, but not enough to justify exclusion. The price can then be checked at the end of the day, and if necessary changed post hoc. Note that this means that simulations will be slightly biased with forward looking information.

    We can exclude bad prices. If a price exceeds my threshold, I don't add it to the database, but my system emails me.

    2015-02-20 20:14:18 CRITICAL: dailyprocess : Sample WHEAT 201612 failed spike check price move from 584.750000 to 565.500000 was -10.71 vols (vol of 1.798195) *RERUN WITH MANUAL ENTRY*  (resolveprice)
    Most of these warnings like this one are on illiquid contracts which I'm not trading yet, that might not see trades for several days. I then have to rerun the price collection process manually, and confirm I'm happy with those prices.

    To give you some insight into what an automated system trader does with their time (apart from spend more time on holiday) a little project I have been working on over the last few days involves refactoring the storage of prices. The simple and dirty software I use doesn't optimise for things like using a small number of large files, or a large number of small files.

    The answer depends on how many processes you are running, wether they are in serial, whether they are likely to be locking files to read, and so on. I came to the conclusion that a large number of small files was better in this use case. This is good old fashioned computer engineering.

    Another thing I've done is to change the way my system diagnostic outputs are produced. This has created a smaller disk footprint allowing the system to be easily run on my less powerful backup machine, and sped up the process. Essentially I'll be using the backup machine to generate additional diagnostic information. This means enforcing backup and replicate, and . All this means it will be easier to switch between the live and backup machines, again something I've neglected. More engineering.

    It's also an opportunity to think about how diagnosis is done. I created quite a lot of over complicated diagnostic tools that I don't really use, which as a result have become deprecated as I've changed the system. The essential problem is to summarise a relatively complicated picture in such a way that issues can easily be spotted, and then to have drilldown tools allowing you to see where things went wrong.

    As part of this effort I want to create a suite of functions to analyse various aspects of live performance, with a deadline of April 1st when I'd like to be able to report on my first year of trading. Rather than the big over complicated generic solution, I'm going to try and build a large number of functions to answer some specific questions, but which will sit on some shared classes to get data out of the system.

    This then leads into simulation. Again my simulation tools are rusty from lack of use. I've now got a better understanding of what I want from them. There are three main areas of interest once I get the tools working again. The first is to be able to use intra-day data better. One is to adapt trading systems to the cost of trading markets (at the moment I tailor my system to the most expensive markets). I didn't want to do this until I had collected enough data on trading costs. The other is to revisit some promising mean reversion systems that I removed from the system as I didn't feel I had enough evidence that they worked.

    This is quite a modest research program, as I have many other things I am doing with my life, and endlessly polishing my trading system isn't something that interests me, nor do I think it is of much value.
    #30     Feb 23, 2015
    traderzbs, panganp and dreturns like this.