Deciding on a Backtesting and Trading Platform

Discussion in 'Trading Software' started by Gyles, Nov 5, 2007.

  1. Gyles

    Gyles

    A brief background to the earlier thread which was:

    Getting Started Developing and Trading System

    I was searching for a back-testing product and have spent the past month researching it. I have done four evaluations so far and have mentioned them in the below posts.

    Since, I thought that these reviews (evaluations) were important and offered some value to the Elite Traders so I have moved them over here; into their own thread. I am sure a lot of people are also going though the same decision process I am.

    Please do note that I still have TradersStudio left to do and shall be testing that soon.
     
    SupermanTrades likes this.
  2. lindq

    lindq

    You should also look into Investor RT at www.linnsoft.com.

    There are many IRT users here on ET.
     
  3. Gyles

    Gyles

    THE FIRST REVIEW

    A Review of TradeStation

    This product has been a standard in the Trading market for a very long time and has two versions available at the moment.

    The first version “2000i” was released in 2000 (hence the name) and is available only outside the US. At the moment, it seems to be past its prime usage, yet its charting is worthwhile. Moreover, its own global server allows you to save data collected on your own hard disk.

    Moreover, I have gone through message boards, forums and the like as well as discussed with a few friends of mine who have the TraderStation 2000i. So far, I have a good review regarding this product.

    Being an “end of day trader”, so I do not pay much attention or rather do not care much over the real time data. Instead the trading portfolio’s of commodities and stocks is what matters to me and is not provided by the TradeStation 2000i.

    Although eight year later on, I felt that its new version, the new online 8.3 one might be able to provide the above. However, that was not the case.

    There were some points that were surprisingly. You can not apply money management rules on a portfolio of systems and markets; nor can you develop, optimize and test a system on a portfolio. It was surprising to note that TradeStation does everything: one market, system and set of parameters at a time!

    Let us forget the fact that I was just testing a bunch of Trading software systems. The impression I get on using TradeStation is that they seem to concentrate their efforts in getting more brokerage. They do not really wish to make advancements in the software as per the requirements of the dynamic face of the markets (as mentioned in the ET forums). Although, they do provide add-ons which can provide the above, they are as expensive as the other software on our list taken for evaluation.

    Anyway, I do admit it has plus points, among them is the availability of all the existing “EasyLanguage” code available.

    As a conclusion, if you as a trader is not trading real time over a short term, the on-line version is not worth the purchase and you are better off using the earlier version, the “TraderStation 2000i”, provided you find it. This is the real problem, as getting one is tough. You can lay your hands on it only if you live outside US or if you find someone willing to sell existing copies of the same software.

    Moreover, on searching the forums of the ET, had come across the following links stating the problems of the TradeStation, please comment.

    - Tradestation Forex down, been 2 hours now...
    - I'm sick of trade station
    - What are the downsides to TradeStation?
    - TradeStation data problems...
    - TradeStation Servers?
     
    waverider7 likes this.
  4. Gyles

    Gyles

    THE SECOND REVIEW

    Metastock Analysis , Product Comparison

    It is not easy to write complex programming like in TradeStation EasyLanguage, although OLE to excel is a plus. Moreover, issues like portfolio analysis and money management are missing. There are also problems with large optimizations

    Question is why are the two most established company products not meeting the requested needs of the customers?


    MetaStock seems to have been a good product before being taken over by Reuters where it has gone down in terms of quality, performance and apathetic customer service. The Reuters data feed is less than accurate and their support is indifferent at Reuters Data.

    Backtester in Metastock is a real issue. Even, the support staff is not aware when the problems would be solved. Moreover, it is consumes a lot of resources, and you shall need a fast computer for using it in real-time with several charts open. Tradestation is a better choice. Moreover, many recommend other programs for serious trading and not MetaStock.

    MetaStock’s strategy programming interface is also limited. It has an issue regarding the optimization. The database files are not cleared (made empty) after each optimization, which is troubling as whatever was saved earlier, a small optimization can cause a crash in MetaStock. Thus, one need to re-install MetaStock and all the historical testing results saved earlier is lost. Not a good thing.

    EOD (older version, not the real-time version) is at least better. However, MetaStock has got good graphics and drawing capabilities.

    MetaStock Pro- If you draw your trading numbers from chart formations and common indicators and use custom formulae, then MetaStock Pro has the capability to do so. Another good point is that, you can archive data on your hard drive (vs. running live charts from the server of your data source), as MetaStock Pro does not consume that much computer resources.

    System Tester in MetaStock has problems and the response to this issue from the support is poor. Although Yahoo Boards have suggested ways to solve the bugs, the ways are not all that easy to use.

    It is a general feeling that although MetaStock may have improved, they feel other products like TraderStation 2000i are a better bet.

    Moreover, TradeStation though popular, does lack some important issues; for instance, the historical testing like portfolio testing or money management is not there. Moreover, you do not have much access except to the TradeStation historical data and require paying a monthly fee in order to be able to run your system on this web-based application.
     
    waverider7 likes this.
  5. Gyles

    Gyles

    THE THIRD REVIEW

    Review of Trading Blox

    I had downloaded and tried out the demo version of Trading Blox. Firstly, it is a top flight back-testing package wherein you can either test a single system on one market or a portfolio of markets. Moreover, you can test a portfolio of systems also. The graphic programming interface “Bloxs” is a useful concept for the programmers.

    However, there was a drawback (a big one), is that there is not any large libraries of public domain code for this product like you see with “Wealth Labs” had earlier before Fedility purchased the product or TradeStation. The backtesting capabilities and control over the backtesting is well designed and though out. These features are top shelf. One drawback is that the product is somewhat slow, and the speed of both the demo and the real-time version is the same (as confirmed by Trading Blox support staff).

    Another issue is that you need to run the orders for tomorrow and back-test the result separately. This is for your kind information only as you might forget to do it.

    Regarding their reports, they are one big HTML like page! Although, you can disable and enable what you wish to see in the reports, it is a big page and to scroll through it is inconvenient. Moreover, they are not organized in a workbook or in different windows, which makes the navigation and reading inconvenient and time-consuming.

    Moreover, their charting does not seem to be up to the standard and seems to be behind the present times and requirements. It is there to just show you the trading arrows. Although, you can plot indicators but they need to be coded within the system. The same is true for some of the other products selected for the test of suitable back-test software. Moreover, I have found out that only the TradeStation and TradersStudio have separate indicators which can be added to the charts.

    Moreover, the charts have no drawing tools, special features like the Paintbars, Showme and expert commentary present in TradeStation. Another thing was that I was unable to figure as to how to create the charts and indicators without having a system. It is in my humble view, that for a product selling at $3000.00, they should have at least had the charting like the TradeStation, even if not that good. Their charting does not seem to match the good performance of the other functionalities provided by the Trading Blox.

    If you are interested in just back-testing, this product is good choice, a top flight product, although it can not replace TradeStation or other platform that provides good charts. This product is good to have, but not a one-stop product as there are some issues as mentioned above. However, if you can afford to spend $3000.00 for just a back-testing tool, then it is okay.

    The question remains, “Are there any other products available that offer similar features and the complete solution like back-testing such as Trading Blox, but at a lower price?”

    Now, the next step is to test out the products “AmiBroker” and “TradersStudio” (which are at a much lower price) and compare their features to Trading Blox.
     
  6. Gyles

    Gyles

    THE FOURTH REVIEW

    Review of AmiBroker

    Firstly, I must say that it is a good product developed by a few people over the years. Actually, it looks like a single developer has developed it, as on researching, realized that the developer and the products’ company are from the same place.

    1. It could do most of the things offered by other products at a lower cost. However, it requires a lot more programming skill; especially if you wish to do anything more than just back-testing. Their language being unique to them thus, the learning curve is steep (tough to learn). Although, you could use C# as reference, but that requires programming in .NET.

      However, I agree that like most products; you just can not use any existing Easy Language code without totally rewriting the same. However, the language is not easy to use for solving complex problems.
    2. Though, the custom back-tester is a powerful concept but not for the faint-hearted programmers.
    3. Their charts are not as good as the ones of TradeStation or Metastock charts. They are just on par with Trading Blox Charts except for the built-in indicators and some drawing tools. Besides this, they are of the same general quality.
    4. It is the same case with their Reports which are not complete or encompassing as the Trading Blox reports’ capabilities.
    5. The interface does look a little disorganized like the features we need to add without making changes to the interface which would have made it clearer.
    6. The Support feature is a disappointing issue as there is no phone number to call for support or help. I do admit that I stay overseas, but this point might be of importance to some of you out there at Elite Trader Forum. Maybe, it is a small company so it is not possible for them to arrange live support.

      Moreover, their Yahoo Groups support response seems to be slow for only a few questions, were replied and had to seek the Elite Trader forums (please have a look at this Questions about Amibroker). Moreover, their replies are not very satisfactory, explanatory enough. Even their knowledgebase, is incomplete.
    7. Even, as an end-of-day trader, you need to own more than just AmiBroker to do your analysis. As an example, you shall need TradeStation and AmiBroker because of the charting just like that of Trading Blox. Thus, it is not a complete solution for your needs.
    8. It has been observed that the people already using this product basically use it due to the interest in the free data show. It seems to address the dream of the trading audience and is not for the actual real traders.
    9. It seems to have a number of issues regarding usage of data with the charting software (Open E Cry data feed) or using data from different sources simultaneously (IB and MetaStock), or plug-ins (opentick) among others.
    10. Some problems which I could not get resolved from both the Yahoo Groups and the Elite Trader Forum are the following:

      • Regarding the futures, the product is basically designed for the Stock Traders, but it can also trade futures. Although, I was unable to figure as to how to set the minimum move, and minimum move value to trade futures, and also where to set the margins.

      • I am unable to get a new High-New Low data into AmiBroker using AmiQuote or any other free source. The answer is No as far as Yahoo is concerned. Although, it seems that you can develop your own using the complete US-Stocks database (Hemscott). Hence, this is a problem of a data issue dealing with free data.

      • Regarding its Trading Features, it has been suggested to use the help of coders to automate the system up to IB. However, it is not recommended as the coding has not been satisfactory or successful.
    11. It does not seem to be a trading program software for serious trading; as have noticed that many people try it out for back-testing purposes, to practice or refine their skills for handling and programming these programs.
      [/list=1]

      However, a point to note that if it was priced at $1000.00, I would not have gone for Trading Blox which is priced at $3000.00. However, it is the low price just $249.00 that makes it a worth-while purchase.

      It is an inexpensive product for people who wish to learn about back-testing and trading. However, it is tough for the same people to reach a more advanced level as to program the more complex things in the language of AmiBroker is real tough and complex. Moreover, the charts offered not as good as the usual charts they where using, so they still require the old charting program along with AMIBroker

      Thus, as a conclusion, its price is attractive enough to get people to start trading with it but it restricts the progress.
     
  7. Gyles

    Gyles

    Please be informed that the following quote is the Personal Message (PM) that arrived in my inbox. Moreover, am posting the same in exactly the same format and wordings, without any changes. Also, please note that I have taken the required permission from the concerned person and hence am posting the same.

    The PM was regarding the above review on AmiBroker. Also, please note that the following PM has been posted for your own judgement and comments. It does not reflect my views, or judgements or comments in any manner.

    Once more, please be informed that the above reflects Tomaszs' viewpoint and is for your kind information only.
     
    waverider7 likes this.
  8. sdeew

    sdeew

    Just a short note to let you know that the new BackTrader module has made an
    overwhelming difference in my trading style utilizing delta neutral
    strategies. The ability to run back through time to test various "profit
    taking" techniques could not have been done without this new capability. I
    also have found numerous ways to test volatility trades by combining
    BackTrader with the use of the the OptionVue volatility charts. It has
    convinced me by "actual proof" that when volatility is at it lowest, you
    can create a great straddle.

    Thanks for your creativity.
    Lou M at Excel Equity

    http://www.optionvue.com/overvue_back_trader.aspx
     
  9. DonKee

    DonKee

  10. Hello Gyles,
    It is hard to know a program when the trial period is only a week, so I’ll add some information about Trading Blox Builder (TBB) from my experience with the software.

    Let’s start off with speed because that area seems to be overly important to people and it should be cleaned up first.
    After seeing this comment I decided to create a simple speed comparison test with my other Trading and Testing platform, TradeStation (TS). In my case I have all the versions of TradeStation going back to the original release in June of 1991 up to the current 8.3.xx version recently released. My version of Trading Blox Builder (TBB) is version 2.2.5 and that is the version I’ll use for the testing, although all of versions in the 2.2.x series run at the same speed.

    Speed Test:
    Apply one of my long-term dual time frame (daily & Weekly data) systems to a portfolio of 9 markets of CSI data files for a time range starting in 1-Jan-1995 to last night. This system generates the exact same trades in both platforms when the trade sizing, and other trade restricting filters are disabled in TBB.

    In TS it needed 17-seconds, and in TBB this back test needed 7-seconds. This result wasn’t different than what I had found previously, but this was the first time I made the effort to get a time estimate documented.

    Orders for Tomorrow:
    This is actually an advantage in TBB. For example, to get the signals from a worksheet in TS you are required to open the same worksheet used in back testing so the open signals and orders can be viewed. Time required: 17-seconds. In TBB, you just hit F7, and the order page appears in 4-Seconds (both times shown here are using the system defined above).

    As to this approach being different than what I am accustomed to in TS, it was a change for me as well, but I like the speed in which it generates the orders, and the Hot Key makes it simple.

    Almost no Public Domain Code:
    This is true right now, and it will be a while before the scope of available indicators and functions available in TBB gets close to what TS provides. TradeStation has been around since 1991, but TBB has only been around a few years. However, this isn’t a big negative for those of us who create our own trading systems. It also isn’t much work to add just about any of the published indicators that are often included in many charting focused platforms. For example, if the user gets Perry Kaufman’s “New Trading Systems & Methods 4th Edition” book. Perry provides a use description and the code in text and TS file format on the included CD. With Trading Blox Basic being very close to Microsoft Basic, Power Basic and Easy Language, the places where some code words are different isn’t much of a stretch to find the replacement word, so getting the code in TBB to create an indicator just takes a little effort and patience right now.

    Still, the user needs to have some inclination to want to create a system or indicator to make this approach work. However, for those of us who’ve been trading for a while and have learned that most of what is provided in most packages isn’t useful, we don’t miss the clutter.

    For certain a lot of extra stuff makes the software look very capable and it can save time if the available code is useful, but without some history on how to apply or why that tool might be useful, any use becomes guesswork and that isn’t a good way to risk money. My recommendation is for people to spend the time research anything they want to use before they make any plans to see if it works. If they follow that approach chances are they’ll find code that is close enough to mold into what they want and should be able to get it running in most any back testing program.

    A neat feature about writing code in TBB is its ability to allow the user to take the output from a calculation or value assignment generated in the system code, and then show that value on the chart. This is really easy way to show Stop Exit location lines or signal entry location without having to jump through hoops with duplicate code and global DLL arrays to get that information into a displayed chart.

    I have more comments to add about Trading Blox Builder that I can post later, but for now it is time to get ready for next week’s signals.
     
    #10     Nov 10, 2007