It’s nice to see some action on this Systems Thread. I read...

It’s nice to see some action on this Systems Thread. I read through all of the recent posts and started to write my own post more than a week ago but only have just now completed what I thought might be a good contribution to the discussion. I’ve been a bit tied up with EOFY and other tasks, hence the delay in my response.

@sci9249 – Sci I was wondering what you were up to. I haven’t heard from you for quite a while. So you’ve given up on ASX Stock Trading using VV and fundamentals and have moved on to trading futures, e-mini etc., using TA and automated Algo trading, now that is a big change! I’ll have to have a look at Nesnidal’s E-Mini Breakout course. I think its relative cheap isn’t it?

@journeyman how are you going with Amibroker and Premium Data? I did have a subscription to Premium Data for US Equities but have just recently swapped over to their new Norgate Data service. The database updates are a lot quicker with the new Norgate Data system. Given your background in coding I don’t think you’ll have any problems with Amibroker coding. My programming experience was with FORTRAN, Visual Basic, SQL and some of the RDBMS specific procedural languages provided by Oracle and Sybase/MS SQL. The trick with the charting package languages like Amibroker and Metastock is to start thinking in terms of Vector or Array processing rather than procedural loop etc. Just about all you need to do with coding indicators an setting up system tests can be achieved using these array functions. Procedural looping constructs are available in Amibroker, and if you want to move towards the more complicated and precise methods of backtesting you need to use the Amibroker Custom Backtest Procedure Interface which requires a basic knowledge of procedural and object oriented programming. All of the tests I describe in the following use the standard Amibroker Portfolio Backtesting functionality, i.e. no complicated looping or object oriented event and method calls were used.

Recently I’ve been working on finding/developing three separate systems, initially with each system trading Australian Stocks. The three systems have different trading objectives and different but also overlapping markets. The first system, and easiest system to develop, is an Investment Trading system. It uses the Weekly timeframe and the stocks within the XAO (ASX 500) are the market. This system is now at the live “paper trading” stage of development. The second system that I am now working on is a Daily timeframe and short holding time (1 to 5 days) DMA CFD trading system focussing on Australian stock CFDs that can be traded on margin of 30% or less, about 250 ASX stocks. The third system will focus on Low Cap Speculative stocks. I’ll probably be allocating around $100k initial capital to each system.

@Thanky and I were looking at a number of potential systems which included the Marwood Mean Reversion system, talked about in some of the earlier posts on this thread, along with other systems based on Bollinger Bands. The Marwood MR system was fully disclosed and on initial backtesting it returned a positive CARG of around 13%. I can remember going through the exercise of stress testing all of the system components and coming up with a reconfigured system that returned around 30% CARG. There were other issues with the system that included high variability of return from year to year, and there were issues with the trade profile (the number of trades held and opened and closed on a given day). This exercise was more of an Amibroker Portfolio Backtester learning exercise for me, and given that we had many more system ideas to test, I didn’t move forward with this system. The final, or should I say last, test results for this Marwood MR system analysis are shown below. The initial equity was $50k. The testing and results shown below are for the ISS (In Sample Segment) data date range only. The system wasn’t walked forward into the OSS (Out of Sample Segment)data range period.

Having rejected this 30% CARG system that could turn an initial $50k into $700k over 10 years, I moved on to testing a number of systems based on Bollinger Bands trade setups. Just about all of the common BB systems that are disclosed and are publically available failed to return any profit at all. The one exception to my surprise was “The Chartist” (Nick Radge) BB trading system. The rules of the system are available in Radge’s book “Unholy Grails”. It’s a very simple BB breakout and trend following system that uses very unusual system settings. I’ll leave it to the reader to confirm Radge’s BB system settings but if my memory is correct they were; Top BB 3 Stdev, Bottom BB 1 Stdev, and MA period 100. The basic signals are Buy on break above Top BB and sell on close below Bottom BB. I think that he used a general XJO moving average Market Filter as well. If you are interested in this system I think Radge sells the Amibroker code for the system for around $700. Why anyone would pay this amount for such a simple system I do not know. I ran hundreds of backtests on this simple system and came up with similar results to those published in his book, but also improved results with my modified system settings. The initial equity used in the BB system tests was $100k. This modified system was walked forward through the OSS data range. The final CARG was 30.83%. Over the total 12 year test period there were only 295 trades taken, an average of 2 per month, and the average holding period was 217 calendar days. For a daily timeframe trading system the trade profile figures suggested to me that I wouldn’t be very comfortable with the daily routine associate with managing such a system. The backtesting results are provided in the following.

So there we go again, another system with a CARG% greater than 30% and this time turning $100k into $2.78M in just 12 years. The drawdowns during the GFC 2008 and more recently during the market corrections at the end of 2016 and again at the end of 2018 can be easily see in the Underwater Equity plot and the Yearly Returns matrix. These large, for me anyway, drawdowns made me think twice as to whether I would be comfortable trading this system. I tried adding an XJO Market Filter, e.g. the XJO Close must be above its 50 day MA for a trade to be taken, but all this did was halve the overall return. The other consideration was that in order to achieve this type of compound return using a portfolio of a maximum of 20 stocks, the resulting trade size as the equity grew approached $140k per position. For the stocks being traded this is quite reasonable as Amibroker applies a minimum percentage of volume traded daily constraint, which I have set at 10%, but you still have to consider how comfortable you would be in placing $140k trades.

So I decided to move on and consider a Weekly Timeframe trading system using some of my favourite indicators. I simplified the setup rules as much as I could and started off by hoping that the Weekly Timeframe would filter out a lot of the noise in the daily price fluctuations of ASX stocks. As mentioned previously, this system has now been progressed to the “Live Paper Trading” stage of development. The final ISS and OSS combined backtest equity curve is shown below. The initial equity was $100k and the resulting CARG was 73.5%. The test period was from start of 2013 to end June 2019.

Once again when using a portfolio with a maximum number of positions constraint, and trade size based on a percentage of available equity,you end up with trade sizes approaching $350k in order to get the compound return shown on the graph. To simulate alternative trade size strategies that change as the equity grows would involve additional code and the use of the Amibroker Custom Backtester Procedure Interface (CBPI). Amibroker provides all of the basic Portfolio Backtest functionality in the default Backtester, but if you want to do anything unusual like manage position sizes dynamically based on something different to available equity and maximum number of positions then you have to write your own Custom Backtest Procedure. For all of the backtesting work I’ve performed so far I’ve use the default Amibroker Portfolio Backtester. However, I know that in order to realistically model and backtest my CFD Trading system I will have develop and use my own Custom Backtest Procedures.

After reading through the recent post on this thread I decided to add to the hundreds of backrests I had already performed on my Weekly Investment Trading system and test the system over the GFC period (2008). The resulting equity curve for that period is shown below.

A very pleasing result I think. The maximum drawdown at the end of 2008 was -16%. You can also see in the graph that I have incorporated an XJO Market filter in the systems, as can be seen from the areas on the graph where the equity curve goes flat. What I have found in the current “Live Paper Trading” exercise for this system is that as I am running the system on a weekly basis and looking at setups on the charts, I keep on thinking that small changes could improve the system, i.e. wanting to change things as I go along is only natural for me. I’ve now backtested about seven system adjustments that my week to week chart and trade observations suggested could improve the system. After backtesting over the original ISS date range all of these suggested system changes have been rejected. For me all this is just part of the process I need to work through in order to develop the trust in the system I need in order to allocate a significant amount of equity to live trading. Once the system goes live of course there won’t be any more system changes tested. The system will be traded as is or switched off.

All this work @journeyman was performed using the standard Amibroker functionality. It’s quite easy once you understand how the Amibroker Portfolio Backtester works. I don’t know how much you know about the standard vector (array) based coding functionality that Amibroker and other charting systems like Metastock provide that allows a coder to develop basic and complicated indicators etc., but once you are comfortable with this type of coding you should then focus on getting a thorough understanding of the Amibroker Portfolio Backtester. Only when and if you want to model more unusual and/or technical aspects of your trading systems in your backtests would I move on to more advanced coding using the Amibroker Custom Backtester Procedure Interface.

So that’s where I am up to with my system development efforts. I do have a few thoughts regarding the other issues bought up in the posts on this thread. I think there is one very significant reason why some people migrate towards trading futures and FX. Apart from the obvious advantage of very much higher liquidity, compared to Australian stocks that is, the main reason I think is that from a systems point of view it’s just so much easier to focus on or concentrate on one or only a few securities (markets). I can remember reading a comment somewhere that was along the lines that it was outright easy to develop a winning system for one Stock or one Security. I wasn’t all that sure as to whether this was true or not but after attempting to develop a system to trade a market that consists of a large number of securities I can now understand that Portfolio Backtesting and Portfolio Trading System development is exponentially more difficult than Individual Security backtesting and system development. Even though it would appear, to me anyway, that the backtesting and development work associate with a trading system for just one Security is relatively easy, I think the trade-off may be that these one Security specific systems may have a relatively short usable life.

Some time ago I did a System Development course called “Systems Development Masterclass”. The course presented a systems development method, or process, that is structured and quite comprehensive. A system for day-trading the 5 minute S&P 500 E-Mini based on a daily timeframe breakout setup is developed during the course, and the development platform is TradeStation. The system only had two degrees of freedom so stress testing using the optimiser wasn’t too difficult a task. When I attempted to apply this development process to a system focussing on trading a portfolio of stocks in the ASX XAO I started to understand very quickly why people always recommend that you keep your trade setups as simple as possible. One of my simple systems involves at least 5 degrees of freedom in the basic setup let alone the additional complications and variables that have to be included in order to model and backtest a portfolio trading system; i.e. Portfolio size, Stocks to include in the target market, and the method used to prioritized the many potential trades that come up in each daily scan.

I ended up following what I call a “Progressive Recursive System Development Process (PRSDP)” which now comprises of three distinct phases. It’s an iterative, recursive and somewhat intuitive process where the idea is to focus on what you discover and believe to be the critical components of the system. Phase One involves performing a number of “All-Trades” backtest where you get an idea of the basic raw performance and possible edge that the “Key Idea” provides. There are two ways you can do this in Amibroker. In Phase Two you introduce all of the practical constraints that apply to portfolio trading. In Phase Three you introduce the additional complications associated with modelling and testing various Order Types and more precise Position Sizing strategies. The critical thing is to staying organised during the process and it is also considered that rigorously documenting backtest setups and results at every point in the process is imperative. In Amibroker you can perform a multi-variable Optimizations over say 500 stocks and a ten year period that effectively involve running hundreds of individual backtests in a very short period of time. Some of my optimization runs involve up to 900 or so backtests with the run time being about 14 minutes or so. An individual backstest generally only takes a few seconds. It’s so quick that it’s very easy to get lost in the analysis very quickly. By “Optimisation” I mean “Stress Testing”. In Amibroker all the Optimiser does is perform a large number of backtests with each test varying the optimiser target variables until all the specified combinations have been tested. So if you were to run an optimisation involving two variables, e.g. Bollinger Band upper band setting and the lower band setting, with a range of say 0.5 to 3.5 and an increment of 0.1, Amibroker would run portfolio backtests for approximately 30*30 = 900 combinations of settings over the specified date range, say 10 years of data. This sort of optimization would take about 14 minutes or so on my desktop PC which has a 64 bit i7 CPU. Stress testing involves using the results of the optimization runs to select what you consider to be the most stable and best performing settings for the setup indicator variables. Not necessarily the settings that provide the highest CARG% for instance. So my PRSDP process involves performing lots of optimizations and re-optimizations and final backtests as the critical components of the system are determined and system settings are changed, and all the time trying to not get caught in the over-optimization trap by treating the optimization process as a stress testing exercise as described above.

So that’s all from me for the moment. I’d be interested in hearing more about your experiences with the BTC Portfolio of Systems approach Sci, and your efforts with Amibroker Portfolio Backtesting journeyman.