Announcement

Collapse
No announcement yet.

Has anyone been successful backtesting a system that is consistently profitable?

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Has anyone been successful backtesting a system that is consistently profitable?

    I've been programming efs strategies in esignal for about a year full time, and want to thank Alex and Brad for all their support and code examples, they have been invaluable to me.

    My problem is that in spite of all the time and effort trying to find something that tests reasonably well, ie., 50 percent profitability, with the winners being about 2:1 greater than the loosers, I've basically thrown in the towel. Occasionally when I find something that does well and I'm talking about 1-10 minute intraday data in cash currenciesor the S&P, that performs well for 30 days or so, when I run it for the earlier 90 days, it rarely does as well.

    I don't optimize my parameters, as curve fitting parameters to match the data is a problem I've had in the past and learned the hard way, that the fewer the parameter variables the better.

    I'm not asking for anyone to "give me" the fruits of their time and pain, but would like to know if anyone has come across a system for the S&P, currencies, using 1-10 minute data intervals that consistently performs well?

    If anyone has, I would appreciate you letting the rest of us know, that success is possible, as I'm starting to have my doubts, and would appreciate very much any suggestions that would point the rest of us in the right direction.

    I would love to hear from Alex and Brad, because as an ex-programmer, I can say with certainty more talented programmers than you guys don't exist, and if you don't have an arsenal of systems that kick butt in certain market environments then the chances for me succeeding is almost impossible.



    Thanks very much for all your help wishing you all much success.

    [email protected]
    Last edited by demarcog; 06-04-2005, 06:40 PM.
    Glen Demarco
    [email protected]

  • #2
    It is possible, but from my experience it takes alot of knowledge of the strategy your attempting to mimic and hours of trial and error. Starting of with the basics, finding where and why the drawdowns occur and fixing them. (which lead to more)

    It is definately not an easy or short process, but it can and has been done.
    Excellent book on JavaScript for beginners

    Comment


    • #3
      Here is a screen shot of today's trade signals for the ES. This system was made for the ES and performs the best on them.

      This is just my basic version and I am working on a newer version as time allows.

      Long ours of trial and error, but it is getting to the point where it worth all the hard work.
      Attached Files
      Excellent book on JavaScript for beginners

      Comment


      • #4
        demarcog,

        I also working on an efs since 1 year to develop a trading system, hard work. I asked also on the forum to develop one together, but no response. Maybe you have, please check my last posts.
        I am checking my efs with the tick replay tool. Sometimes (often) I don´t understand the trading, e.g. when my efs makes a loss. So I thought I have only to change long with short and to change also the exits to make profit. But this don´t work. Maybe Alex can explain it.

        regards
        sams

        Comment


        • #5
          Hi guys,

          I read this thread expecting to see a lot of positive replies and examples, many refered to by eSignal people [thinking how typical sales people give you a lot of references of success stories to convince you their product is good], but I was so wrong.

          LOL
          Last edited by mbuta; 06-05-2005, 01:20 PM.
          Mihai Buta

          Comment


          • #6
            It's possible - very possible. After a lot (emphasis on alot) of work, I have ended up with something that finally works. I have been working on this for a while now and the accuracy rate is pretty high (about 80 to 90 percent for backtesting. The success rate is ranged because I'm still tweaking some of the variables). It only gives about a trade a day, sometimes two, but with an 80 to 90 percent success rate, I don't mind

            The results below represents a trading interval for AB Z6 (E-mini Russell 2000) in which 6 points ($600) of profit was made in backtesting results. The system lost 2 points ($200) today but following the strategy, I am confident that it will make it back and then some by the end of the week. The chart below is that of two days back. I'm not posting today's results because it shows less trades so it wouldn't give a very good visual of of the system. The system is very dependant on using exact stop losses and profit targets (2 points/$200 for each) to end up with a sizeable profit at the end of each week.

            Dark Red Background - Short entry
            Dark Green Background - Long entry
            Yellow Background - Exit the position if the profit target has not yet been reached
            Attached Files
            Last edited by naifwonder; 09-27-2006, 10:50 AM.

            Comment


            • #7
              Simple answer..

              if creating an automated trading system that made consistent $ was easy, everybody would have done it by now and everybody would be rich beyond their wildest dreams...

              *poof* - back to the real world..

              It's not easy and not everybody can do it. Most of the people that I've actually seen (clients) do this have been from two breeds..

              a. someone that has a system in their head (or partially developed) they are confident in and need my help to automate it.

              b. someone that has an idea for a system, and wants help proto-typing it..

              The type #b peopls typically have a much further path to go thru to achieve their goals as they are just beginning the process.

              The way I look at Consistency is as follows...

              A. Consistent profits over a 3+ month time frame with limited drawdowns.

              B. A contained loss limiter (of some type)

              C. An ability to manage itself.

              I've seen/created systems that work well and others that don't. This can be done - but again - IT'S NOT EASY.
              Brad Matheny
              eSignal Solution Provider since 2000

              Comment


              • #8
                Originally posted by naifwonder
                It's possible - very possible. After a lot (emphasis on alot) of work, I have ended up with something that finally works. I have been working on this for a while now and the accuracy rate is pretty high (about 80 to 90 percent for backtesting. The success rate is ranged because I'm still tweaking some of the variables). It only gives about a trade a day, sometimes two, but with an 80 to 90 percent success rate, I don't mind


                I've done alot of work testing many, many systems and developing my own and also trade the russell and have to tell you I've never come close to 90 percent profitable for any system on any market, that is truly an amazing accomplishment, congratulations.

                I had a few that back tested that well, but when I started trading it realtime, I discovered many problems with the way I coded the strategy....necessary learning curve I guess...

                I'd be interested in any suggestions you have regarding indicators methods etc, and would be very interested in seeing the strategy reports or any other information you are willing to share with us....what indicators, how it works, etc.,

                The results below represents a trading interval for AB Z6 (E-mini Russell 2000) in which 6 points ($600) of profit was made in backtesting results. The system lost 2 points ($200) today but following the strategy, I am confident that it will make it back and then some by the end of the week. The chart below is that of two days back. I'm not posting today's results because it shows less trades so it wouldn't give a very good visual of of the system. The system is very dependant on using exact stop losses and profit targets (2 points/$200 for each) to end up with a sizeable profit at the end of each week.

                Dark Red Background - Short entry
                Dark Green Background - Long entry
                Yellow Background - Exit the position if the profit target has not yet been reached
                It's been a few weeks, how is the system going, have you started actually trading it yet?

                Would be interested in how the actual trading is going relative to the back tested results and of course any information, as 90 percent profitibility is fantastic....good luck..
                Last edited by demarcog; 10-07-2006, 10:37 PM.
                Glen Demarco
                [email protected]

                Comment


                • #9
                  Re: Simple answer..

                  Originally posted by Doji3333
                  [B]if creating an automated trading system that made consistent $ was easy, everybody would have done it by now and everybody would be rich beyond their wildest dreams...
                  Not sure what you mean by "everybody". If there are 100 people placing an order for a given stock on a given day, what percent of those everybodies, are professional traders, let's estimate that at 50 percent.

                  Out of those 50 traders, how many are actually successful, less then 50 percent ....now we are down to 25 percent. Of those 25 percent let's say they are all "technically/chart based traders". Out of those 25 how many have ever heard of easylanguage, or javascript, 1 or 2??

                  Of those 1 or 2 how many know enough about, programming, tecnhnical analysis, strategy development, money management, placing orders, selecting stocks, time intervals, and most importantly spent a few years placing orders and actually trading......my point is hardly ...everybody...closer to NOBODY...

                  and I've worked for 20 years at well know Wall Street firms and you are talking about far fewer people then one would think ......and yes those people and I'm including us in this, if we stick to it can definatley make alot of money....if we have to discipline to stick with the system, the business savy to get the money to trade, etc etc..........not a trivial accomplishment





                  My inquiry was the people in this forum are not EVERYBODY, but the .5 percent of active traders and I had not seen a single profitable trading system published anywhere, ever, by anyone and was wondering at one point not too long ago for a time period is such a thing was possible....

                  You published what I thought was an amazing moving average system (MENT-X), complete with very sophisticated money management features and I played with it for a week and thought that if I can't get this thing to make money ,then nothing will make money.

                  My post was simply, hey guys, if the people in this forum aren't doing it, then it's maybe the "random walkers" of the world are correct, and it is impossible to "predict the market", because that's ultimately what we are talking about doing, predicting the market....f

                  I could accept, and expected many post like.."hey I've done it and that's for me to know and you to find out",

                  ( God forbid someone gives something profitable away to someone else....can't be that generous...might as well give a dollar to a homeless person... and what percent of the population walking down the street does that?)

                  But the silence was very telling, and I appreciate you comments and *poof* - back to the real world.....we are hardly everybody and it is much more difficult then I imagined but I've done it...and the best strategies are the least complicated ones, it took me too long to realize that.

                  Look at a plain old boring low tech 20 period EMA..

                  which way is it pointing up or down..(say up ...)

                  is the probability that it will continue in that direction for some length of time and go higher rather then suddenly reversal in the opposite direction...absolutely...that all we need to know..

                  that''s the key to a trading system...a slight favorable probability, that when a certain "directional price/indicator movement" occurs in the market, that that "movement" has a higher probability of continuing rather then reversing....and that is achievable...not easily by definately achievable.
                  Glen Demarco
                  [email protected]

                  Comment


                  • #10
                    I didn't trade the 90% system because while I was tweaking it, literally the day before I started actually using it, I found a more effective strategy that gave less drawdown and more profits per trade. I hate drawdowns because they make me nervous (my account size doesn't leave much room for risk). Being the obsessive compulsive that I am, I can't use something when I know something so much better is within my grasp lol.

                    Furthermore, I also wanted to find a way to make my profit targets greater than my stop losses. This new system fixes that. So after much work, I present you with my new trading system. I wanted to trade on it today but it didn't give any trades. It gives about one trade every 1.7 days. The good news is that in the past 20 days of data (1 month of trading), it gave 12 trades and all of them were successfull.. yes, 100% profit ($3450 total)

                    If you want to see the system in action, I set up a mini-site with more details and pictures. BTW, to anyone thinking it, I'm not spamming the page. I simply made the page because I think it is a better way of showcasing the system.

                    Click here to see the system in action

                    As for your request Demarcog, the indicators used in this system are a combination of some very basic indicators and a few more intricate ones that I made myself. Together, they give a pretty accurate picture of how much momentum the instrument has to go in a certain direction and how much of this momentum has been used so far. If the momemtum is great enough that it can reach a preset profit target with little to no drawdown, than the signal is given.

                    All signals are given on the close of the bar and the entry price should be the closing price of the bar or better. This method doesn't always give the best price, but it does for the most part give a good price and prevents entries that can easily lead to the trade being stopped out. Slippage has been taken into consideration by setting profit targets that are actually lower than what the momentum can potentially carry the instrument too. I also use fairly high time frames for my chart (minutes) as I find that this gives the best overall picture of the market. Tick charts have their advantages as well, but they do not work as well with this system due to their lack of consideration for the time factor.
                    Last edited by naifwonder; 10-09-2006, 12:36 PM.

                    Comment


                    • #11
                      naifwonder,

                      I made that post well over a year ago, and you are the first to respond with a profitable system, and a 100 percent profitable one at that.

                      CONGRATULATIONS on an amazing job..just looking briefly at the signals it looks like you could have actually gotten alot more profit out of 11 of 12 of those signals.

                      I hate drawdows also, but if you could double the profit, by getting stopped out of 10 or 20 percents of your trades it's a favorable tradeoff, check it out in the backtester.

                      I would like to see the results over the maximum 120 days, does it do as well historically?

                      Nice job....thanks again for sharing your expeience..
                      Glen Demarco
                      [email protected]

                      Comment


                      • #12
                        Thanks for all the feedback guys. I will backtest the system to the 120 day mark very soon and will be posting the results. I haven't done it yet because I haven't got arround to properly coding the system for backtesting it - but I will soon. As for the 17 minute bars, ill tell you this much - odd numbered time intervals usually give a better picture of the market than even number time interval bars. I can fairly accurately predict the outcome of the next 2 bars or so, but after that it gets tricky. That is why I used such a high time frame - to be able to take profits within one bar that I can predict with consistent accuracy. I tried using lower time frames and more bars, but I couldn't find a way to not get whipsawed around

                        As for taking larger profits, I definetely know what you mean demarcog and that is actually what I was originally thinking. However, due to my account size not giving me much room for loss, I was far more focused on the accuracy rate rather than the profit yield of the trades. I am working on a way to lock in those larger profits while still maintaining the accuracy rate and if I succeed, I will ofcourse let you guys know.

                        If the system backtests the full 120 days with good results and I make good money with it in the first month, I might consider selling a subscription to the system. I will still be trading based on it, but since my account size doesn't really let me use the system to its utmost advantage, I am thinking that a subscription system would be a a good way to raise money for my account. Would you guys think it is worth it?
                        Last edited by naifwonder; 10-09-2006, 08:38 PM.

                        Comment


                        • #13
                          Originally posted by naifwonder
                          [B]Thanks for all the feedback guys. I will backtest the system to the 120 day mark very soon and will be posting the results. I haven't done it yet because I haven't got arround to properly coding the system for backtesting it - but I will soon.

                          How did you test this system to determine it's effeciveness if not by backtesting?

                          A dozen trades over 20 days at 100 percent is a great start but you would ideally like a much larger number of trades over as long a period as possible to have a sound statistical basis prior to risking you money.

                          I sincerely hope I'm wrong but any system that only generates a small mumber of trades in a limited time period, and in particuliar is entering and exiting on the same bar and is 100 percent profitable should be checked thoroughly using tick replay and then forward tested realtime, as I don't want to see you loose money. Even the emini Russell2k is a BIG contract, about $70K worth of stock and can move quickly.

                          JasonK has recently published several invaluable, extremely comprehensive, and very well written documents on backtesting in the Knowledge Base tutorial section that are a must read IMHO prerequisite to any serious attempt at a valid backtest of any trading system.

                          He documents several technique to be avoided such as using MARKET-THISBAR when evaluating trade decisions on bar 0, which will grossly over inflate the profits of most backtested systems.

                          One other problem is the "curve fitting" issue. When there are a large number of variable inputs into a system, tweeking too much will fit the variables to the data and show great results, but rarely do as well when applied to other historical or future time periods.

                          I would definately test this thoroughly using the full 120 days of data available. Using 17 minute intervals it should ran in 30 seconds or less.

                          Then test it on 10 days of tick data after downloading the trades as the backtester only has completed bar information which will not always mimic 100 percent "realtime"market action.
                          Then manually watch it realtime for a week or so, or use the generic broker functions and the Esignal Paper Broker to send traded realtime as a precaution prior to actual trading.

                          You mentioned that your account cannot handle any losses better to be safe then sorrow, and worse yet out of trading capital (trust me I speak from too many real experiences on this one).





                          As for the 17 minute bars, ill tell you this much - odd numbered time intervals usually give a better picture of the market than even number time interval bars. I can fairly accurately predict the outcome of the next 2 bars or so, but after that it gets tricky. That is why I used such a high time frame - to be able to take profits within one bar that I can predict with consistent accuracy. I tried using lower time frames and more bars, but I couldn't find a way to not get whipsawed around
                          I've learned that the larger the time period the easier it is for a system to be profitable, and 15 or 17 minute is still a relatively short term time period.

                          What worries me a little is the "curve fitting issue" if this system makes money on 17 minute, but not 18 minute or 16 minute, that's a definite sign of overly fitted variables. If it were a moving average cross over for example and 5 and 20 does well, and 4 and 19, or 6 and 21 looses money it's definately curve fitted, there should be a bell curve of profitability with the most optimized values making the most money, but the variables closes in value also being profitable, and even distribution surronding the optimum values or the probability is it will not do well.



                          As for taking larger profits, I definetely know what you mean demarcog and that is actually what I was originally thinking. However, due to my account size not giving me much room for loss, I was far more focused on the accuracy rate rather than the profit yield of the trades. I am working on a way to lock in those larger profits while still maintaining the accuracy rate and if I succeed, I will ofcourse let you guys know
                          This is where the backtesting comes in, there are enough strategy EFS examples in JasonK's fileshere and esignals libraries that take the guess work out of it. I can see just by looking at the signals that most of them there was no reason to get out, bars were green (when long), higher lows, higher highs, etc. you can find out in a few minutes testing it with a profit target specified to determine what values to specify to maximize profits/stops.

                          What I look for at the end of that process (and there are alot of fancy statistical measurements), but once a technical analyst, always a technical analyst is a Equity Curve that looks like this (see attached) AFTER a worse case slippage and commission deduction.

                          If the system backtests the full 120 days with good results and I make good money with it in the first month, I might consider selling a subscription to the system. I will still be trading based on it, but since my account size doesn't really let me use the system to its utmost advantage, I am thinking that a subscription system would be a a good way to raise money for my account. Would you guys think it is worth it?
                          I'm not the right person to ask, as I wouldn't know anything about buying or selling a system. If there is a system out there increases someones trading success it seems like a reasonable choice. Personally, I'm also like you somewhat obsessive, either I do it with my own system or I die trying (and broke).....so far it can go either way......
                          Attached Files
                          Glen Demarco
                          [email protected]

                          Comment


                          • #14
                            I went back and tested through the charts myself, never actually put it into the backtester. I'll see if I can get it run on the backtest system by tommorow (Oct. 10). I guess it is wrong for me to claim this to be a good system until I do otherwise so it is a top priority of mine - my apologies.

                            As for working only on 17 minute charts, that is sort of right. I can get it to work on different interval charts, but than I would have to adjust the momentum indicator variables. The momentum indicator must be very specific since even a one tick difference in a bar can mean the difference between there being an entry and no entry. After trying out other intervals and variables for the indicator, I ended up finding that the 17 minute seems to work best (lowest drawdown and leastt losses).

                            Yes it is true that the variables for this system are very tightly fitted but I have seen many systems use more generic trade conditions and they, for the most part, ultimately lead to loss or drawdowns that are too much of a risk for my taste. I think you would agree that very strict variables are better than loosely fitted variables that yield more failed trades and drawdowns. I will try and find more conditions that yield consistently profitable trades and if/when I do, I will be adding them in to increase the frequency of trades.

                            As for taking profits within one bar, the trades still would not get stopped out. Assuming the order is filled at the close price of the signal bar, the stop loss limit is never actually reached before the profit target is. For that reason, the signal is given on the close of the bar instead of in real time - to ensure a well priced entry.

                            If I can up the profit targets while still staying within a high rate of successful trades, than I will implement it.

                            Like I said, it would not be fair of me to claim anything more of this system until the actual backtest results are in so I will be getting that to you guys shortly. If it works, I will leave it on auto trade with my demo account for my broker for 10 days. If that is profitable... well my college tuition should become more than taken care of
                            Last edited by naifwonder; 10-10-2006, 12:31 PM.

                            Comment


                            • #15
                              This is very strange.. I went to backtest it right now and when I load the 120 days worth of data this is what my charts look like once I go back past about 25 days worth of data. Well actually for about 10 days past that, the bars have large gaps in between them, and than it starts to look like this. Correct me if I'm wrong, but I seriously doubt the charts actually looked like this. As soon as I figure out how to fix this, I'll get you the results. .
                              Attached Files

                              Comment

                              Working...
                              X