Optimizing studies and systems

[gspiker]

michaelm,

Thanks for the reply. Two votes for optimizing the system as a whole. This of course is harder, and I really wanted someone to convince me that I was wrong and could do it the simple way ;-).

2) Great question. However, how do YOU define optimal? Is it simply testing a system to produce the most winners and least losers? Is it risk/reward? Is it finding consistency? Is it scooping most of a trend?

This is a great question. I suspect it is the one bfry was asking but I didn't understand what he was driving at. And it does show that a pure logic approach (that which makes the most $$) may not be the best to trade, since I know from personal experience that I will not put up with large drawdowns or a large number of loosing trades even if the system has shown that I will likely make a ton of money following it.

If I had to rate it, I would say the most important is a good risk reward, followed by most $$, followed by a reasonable win/loss. Are there other parameters I should think about here?

I supposed one way to approach this is to rank each run of the system (with some unique change to the rule sets) on how well it does on each of the above. Then apply some weight factor (rank) to how important it is and come up with a total for each. I'll have to think about this some more...

Garth

[gspiker]

All,

PM me if you want to discuss further, since this is probably getting off-topic. But the right analysis reveals volume patterns which are highly predictive of short-term price movements.

This is off topic, but I bet is of interest to a large number of traders (including me). Instead of moving it to PM, why not start another thread in a more appropriate group and then put a pointer here?

Thanks,

Garth

Garth,

Several questions for you. First, you are talking 10 years of data. Are you getting the data from Real Tick or another data provider.

Also, what interval are you looking at using when running optimizations over this long of a period.

Finally, what language are you using to run your optimizations, or are you using a canned program.

You may want to discuss optimization algorythms with Chris Kryza (Divergence Software) as I believe he has some experience with GA and AI coding and their resources.

Also, as you probably know , be wary of the traps associated with optimization.

Regards,

Garth,

Make it three votes for optimization as an entire system.

[gspiker]

Hi Steve,

Several questions for you. First, you are talking 10 years of data. Are you getting the data from Real Tick or another data provider.

daily bars... eSignal has that many daily bars...no worry about importing data.

Finally, what language are you using to run your optimizations, or are you using a canned program.

It's all EFS for now. I do think I would have been better served writing my own scripting/modeling language and exporting the data, but I'm now so far along that I will continue with EFS.

Garth

Garth,

Thank You for the info, honestly I had not even considered you were looking at daily bars, I just assumed hourly or another small interval. Daily bars are much more manageble. Even using multiple individual indicators, optimization would not be that difficult or time consuming based on a that relatively small amount of bars. btw, when I tried to load up daily bars, I could only get data back to 9/09/1997. (Correction, looking at wrong index, had no prob getting back to 89)

Regards,

[gspiker]

Thanks Steve,

A comment on a comment you made in your original post. I agree you have to worry about optimization (I think I have even pointed this out in a few threads). My points have always been:

1) Make sure you have a few thousand bars of data...

and as important:

2) Make sure it works in both a bull and a bear. Given the last 10 years we have seen both actions, I feel pretty good about this. It was getting a wee bit hard pre-2000 to have a good bear to test against. My guess is a lot of systems fell apart post 2000, even those that were highly backtested back 4-10 years.


Now I have to go and think about a good way of doing all of this. I have some idea's, but I think for this all to work I need some very clever master EFS.

Garth

[z11]

Originally posted by gspiker
Spike,



Combining multi-timeframes isn't an issue, I already do that and backtest a system that uses the results from higher time frames to influence lower.


EFS2 will be a huge help when it is finially stable and released.

Garth

hi Garth... was doing some looking around and found this post from you. I am now using multiple time frames and am finding backtesting with multiple time frames a daunting task...
the route I am thinking is dumping all relevant data from each time frame to a file (or files) and read that back into the EFS that uses all of these data...but one have to synch up all the time stamps...really a cumbersome task ...

would you kindly share your methodology of backtesting with multiple time frames if done differently?

Also, would anyone know when this mystical EFS2 will actually be manifesting itself into a form of reality?

regards,