A back test is where a trader takes a series of historical price action data and tries to find patterns that were profitable in the past. The trouble is when a trader tries to keep changing the parameters of a back test until they fit in to a path to profitability in the past.
A good back test does not evolve out of hindsight, we have to look and see what the data shows us. A better filter of data is to set the parameters of signals around our theories about what would have been profitable based on chart studies and let the data speak for itself. The other filters we must use is a diversified watchlist of markets to see if the signals held up across multiple markets and also a large enough sample size to cover different market environments: up trends, downtrends, range bound markets, bubbles and crashes. A great system in a bull market may cause ruin in a bear market.
Buy and hold investors feel like geniuses when the stock market makes a new all time high but wished they had exit signals at the bottom of a bear market. Swing traders love markets with defined support and resistance but can be left behind during strong trends in one direction. All trading systems have drawdowns no matter how high the win rate is because nothing works all the time.
All a back test is really trying to do is use mechanical and repeatable signals that give profitable entry and exit signals that create bigger wins and smaller losses over the long term. This is accomplished by cutting losses short and letting winners run. A system has to have signals that filter out a lot of the price action noise that causes over trading and instead signal the meaningful moments in price action. Trading back tested signals also filters out the emotion of trying to decide what to do each day from your own perspective. You convert from and opinionated predictor to the follower of a systematic process. Your job becomes to follow signals and ignore your own feelings.
There is no perfect back tested system there is just the system that you can confirm that worked over multiple markets in the past and has a great potential for working in the present and future over the long term.
“The moral is simple: Don’t draw any conclusions about a system (or indicator) on the basis of isolated examples. The only way you can determine if a system has any value is by testing it (without benefit of hindsight) over an extended time period for a broad range of markets.” – Jack Schwager