CluenexWhat is backtesting?
Backtesting is applying a trading rule, written in exact terms, to historical prices to measure what it would have returned and how bumpy the ride would have been. It replaces a plausible story with a list of real outcomes, one for every time the rule would have fired.
Compare two forms of the same idea. The story: "Hurricanes damage homes, repairs need hardware, so home improvement retailers should rise." Every link sounds sensible, and the story cannot be proven wrong. The rule: "Buy on the close the day before a forecast landfall of a major hurricane, sell five trading days later." The rule produces a specific number for each past storm.
When the story and the rule disagree, the rule is the evidence.
Why are widely known events already priced in?
A share price is a claim on expected future profits, so it moves when those expectations change, not when the expected thing happens. If everyone can see an event coming, buyers position early and the price adjusts before the event.
Atlantic hurricane season officially runs from June 1 to November 30 each year, and the National Hurricane Center publishes forecasts days ahead of landfall. Large investors read the same forecasts, with faster tools and lower costs than a person trading from a brokerage app.
What moves a price at the event is the surprise: the gap between what happened and what was expected. The direction of that gap cannot be known in advance. This is the practical idea behind the semi-strong form of the efficient market hypothesis, which says public information is already reflected in prices. It does not say prices are always right. It says that acting on widely known information is not, by itself, an edge. For more on this, see why markets price probability, not headlines.
How do you backtest a seasonal trade honestly?
Write the rule first, test it on the full history, count every variation you tried, subtract costs, and check it on data the rule never saw. Skipping any step tends to flatter the result.
- Define the rule before looking at results. Fix the entry, exit, position size and list of stocks in advance. A rule tweaked after seeing returns is no longer being tested.
- Report the whole distribution. Show hit rate (share of winning trades), median, worst single loss and the drawdown path. An average can hide one giant outlier. See what a drawdown is.
- Benchmark against doing nothing. Compare with holding the same stock and with holding an S&P 500 index fund over the same days.
- Subtract realistic costs. Include the bid-ask spread, any commission, slippage on the size you actually trade, and taxes. Gains on positions held one year or less are taxed at ordinary income rates in the US.
- Count how many rules you tested. Reporting the best of many date pairs is not a finding.
- Hold out data. Build the rule on one period and test it on another it has never seen. A rule that works only in the first period was probably fitted to noise.
FINRA treats backtested and hypothetical performance as something that needs strict guardrails in advertising, which is a useful reminder that these numbers are easy to overstate.
How many tests does it take to find a fake winner?
About 14 random rules give a better than even chance that one passes a standard 5% test by luck, and 20 rules give about 64%. The formula is 1 minus 0.95 raised to the number of tests.
| Rules tested | Chance at least one looks significant by luck |
|---|---|
| 1 | 5.0% |
| 5 | 22.6% |
| 10 | 40.1% |
| 20 | 64.2% |
| 50 | 92.3% |
This is why "I tried buying on every date from March to October and found one that worked" is not evidence. Searching 30 start dates and 30 exit dates means testing hundreds of combinations.
Worked example: a thin edge that does not survive
Here is a hypothetical seasonal rule, made up to show the arithmetic. It is not a real result for any stock.
Suppose a rule fires 10 times over 30 years. The average gross return per trade is 2.7%, and the typical swing between trades (standard deviation) is 6%.
- Standard error: 6% divided by the square root of 10 is about 1.9%.
- Test statistic: 2.7% divided by 1.9% is about 1.4. The usual bar is roughly 2.
- Verdict: with 10 observations, an average of 2.7% is statistically hard to tell from zero.
Now add costs. If the round trip costs 0.3% in spreads and slippage, the net average is 2.4%. If it also lost money on 4 of the 10 trades, the strategy needs small positions to be survivable, and a small position times 2.4% a few times a decade barely registers in a portfolio.
The lesson is the shape of the problem: rare events give few observations, and few observations give wide error bars.
Rewriting a hunch as a testable rule
The hunch "buy home improvement stocks around hurricane season" cannot be tested because it names no entry, exit or condition. Making it testable forces choices, and each choice exposes a weakness.
Version A: calendar-based. Buy June 1, sell November 30, every year. This uses no storm information. Its result mostly reflects whatever the broad market did in those six months, so any gain is market exposure labeled as a weather strategy.
Version B: event-based. Buy before a forecast major landfall, sell five trading days later. This needs the storm's strength and location in advance, which the market sees too. The forecast is a second prediction problem the backtest does not measure.
Version C: surprise-based. Buy only when actual damage exceeds the expected damage. This has a real logic, since it trades the surprise. It is also nearly impossible to carry out, because damage estimates arrive after prices have already moved.
Written precisely, most everyone-knows trades turn into one of three things: a market-return proxy, an already-priced expectation, or a rule that needs information nobody has in time.
Do seasonal patterns exist at all?
Some calendar effects have been documented in research, such as the turn-of-the-month effect and the January effect in small stocks. But published patterns tend to fade once people know about them.
McLean and Pontiff studied 97 return predictors from academic papers. They found the predictors' returns were about 26% lower out of sample and about 58% lower after publication. The gap is consistent with traders acting on the findings and competing the edge away.
A pattern that existed in the past does not prove it will continue.
Where does backtesting fail?
Backtests fail in several predictable ways, and a good result can be wrong even if the arithmetic is correct.
- Look-ahead bias: using information that was not available at the time, such as restated financials, or trading at a closing price that was used to trigger the signal.
- Survivorship bias: testing only companies that still exist today, which leaves out the failures and flatters the result.
- Overfitting: tuning a rule until it fits the past perfectly, which usually means it fits noise.
- Regime change: interest rates, market structure and trading costs differ across decades, so old results may not transfer.
- Causal plausibility is not profit: a mechanism can be completely real (storms do raise repair spending) while the price impact is already in the price.
- More years is not always more evidence: 30 years of data on a rule that fires once every three years is still only 10 observations.
Backtesting is also not only for professionals. A spreadsheet and public price history are enough. The hard part is the discipline of not changing the rule after seeing the answer.
How does Cluenex handle testing?
Cluenex publishes the test results behind its named signals, including signals that failed and were retired, so a pattern has to earn its place. Its charts include technical indicators and harmonic pattern zones, but Cluenex treats chart patterns as context, not as buy or sell signals, because its own tests found no reliable directional edge in them. Cluenex also seals every verdict nightly with a SHA-256 fingerprint and scores it against the S&P 500, with public pages revealing verdicts 30 days later, on its track record page.
What this means for you
If you hold a regular job and a brokerage account, the useful habit is to treat every calendar tip as a hypothesis to test, not a fact.
- Write the rule in one sentence with an exact entry, exit and size before looking at any chart.
- Ask how many times it fired. Fewer than about 30 independent trades means the result is mostly noise.
- Compare to a plain index fund over the same days, after taxes and costs.
- Ask who is on the other side. If the pattern is in a news article or a social post, assume professionals saw it first.
- Size small. Even a rule that passes should risk only a small share of your money. See position sizing.
- Be wary of fake precision. Claims of very high accuracy on breakouts or patterns deserve the same scrutiny, as covered in breakout vs fakeout.
A tested rule that fails has still saved you money. An untested rule that sounds smart is just a story about the market.