Historical Performance and Backtesting

Educational Resource: This guide explains quantitative analysis and historical testing methods. These are educational tools for understanding market patterns, not predictive guarantees. All investments involve risk and past performance does not guarantee future results.

What is Backtesting?

Backtesting is how we validate whether a pattern actually worked in the past. Think of it like checking the weather forecast's accuracy by comparing predictions to what actually happened.

For every opportunity score you see, we've tested that pattern against years of historical data to verify it's real - not random noise.

Why Historical Testing Matters

Anyone can find a pattern that worked once or twice. The hard part is finding patterns that work consistently across:

• Different market conditions (bull markets, bear markets, sideways markets)
• Multiple companies and sectors
• Various time periods
• Different volatility regimes

Our backtesting answers one question: "If I saw this pattern before, what happened next?"

How We Validate Patterns

Step 1: Pattern Identification

We scan historical data looking for relationships between events (like insider buying) and outcomes (like price changes).

Example pattern:

    
    WHEN insiders bought heavily (signal)
WHAT HAPPENED to the stock 30 days later? (outcome)

  

Step 2: Historical Testing

We find every time this pattern appeared in our database and measure what happened next.

For a pattern to qualify, we need:

• Minimum 30 occurrences - No cherry-picking
• Correlation ≥ 0.3 - Pattern must be meaningful
• p-value < 0.05 - Less than 5% chance it's random

Step 3: Out-of-Sample Validation

This is the key that separates real patterns from overfitting.

Training Period: We identify the pattern using data from 2010-2020 Testing Period: We verify it still works on data from 2021-2024 (data the model never saw)

If the pattern only works on training data, we discard it. We only show patterns that work on fresh data.

Reading Backtest Results

When you view an opportunity's historical performance, here's what each metric tells you:

Success Rate

What it is: Percentage of times the pattern preceded the expected outcome

Example: "72% success rate" = the pattern worked 72 out of 100 times

How to interpret:

• Above 70%: Strong historical reliability
• 60-70%: Good consistency
• 50-60%: Moderate - requires additional confirmation
• Below 50%: Not worth trading (worse than coin flip)

Average Return

What it is: The average price change when this pattern appeared historically

Example: "+8.2% average return over 30 days"

Important caveats:

• Averages hide outliers (one +50% move can skew the average)
• Check median return too (less influenced by extremes)
• Look at distribution - are returns consistent or highly variable?

Sample Size

What it is: How many times we've tested this pattern

Example: "n=156 occurrences"

Why it matters:

• n ≥ 100: High confidence in the pattern
• n = 50-100: Moderate confidence
• n = 30-50: Use caution, smaller sample
• n < 30: Not enough data (we filter these out)

Learn more: Sample Size Importance

Time Periods Tested

What it is: The date range we validated this pattern across

Example: "Tested across 2015-2024 (10 years)"

Why it matters: Patterns that work across different market regimes (2015 bull market, 2020 crash, 2022 bear market) are more robust than patterns that only worked in one environment.

Real Example: Insider Buying Pattern

Let's walk through actual backtest results for an insider buying pattern:

Pattern Description

Signal: Net insider buying ≥ $500K over 20 days Outcome: 30-day forward returns

Backtest Results

What This Tells Us

Reliability: With 156 tests and 68% success rate, this pattern has strong historical support.

Expected Outcome: When this pattern appears, the median stock rose 4.2% over the next 30 days. The average is higher (6.4%) suggesting some big winners pulled the average up.

Statistical Confidence: The p-value <0.001 means there's less than 0.1% chance this is random. It's a real pattern.

Practical Application: This is a high-quality setup. Not guaranteed, but historically it's worked more often than not.

Understanding Performance Distribution

Historical performance isn't just about averages - it's about the range of outcomes.

Best Case Scenario

The top 10% of historical occurrences:

    
    +15% to +25% returns

  

These are the home runs - but they're not typical.

Typical Scenario

The middle 50% of historical occurrences:

    
    +2% to +8% returns

  

This is what you should expect most of the time.

Worst Case Scenario

The bottom 10% of historical occurrences:

    
    -5% to -12% returns

  

Even strong patterns fail sometimes. This shows the downside risk.

Limitations of Historical Testing

1. Past ≠ Future

The most important limitation: history doesn't repeat perfectly.

Market conditions change. Regulatory environments shift. Technology evolves. A pattern that worked for 10 years might stop working tomorrow.

2. Survivorship Bias

We can only test on companies that still exist. Companies that went bankrupt aren't in the database anymore.

This means historical returns might be slightly optimistic because they don't include complete failures.

Learn more: Survivorship Bias

3. Market Impact

Our backtests assume you could trade at historical prices. In reality:

• Large orders move the market
• Liquidity varies
• Slippage occurs
• Trading costs matter

4. Regime Changes

Market structure changes over time:

• Algorithm trading increased
• Options markets grew
• Information spreads faster
• Retail participation exploded

A pattern from 2015 might work differently in 2024.

How We Address These Limitations

1. Out-of-Sample Testing

We always test on data the model hasn't seen. This catches overfitting before it reaches you.

2. Multiple Time Periods

We validate across bull markets, bear markets, and everything in between. Patterns must work in different environments.

3. Statistical Rigor

We use strict significance thresholds (p < 0.05) and require minimum sample sizes (n ≥ 30).

4. Regular Revalidation

We continuously retest patterns as new data arrives. If a pattern stops working, we remove it.

5. Transparent Reporting

We show you the sample size, correlation strength, and confidence levels. You can judge quality yourself.

Using Historical Performance in Your Research

Step 1: Check the Sample Size

Before anything else, verify n ≥ 50. Smaller samples are less reliable.

Step 2: Examine Success Rate

Look for patterns with ≥60% historical success rate for your risk tolerance.

Step 3: Understand the Distribution

Don't just look at average returns. Check the range of outcomes. Can you handle the downside?

Step 4: Verify Statistical Significance

Confirm the correlation is strong (r ≥ 0.5) and the p-value is low (p < 0.05).

Step 5: Consider Current Context

Is the market regime similar to the historical testing period? Or have things changed?

Common Questions

Q: If the backtest shows 70% success rate, will my trade have a 70% chance of success?

A: Not exactly. The 70% reflects historical frequency. Your actual trade faces unique circumstances the backtest never saw. Think of it as "similar setups worked 70% of the time historically."

Q: Why don't you show me every pattern you test?

A: We filter aggressively. For every pattern you see, we tested dozens that failed our quality standards. We only show high-probability setups.

Q: Can I see the raw backtest data?

A: Pro subscribers can export historical performance data for deeper analysis. The Opportunity Detail panel shows summary statistics for all users.

Q: How often do you update backtests?

A: Continuously. As new data arrives, we revalidate patterns. If performance degrades, we adjust scores or remove the pattern.

Q: What if a pattern stops working?

A: This happens. When we detect degradation, we reduce the pattern's weight in the score or remove it entirely. We monitor performance in real-time.

Next Steps

Ready to understand the technical details? Learn about our Backtesting Methodology to see how we build and validate patterns.

Want to understand why some patterns are more reliable? Read about Sample Size Importance.