Understanding Confidence Levels
Educational Resource: This guide explains quantitative analysis and statistical methods. These are educational tools for understanding market patterns, not predictive guarantees. All investments involve risk and past performance does not guarantee future results.
What is Confidence?
Confidence measures how thoroughly we've tested a pattern. It answers the question: "How many times have we seen this pattern play out in historical data?"
Think of it like a medical trial:
- Low confidence = Small study (30 patients)
- High confidence = Large study (1,000 patients)
Which study would you trust more? The larger one, obviously. Same principle applies here.
Why Confidence Matters
Two Opportunities, Same Score
Opportunity A:
- Overall Score: 85
- Confidence: 90
- Sample Size: 200+ tests
Opportunity B:
- Overall Score: 85
- Confidence: 35
- Sample Size: 35 tests
Which is better?
Opportunity A is far more reliable. Both have the same score, but Opportunity A has been validated 200+ times. Opportunity B might be real... or it might be statistical noise that disappears with more data.
Key insight: Always check BOTH score and confidence. High score with low confidence is a trap.
The Confidence Scale
Confidence ranges from 0-100, where:
80-100: Very High Confidence
What this means:
- Pattern tested 150+ times in historical data
- Proven across multiple market conditions (bull, bear, sideways)
- Strong statistical reliability
- Low chance of being random noise
Trading implication: These are your "bread and butter" setups. Start here when learning the system.
Example patterns:
- Insider buying before earnings in large-cap tech (tested 200+ times)
- Post-FDA approval biotech momentum (tested 180 times)
60-79: High Confidence
What this means:
- Pattern tested 80-150 times
- Good statistical evidence
- Reliable across most conditions
- Some variability possible
Trading implication: Solid opportunities. You can trade these with normal position sizing.
Example patterns:
- Insider buying in mid-cap industrials (tested 120 times)
- Momentum reversals after oversold conditions (tested 95 times)
40-59: Moderate Confidence
What this means:
- Pattern tested 50-80 times
- Decent evidence but not extensive
- May be sector-specific or condition-dependent
- Higher chance of edge cases
Trading implication: Verify the logic carefully. Use smaller position sizes. Combine with other confirming signals.
Example patterns:
- Insider buying in small-cap energy (tested 65 times)
- New pattern emerging in last 2-3 years (tested 55 times)
20-39: Low Confidence
What this means:
- Pattern tested 30-50 times
- Meets minimum statistical threshold
- Limited historical evidence
- Could be emerging signal or false positive
Trading implication: Be very cautious. This could be a great early signal... or complete noise. Only trade if you deeply understand the underlying logic.
Example patterns:
- New regulatory change creating pattern (tested 35 times)
- Rare event in micro-cap stocks (tested 42 times)
Below 20: Very Low Confidence
What this means:
- Fewer than 30 tests
- Insufficient evidence for reliability
- High chance of being statistical noise
Trading implication: Don't trade these. We filter these out on the dashboard - you shouldn't see them.
Sample Size: The Raw Number
Confidence is derived from sample size (n) - the number of historical tests.
What "n=156" Means
When you see a driver with n=156, it means:
"We found 156 historical instances where this pattern occurred and measured the subsequent outcome."
Example:
insider_buying_20d → forward_return_30d
r=0.74, n=156, p<0.01
Translation: We found 156 cases in history where insiders bought heavily over 20 days. We tracked what the stock did over the following 30 days. In 74% of those 156 cases, the stock rose.
Why Sample Size Matters
Small sample problems:
n=10 - You flip a coin 10 times and get 7 heads. Is the coin biased? Maybe. Or maybe just luck.
n=1000 - You flip 1000 times and get 700 heads. Now you're confident the coin is biased.
Large samples filter out randomness. They give you confidence the pattern is real, not luck.
Minimum Thresholds
We enforce strict minimums:
Dashboard display: Patterns must have n ≥ 30 to appear. Anything below 30 is filtered out automatically.
High quality focus: For best results, focus on n ≥ 100. These are battle-tested patterns.
Gold standard: Look for n ≥ 150. These patterns have been validated extensively.
Confidence vs. Score: The Matrix
Let's map out all combinations:
| Score | Confidence | Assessment | Action |
|---|---|---|---|
| 80+ | 80+ | 🔥 Elite | Highest priority. Maximum conviction. |
| 80+ | 60-79 | ✅ Excellent | Very strong. Trade with confidence. |
| 80+ | 40-59 | ⚠️ Good but verify | High score but moderate testing. Verify logic carefully. |
| 80+ | <40 | 🚫 Trap | Don't trade. High score is likely noise. |
| 70-79 | 80+ | ✅ Excellent | Well-tested moderate strength. Very reliable. |
| 70-79 | 60-79 | ✅ Very Good | Solid setup. Standard trades. |
| 70-79 | 40-59 | ⚠️ Proceed with caution | Verify and use smaller size. |
| 70-79 | <40 | 🚫 Skip | Insufficient evidence. |
| 60-69 | 80+ | ✅ Good | Lower score but very well-tested. Conservative play. |
| 60-69 | 60-79 | ⚠️ Moderate | Decent setup. Combine with other signals. |
| 60-69 | <60 | 🚫 Skip | Too weak, too untested. |
| <60 | Any | 🚫 Skip | Score too low regardless of confidence. |
Key takeaway: You need BOTH high score AND high confidence for best opportunities.
How Confidence is Calculated
Confidence is a normalized metric based on:
1. Raw Sample Size
Base factor: More tests = higher confidence
Formula component: confidence ∝ log(sample_size)
We use logarithmic scaling because the jump from 30 to 60 tests matters more than 200 to 230.
2. Statistical Significance
p-value check: Pattern must be statistically significant (p < 0.05)
Lower p-values boost confidence. p < 0.01 is better than p < 0.05.
3. Pattern Consistency
Variance check: Did the pattern work consistently, or was it sporadic?
If the pattern worked in 74% of cases evenly distributed, that's good. If it worked in 74% of cases but only in bull markets, confidence is adjusted down.
4. Temporal Distribution
Time spread: Are the 156 tests spread across 10 years, or all from 2022?
Patterns validated across multiple years and market conditions get higher confidence.
Example Calculation
Pattern: Insider buying → 30-day forward returns
- Sample size: 156 tests ✅
- Correlation: r=0.74 (strong) ✅
- P-value: p<0.01 (highly significant) ✅
- Consistency: 72-76% across bull/bear markets ✅
- Time spread: 2018-2024 (6 years) ✅
Resulting confidence: 76
This is a well-tested, reliable pattern.
Confidence in Driver Breakdown
When you click into an opportunity and see the drivers, each driver has its own sample size:
Example Opportunity:
Overall Score: 82Overall Confidence: 74
Top Drivers:
- Insider Buying (r=0.74, n=156, contribution=32%)
- Momentum Bullish (r=0.62, n=203, contribution=24%)
- Valuation Gap (r=0.48, n=89, contribution=18%)
How overall confidence is computed:
It's a weighted average of individual driver sample sizes, where weights are the contribution percentages:
Overall confidence ≈
(32% × confidence[Insider Buying]) +
(24% × confidence[Momentum Bullish]) +
(18% × confidence[Valuation Gap]) +
...
Driver 1 has high sample size (156) → high individual confidence Driver 2 has very high sample size (203) → very high individual confidence Driver 3 has moderate sample size (89) → moderate individual confidence
Weighted average: Overall confidence = 74
Real-World Examples
Example 1: The "Obvious" High-Score Trap
TSLA appears with:
- Score: 88 (Wow! Very high!)
- Confidence: 32 (Uh oh...)
What happened:
- TSLA had a rare event (e.g., major contract announcement)
- We've only seen this exact pattern 35 times historically
- 30 of those 35 times led to big moves (86% success rate = high score)
- But with only 35 tests, we can't be confident it's not just luck
Decision: Skip it. The sample size is too small to trust.
Example 2: The Reliable Workhorse
AAPL appears with:
- Score: 71 (Moderate-to-strong)
- Confidence: 87 (Very high)
What happened:
- AAPL shows a common pattern: insider buying + momentum
- We've seen this exact combination 180+ times in large-cap tech
- It works ~71% of the time
- Validated across 10+ years and multiple market cycles
Decision: Strong trade candidate. Lower score than TSLA, but far more reliable.
Example 3: The Emerging Signal
A biotech stock shows:
- Score: 76
- Confidence: 54
What happened:
- New FDA approval pathway created a pattern
- Pattern only exists for 2-3 years (65 historical tests)
- Works well when it appears (76% success)
- But limited long-term validation
Decision: Interesting but risky. If you understand biotech deeply and the pattern makes sense, consider smaller position. Otherwise skip.
Practical Trading Rules
Conservative Approach
Only trade opportunities where:
- Confidence ≥ 70
- Score ≥ 70
Philosophy: I want proven patterns only. Quality over quantity.
Typical hit rate: 5-10 opportunities per day
Moderate Approach
Trade opportunities where:
- (Confidence ≥ 65 AND Score ≥ 70) OR
- (Confidence ≥ 75 AND Score ≥ 60)
Philosophy: Balance quality and quantity. Accept moderate scores if very well-tested, or strong scores if decently tested.
Typical hit rate: 15-25 opportunities per day
Aggressive Approach
Trade opportunities where:
- Confidence ≥ 50
- Score ≥ 60
Philosophy: I'm willing to trade emerging patterns if I understand the logic. I'll manage risk with position sizing.
Typical hit rate: 30-50 opportunities per day
Risk management: Use smaller position sizes on lower confidence setups.
Common Questions
Q: Can confidence change over time?
A: Yes. As we collect more data, confidence can increase (more tests validate the pattern) or decrease (more tests show inconsistency).
Q: Why do some high-score opportunities have low confidence?
A: Rare events or emerging patterns. The pattern works when it appears (high score), but we haven't seen it enough times yet (low confidence).
Q: Should I ever trade low confidence opportunities?
A: Only if:
- You deeply understand the underlying logic
- You're using small position sizes
- You're willing to be wrong and cut losses quickly
Q: What's more important: score or confidence?
A: Both matter, but confidence is often overlooked. Many traders chase high scores and ignore confidence, leading to low-quality trades. Don't make that mistake.
Q: Can I filter the dashboard by confidence?
A: Yes. Use the confidence filter to set a minimum threshold (e.g., only show confidence ≥ 70). This dramatically improves signal quality.
Q: Does confidence predict accuracy?
A: No. Confidence tells you "how well-tested is this pattern," not "how likely it is to work this time." High confidence patterns still fail sometimes. They just fail less often than low confidence patterns.
Next Steps
You now understand confidence and why sample size matters. Continue building your analytical framework:
Understand direction: Direction Indicators - Learn how to interpret bullish, bearish, and neutral signals
Advanced filtering: Filtering Opportunities - Master sorting and filtering by confidence thresholds
Statistical deep dive: Statistical Significance - Learn about p-values and why they matter