Johan Holm
711c2805e4
feat(search): Add robust fallback system with configurable retries and enhanced error handling
- Implement multi-engine failover system with configurable fallback order
- Add retry logic with exponential backoff and rate limit detection
- Introduce search configuration options:
* fallback_engines: Ordered list of backup search providers
* retry_delay: Seconds between retry batches (default: 60)
* max_retries: Maximum system-wide retry attempts (default: 3)
- Improve error resilience with:
- Automatic engine switching on 429/Too Many Requests
- Full system retries after configurable cooldown periods
- Detailed logging for diagnostics and monitoring
- Enhance engine prioritization logic:
1. Primary configured engine
2. Configured fallback engines
3. Remaining available engines
Example configuration:
[search]
engine = "Google"
fallback_engines = ["DuckDuckGo", "Baidu"] # Cascading fallback order
retry_delay = 60 # 1 minute between retry batches
max_retries = 3 # Attempt 3 full system retries
This addresses critical reliability issues by:
- Preventing search failures due to single-engine rate limits
- Enabling recovery from transient network errors
- Providing operational flexibility through configurable parameters
- Improving visibility through granular logging (INFO/WARN/ERROR)