Troubleshooting

Troubleshooting

This page captures recurring issues and the quickest path to resolution. If you hit a new failure mode, add it here to save future operators time.

Common failures

• Curated dataset missing for <SYMBOL> (report build or risk step)
  • Cause: raw download skipped the symbol (network failure, typo in universe, data provider outage).
  • Fix: rerun ts run ... --force after confirming connectivity; ensure the symbol exists on Yahoo; shrink the universe or lookback while debugging.
• Curated directory already populated: ... Use --force to overwrite.
  • Cause: rerunning with the same as-of date without clearing outputs.
  • Fix: remove the dated directory under data/curated/<date> and reports/<date> or pass --force when you intentionally overwrite.
• Configuration missing 'paths' mapping.
  • Cause: YAML missing the paths block or indentation issue.
  • Fix: copy the block from Configuration and verify spacing with yamllint.
• Symbol XYZ unavailable from Yahoo; skipping.
  • Cause: provider returned an error (timeout, delisted ticker, corporate action window).
  • Fix: retry; if persistent, switch to a supported symbol or use a paid Point-in-Time data source.
• Slack webhook returned unexpected status: 404 (or similar) when notifications run without --dry-run.
  • Cause: placeholder webhook (https://hooks.slack.com/services/XXXXX/...) or revoked credential.
  • Fix: configure a valid webhook secret or append --dry-run until credentials are ready.
• Email recipient not configured in notify.email
  • Cause: notify.email missing from config but the pipeline requested email delivery.
  • Fix: populate the field or set --channel slack/--channel all with the appropriate secrets.
• Raw data for SYMBOL missing required columns
  • Cause: vendor schema changed or partial download.
  • Fix: inspect the raw parquet in data/raw/<date>; update the adapter or retry after the vendor recovers.
• Time zone offsets (dates off by one)
  • Cause: running before the latest close is available or using non-US calendars.
  • Fix: check meta_run.json last_bar_date; if it lags, rerun later or adjust the provider start/end.

Debug workflow

1. Re-run the failing command with --log-to-file so you can grep the JSON logs.
2. Inspect reports/<date>/run.log for the failing step – look for step_failed entries.
3. Validate the manifest: poetry run ts observability manifest --run reports/<date>.
4. Use targeted commands (ts risk evaluate, ts signals build, ts report build) to isolate the failing stage without re-running the world.

When to open an issue

Open an issue on the repo when:

• Failures persist after retries and the fix is not documented here.
• Provider schemas change, requiring adapter updates.
• You discover a reproducible bug with steps to trigger it.

Template:

### Summary
<one sentence failure description>

### Steps to Reproduce
1. ...
2. ...

### Expected vs Actual
- Expected: ...
- Actual: ...

### Artifacts / Logs
- Attach `reports/<date>/manifest.json`
- Attach relevant excerpts from `run.log`

### Environment
- trading-system commit hash
- Python version (`poetry run python --version`)
- OS / scheduler

For conceptual support or workflow questions, check FAQ before escalating.