Run a reduced results analysis periodically during backtests

[jhonabreul]

Description

Runs a reduced suite of the backtest results analyses periodically while the backtest is still running, so error conditions (order response errors, margin calls, stale fills, non-positive equity, high margin usage) surface to the user early instead of only after the backtest completes.

• ResultsAnalyzer makes its analysis set overridable (GetAnalyses) and lets subclasses skip building the equity/benchmark curves (RequiresEquityCurves), avoiding a benchmark history request on every run. The analyses are stateless, so they are created once and reused across runs.
• New InRunResultsAnalyzer runs only cheap, snapshot-based analyses. It is kept alive for the duration of the backtest and runs incrementally: each run analyzes only the order events and log lines produced since the previous run, and merges the new findings into the accumulated ones (stream-based findings keep the first sample and total the counts; state-based findings are replaced on every run).
• BacktestingResultHandler invokes it on the periodic intermediate result updates against a thread-safe snapshot of the current backtest state, with a small time budget (1s) since it runs on the result handler thread. Only the charts the in-run analyses read (InRunResultsAnalyzer.RequiredCharts, currently the portfolio margin chart) are cloned into the snapshot.
• The statistics are withheld from the analysis until the algorithm is done warming up and the first equity sample exists: before that they are all-zero defaults that would flag a false non-positive portfolio value finding. PortfolioValueIsNotPositiveAnalysis skips when they are withheld.
• Findings are sent to the browser in their own result packet, only when they changed since last sent.
• When the time limit truncates a run, positions still advance: analyses that didn't run miss that delta until the final analysis re-scans the complete streams. State-based findings similarly drop if a truncated run skips their analysis, reappearing once a later run reaches it again. This is accepted behavior (stress runs complete in a fraction of the time limit) and is documented in the code.
• New in-run-only AlgorithmSpeedAnalysis tracks the algorithm's speed so the user can decide to stop a slow backtest early:
  • BacktestingResultHandler feeds an AlgorithmSpeedSample of the engine counters (data points from the PerformanceTrackingTool, history data points, processed/total days, wall-clock elapsed) into the analyzer on each run; AlgorithmSpeedTracker accumulates the samples and computes cumulative and recent-window rates, calendar-days-per-second, projected remaining time, and history-request share.
  • Sub-findings:
    • SlowExecution (recent pace below the 40k data points/s benchmark, with progress and projected remaining time).
    • LongProjectedRuntime (over an hour left at the recent pace, or stalled calendar progress).
    • ThroughputDegradation (recent pace below half the early-run baseline).
    • HistoryRequestLoad (most processed data points served by history requests).
  • Guardrails: a one-minute minimum sampled span, and every condition must hold for two consecutive windows, so warm-up noise or a single slow cycle doesn't flag (or clear) a finding.
• The speed counters are not sampled while the algorithm warms up, so the warm-up pace doesn't skew the speed metrics. The in-run analyses themselves do run during warm-up, so conditions like orders submitted while the algorithm is warming up surface immediately instead of waiting for warm-up to end.

Related Issue

N/A

Motivation and Context

Users currently only get the results analysis findings when a backtest finishes. For long backtests, surfacing failures like margin calls or persistent order errors while the run is in progress lets the user stop it early instead of waiting for completion.

Requires Documentation Change

N/A

How Has This Been Tested?

• Ran backtests through the Launcher and verified the in-run analysis findings are produced on the intermediate updates and delivered to the messaging handler in their own result packet.

• Stress-tested the in-run analyzer with an instrumented build (local-only timing instrumentation, not part of this PR) logging each cycle's elapsed time and delta sizes — worst observed cycle: 365 ms against the 1 s budget.

  Stress test setup and per-cycle numbers

  • Baseline run — 4 minute-resolution symbols, ~30 orders per time step: a cycle processing a delta of 31,489 order events + 764 log lines against 14,818 accumulated orders took 85 ms.

  • Heavy run — 133 minute-resolution symbols over a month, ~75 orders per time step (~30k orders/day) with fills, cancellations, rejections, and a log line per bar:

    Cycle	Order-event delta	Accumulated orders	Elapsed
    1	65,035	29,589	133 ms
    2	194,935	118,282	252 ms
    3	194,086	206,588	297 ms
    4	194,262	294,978	311 ms
    5	124,329	351,544	365 ms
    6	64,135	380,803	289 ms

  • Worst observed cycle: 365 ms against the 1 s budget, on a workload an order of magnitude beyond a realistic algorithm. Elapsed time is delta-dominated (~190k events ≈ 250–310 ms) with a mild ~+60 ms per 100k accumulated orders from the state-based analyses; extrapolating to ~600k accumulated orders with a maximal delta lands around 500–600 ms. The analyzer's time-limit trace never appeared in the logs.

• Unit tests (44, all passing):

  • InRunResultsAnalyzerTests — incremental position advancement (including on time-limit-truncated runs), stream-based finding accumulation (first sample kept, counts totaled), state-based finding replacement and dropping, weight ranking, the max-findings cap, speed samples being tracked only when provided (none are taken during warm-up), the RequiredCharts contract, and the analysis set being created once and reused across runs.
  • ResultsAnalyzerTests — overridable analysis set, skipped equity-curve construction, weight-ordered execution, the time-limit and max-failures early exits, and analysis instance reuse across runs.
  • PortfolioValueIsNotPositiveAnalysisTests — no findings when the statistics are withheld (warm-up / no equity samples yet), the non-positive ending equity flag, and no actionable finding on positive equity.
  • AlgorithmSpeedTrackerTests — cumulative/windowed rate math, history data point share, remaining-time estimation, and sample validation.
  • AlgorithmSpeedAnalysisTests — each sub-finding's threshold, the warm-up grace span, the two-consecutive-window hysteresis, and the suppression of data-point findings when the counters aren't wired in.

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• Refactor (non-breaking change which improves implementation)
• Performance (non-breaking change which improves performance. Please add associated performance test and results)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Non-functional change (xml comments/documentation/etc)

Checklist:

• My code follows the code style of this project.
• I have read the CONTRIBUTING document.
• I have added tests to cover my changes.
• All new and existing tests passed.
• My branch follows the naming convention bug-<issue#>-<description> or feature-<issue#>-<description>