Skip to content

Add adversarial testing for Dashboard and Installer #543

@erikdarlingdata

Description

@erikdarlingdata

Summary

The Lite analysis engine now has 24 adversarial/misery-path tests (FactCollectorMiseryTests) that probe division by zero, empty tables, reversed time ranges, DMV corruption, overflow, boundary conditions, and full-pipeline resilience. This uncovered a real scoring bug (PLE=0 scored as harmless).

The Dashboard and Installer projects would benefit from the same treatment — tests designed to break things, not just confirm happy paths.

Areas to cover

Dashboard

  • Empty/null data from SQL Server (no rows in PerformanceMonitor tables)
  • Connection failures mid-refresh (network drop, timeout)
  • Corrupt or truncated query plan XML
  • Charts with zero data points, single data points, or extreme outliers
  • Very large result sets (memory pressure on the UI thread)
  • Missing or renamed columns after a schema upgrade

Installer (CLI + GUI)

  • Upgrade from a database with missing tables or columns (partial prior install)
  • Upgrade scripts that encounter locked tables or active connections
  • Fresh install on a server with restricted permissions
  • Upgrade folder with out-of-order or duplicate scripts
  • Database already at target version (idempotency)
  • Installer run against an incompatible SQL Server version
  • Disk full or path not found during file copy
  • Interrupted upgrade (power failure simulation — partial script execution)

Context

This came out of the ErikAI work on feature/erikai where misery-path testing caught a real bug in the scorer. The same discipline should be applied more broadly.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions