[BUG] PerformanceMonitorInstaller-2.2.0 failed and deleted the existing PerformanceMonitor database

[ghauan]

Component

Installer (GUI)

Performance Monitor Version

2.2.0

SQL Server Version

Microsoft SQL Server 2019 (RTM-CU32) (KB5054833) - 15.0.4430.1 (X64)

Windows Version

Windows server 2016 (on the sql box)

Describe the Bug

Not sure if it counts as a bug, but here is what happend:
Ran the install via the GUI for version 2.2.0. It reported before install that I had version 1.3.0 installed (this was wrong, as I had version 2.1.0 installed. This was due to the previous bug where the version number would not be updated when using the GUI installer.)

Started the install process, and after running for some time, it failed. After this, the entire PerformanceMonitor database had been deleted. (I did not chose clean install)

Steps to Reproduce

Open the GUI installer and run it for a server where a previous install had been done using the GUI installer.

Expected Behavior

Installation success and excisint data compressed.

Actual Behavior

Installation failed, PerformanceMonitor database gone.

Error Messages / Log Output

================================================================================
Performance Monitor Installation Report
================================================================================

INSTALLATION SUMMARY
--------------------------------------------------------------------------------
Status:              FAILED
Start Time:          2026-03-12 20:44:03
End Time:            2026-03-12 20:44:30
Duration:            26,5 seconds
Files Executed:      1
Files Failed:        1

SERVER INFORMATION
--------------------------------------------------------------------------------
Server Name:         SERVERNAME-HERE
SQL Server Edition:  Enterprise Edition (64-bit)

SQL Server Version:
  Microsoft SQL Server 2019 (RTM-CU32) (KB5054833) - 15.0.4430.1 (X64)
  Feb 21 2025 17:28:26
  Copyright (C) 2019 Microsoft Corporation
  Enterprise Edition (64-bit) on Windows Server 2016 Standard 10.0 <X64> (Build 14393: ) (Hypervisor)

INSTALLER INFORMATION
--------------------------------------------------------------------------------
Installer Version:   2.2.0+8ed7a95c6484fda201bfe6f0c56931dc28091a50
Working Directory:   D:\PerformanceMonitor\PerformanceMonitorInstaller-2.2.0
Machine Name:        APP-SERVERNAME
User Name:           my_username

ERRORS
--------------------------------------------------------------------------------
File: 01_install_database.sql
Error: Batch 2 failed:
/*
██████╗ ███████╗██████╗ ███████╗ ██████╗ ██████╗ ███╗   ███╗ █████╗ ███╗   ██╗ ██████╗███████╗
██╔══██╗██╔════╝██╔══██╗██╔════╝██╔═══██╗██╔══██╗████╗ ████║██╔══██╗████╗  ██║██╔════╝██╔════╝
██████╔╝█████╗  ██████╔╝█████╗  ██║   ██║██████╔╝██╔████╔██║███████║██╔██╗ ██║██║     █████╗  
██╔═══╝ ██╔══╝  ██╔══██╗██╔══╝  ██║   ██║██╔══██╗██║╚██╔╝██║██╔══██║██║╚██╗██║██║     ██╔══╝  
██║     ███████╗██║  ██║██║     ╚██████╔╝██║  ██║██║ ╚═╝ ██║██║  ██║██║ ╚████║╚██████╗███████╗
...

DETAILED INSTALLATION LOG
--------------------------------------------------------------------------------
Performance Monitor Installer v2.2.0+8ed7a95c6484fda201bfe6f0c56931dc28091a50
Copyright (c) 2026 Darling Data, LLC
Found 3 upgrade(s) to apply
Applying upgrade: 1.3.0-to-2.0.0
[OK]   01_memory_grant_stats_schema.sql - Success
[OK]   02_session_wait_stats_cleanup.sql - Success
[OK]   03_default_trace_schema.sql - Success
[OK] Upgrade 1.3.0-to-2.0.0: 3 succeeded, 0 failed
Applying upgrade: 2.0.0-to-2.1.0
[OK]   01_collection_schedule_optional_text.sql - Success
[OK]   02_default_trace_events_new_columns.sql - Success
[OK] Upgrade 2.0.0-to-2.1.0: 2 succeeded, 0 failed
Applying upgrade: 2.1.0-to-2.2.0
[ERROR]   01_compress_query_stats.sql - FAILED: Batch 6 failed:
BEGIN TRY

    /*
    Step 1: Create the _new table with compressed column types
    */
    IF OBJECT_ID(N'collect.query_stats_new', N'U') IS NOT NULL
    BEGIN
        DROP TABLE collect.query_stats_new;
        PRINT 'Dropped existing collect.query_stats_new';
    END;

    CREATE TABLE
        collect.query_stats_new
    (
        collection_id bigint IDENTITY NOT NULL,
        collection_time datetime2(7) NOT NULL
            DEFAULT SYSDATETIME(),
        server_start_time ... [truncated, total length: 11086]

Original error: Execution Timeout Expired.  The timeout period elapsed prior to completion of the operation or the server is not responding.
Operation cancelled by user.
Created collect.query_stats_new
Checking identity information: current identity value 'NULL'.
DBCC execution completed. If DBCC printed error messages, contact your system administrator.
Reseeded IDENTITY to 2656731
 
Migrating data in batches of 10000...
  Migrated 10000 rows so far...
  Migrated 20000 rows so far...
  Migrated 30000 rows so far...
  Migrated 40000 rows so far...
  Migrated 50000 rows so far...
  Migrated 60000 rows so far...
  Migrated 70000 rows so far...
  Migrated 80000 rows so far...
  Migrated 90000 rows so far...
  Migrated 100000 rows so far...
  Migrated 110000 rows so far...
  Migrated 120000 rows so far...
The statement has been terminated.
[OK]   02_compress_query_store_data.sql - Success
[OK]   03_compress_procedure_stats.sql - Success
[OK]   04_create_tracking_tables.sql - Success
[OK]   05_add_finops_collectors.sql - Success
[WARN] Upgrade 2.1.0-to-2.2.0: 4 succeeded, 1 failed
Starting installation...
Executing 00_uninstall.sql...
[OK] 00_uninstall.sql - Success
Executing 01_install_database.sql...
[ERROR] 01_install_database.sql - FAILED: Batch 2 failed:
/*
██████╗ ███████╗██████╗ ███████╗ ██████╗ ██████╗ ███╗   ███╗ █████╗ ███╗   ██╗ ██████╗███████╗
██╔══██╗██╔════╝██╔══██╗██╔════╝██╔═══██╗██╔══██╗████╗ ████║██╔══██╗████╗  ██║██╔════╝██╔════╝
██████╔╝█████╗  ██████╔╝█████╗  ██║   ██║██████╔╝██╔████╔██║███████║██╔██╗ ██║██║     █████╗  
██╔═══╝ ██╔══╝  ██╔══██╗██╔══╝  ██║   ██║██╔══██╗██║╚██╔╝██║██╔══██║██║╚██╗██║██║     ██╔══╝  
██║     ███████╗██║  ██║██║     ╚██████╔╝██║  ██║██║ ╚═╝ ██║██║  ██║██║ ╚████║╚██████╗███████╗
╚═╝     ╚══════╝... [truncated, total length: 3612]

Original error: Could not obtain exclusive lock on database 'model'. Retry the operation later.
CREATE DATABASE failed. Some file names listed could not be created. Check related errors.
User does not have permission to alter database 'PerformanceMonitor', the database does not exist, or the database is not in a state that allows access checks.
ALTER DATABASE statement failed.
Created PerformanceMonitor database at d:\data\PerformanceMonitor.mdf
[ERROR] Critical installation file failed. Aborting installation.

================================================================================
Generated by Performance Monitor Installer GUI
Copyright (c) 2026 Darling Data, LLC
================================================================================

Screenshots

Additional Context

The existing PerformanceMonitor database was pretty big (about 240 GB).
I tried this install on a testserver with a very small PerformanceMonitor database before this, and that installed normally.

[ghauan]

Some additional info: I tried installing on another server with a big PerformanceMonitor database (156 GB). This install was successful, but it seems that all data was deleted during installation (see disk usage before and after install). I think I read in the version info that it would not delete existing data?

[erikdarlingdata]

Well, the first issue you posted is a proper interesting one. I ran through every install scenario I could on the upgrade path and didn't run into anything close to that.

For the comment you just left, check row counts in the tables. The compression effect is profound.

[ghauan]

Tried another install using the non GUI. This time on an even bigger PerformanceMonitor database (these are datawarehouse servers with a lot going on). Got a similar error message during install, but the PerformanceMonitor database was not deleted this time.

It is not critical to me to have lost these data, just thought you might want to know the situation.

From the install-window:

Testing connection...
Connection successful!

Found 55 SQL files in: D:\PerformanceMonitor\PerformanceMonitorInstaller-2.2.0\install
Using new folder structure (install/ subfolder)

Drop existing PerformanceMonitor database if it exists? (Y/N, default N): N

Existing installation detected: v1.3.0.0
Checking for applicable upgrades...
Found 3 upgrade(s) to apply.

================================================================================
Applying upgrades...

Applying upgrade: 1.3.0-to-2.0.0

01_memory_grant_stats_schema.sql... V Success
02_session_wait_stats_cleanup.sql... V Success
03_default_trace_schema.sql... V Success

Applying upgrade: 2.0.0-to-2.1.0

01_collection_schedule_optional_text.sql... V Success
02_default_trace_events_new_columns.sql... V Success

Applying upgrade: 2.1.0-to-2.2.0

01_compress_query_stats.sql... ? FAILED
Error: Batch 6 failed:
BEGIN TRY

/*
Step 1: Create the _new table with compressed column types
*/
IF OBJECT_ID(N'collect.query_stats_new', N'U') IS NOT NULL
BEGIN
    DROP TABLE collect.query_stats_new;
    PRINT 'Dropped existing collect.query_stats_new';
END;

CREATE TABLE
    collect.query_stats_new
(
    collection_id bigint IDENTITY NOT NULL,
    collection_time datetime2(7) NOT NULL
        DEFAULT SYSDATETIME(),
    server_start_time ... [truncated, total length: 11086]

Original error: Execution Timeout Expired. The timeout period elapsed prior to completion of the operation or the server is not responding.
Operation cancelled by user.
Created collect.query_stats_new
Checking identity information: current identity value 'NULL'.
DBCC execution completed. If DBCC printed error messages, contact your system administrator.
Reseeded IDENTITY to 1601802

Migrating data in batches of 10000...
Migrated 10000 rows so far...
Migrated 20000 rows so far...
Migrated 30000 rows so far...
Migrated 40000 rows so far...
Migrated 50000 rows so far...
Migrated 60000 rows so far...
Migrated 70000 rows so far...
The statement has been terminated.
02_compress_query_store_data.sql... V Success
03_compress_procedure_stats.sql... V Success
04_create_tracking_tables.sql... V Success
05_add_finops_collectors.sql... V Success

Upgrades complete: 9 succeeded, 1 failed

================================================================================
Starting installation...

Executing 00_uninstall.sql... V Success
Executing 01_install_database.sql... V Success
Executing 02_create_tables.sql... V Success
Executing 03_create_config_tables.sql... V Success
Executing 04_create_schedule_table.sql... V Success
Executing 05_delta_framework.sql... V Success
Executing 06_ensure_collection_table.sql... V Success
Executing 07_collect_wait_stats.sql... V Success
Executing 08_collect_query_stats.sql... V Success
Executing 09_collect_query_store.sql... V Success
Executing 10_collect_procedure_stats.sql... V Success
Executing 11_collect_query_snapshots.sql... V Success
Executing 12_query_snapshots_create_views.sql... V Success
Executing 13_query_snapshots_retention.sql... V Success
Executing 14_collect_memory_stats.sql... V Success
Executing 15_collect_memory_grant_stats.sql... V Success
Executing 16_collect_memory_clerks_stats.sql... V Success
Executing 17_collect_cpu_scheduler_stats.sql... V Success
Executing 18_collect_cpu_utilization_stats.sql... V Success
Executing 19_collect_perfmon_stats.sql... V Success
Executing 20_collect_file_io_stats.sql... V Success
Executing 21_setup_blocked_process_xe.sql... V Success
Executing 22_collect_blocked_processes.sql... V Success
Executing 23_process_blocked_process_xml.sql... V Success
Executing 24_collect_deadlock_xml.sql... V Success
Executing 25_process_deadlock_xml.sql... V Success
Executing 26_blocking_deadlock_analyzer.sql... V Success
Executing 27_collect_ring_buffer_events.sql... V Success
Executing 28_collect_system_health_wrapper.sql... V Success
Executing 29_collect_default_trace.sql... V Success
Executing 30_collect_trace_management.sql... V Success
Executing 31_collect_trace_analysis.sql... V Success
Executing 32_collect_latch_stats.sql... V Success
Executing 33_collect_spinlock_stats.sql... V Success
Executing 34_collect_tempdb_stats.sql... V Success
Executing 35_collect_plan_cache_stats.sql... V Success
Executing 36_collect_session_stats.sql... V Success
Executing 37_collect_waiting_tasks.sql... V Success
Executing 38_collect_server_configuration.sql... V Success
Executing 39_collect_database_configuration.sql... V Success
Executing 41_schedule_management.sql... V Success
Executing 42_scheduled_master_collector.sql... V Success
Executing 43_data_retention.sql... V Success
Executing 44_hung_job_monitor.sql... V Success
Executing 45_create_agent_jobs.sql... V Success
Executing 46_create_query_plan_views.sql... V Success
Executing 47_create_reporting_views.sql... V Success
Executing 48_create_additional_reporting_views.sql... V Success
Executing 49_health_parser_tables.sql... V Success
Executing 50_configuration_issues_analyzer.sql... V Success
Executing 51_collect_running_jobs.sql... V Success
Executing 52_collect_database_size_stats.sql... V Success
Executing 53_collect_server_properties.sql... V Success
Executing 54_create_finops_views.sql... V Success

================================================================================
Installing community dependencies...

Installing sp_WhoIsActive... V Success
Query activity monitoring by Adam Machanic (GPLv3)
Installing DarlingData... V Success
sp_HealthParser, sp_HumanEventsBlockViewer, and others by Erik Darling (MIT)
Installing First Responder Kit... V Success
sp_BlitzLock and other diagnostic tools by Brent Ozar Unlimited (MIT)

Dependencies installed: 3/3
Executing 98_validate_installation.sql... V Success

================================================================================
File Execution Summary

Upgrades: 9 succeeded, 1 failed
Installation: 55 succeeded, 0 failed

================================================================================
Installation Summary

Installation completed with 1 error(s).
Review errors above and check PerformanceMonitor.config.collection_log for details.

[ghauan]

Regarding the comment about deletion/compression. I have a before and after of the top tables, and it seems to have deleted a lot of records:

[erikdarlingdata]

Root Cause Analysis

I owe you a sincere apology — this is a serious bug that should never have shipped, and I'm sorry you lost data because of it. That's unacceptable, and I take full responsibility for the gap in our pre-release testing that let this through.

Here's what happened:

The primary cause: 00_uninstall.sql runs on every install

In v2.2.0, we added a dedicated uninstall feature (PR #431). The uninstall script was placed at install/00_uninstall.sql so SSMS users could run it standalone, and so the Installers could perform the process. The problem: both the CLI and GUI installers build their file list by globbing install/*.sql sorted alphabetically — so 00_uninstall.sql became the first file executed on every install, including upgrades. It unconditionally drops the PerformanceMonitor database.

This means every 2.2.0 install (upgrade or fresh) was silently dropping the database before recreating it empty. On a clean install with no prior data, this is invisible. On an upgrade with months of collected data, it's catastrophic.

The secondary cause: upgrade timeout + no abort

The 01_compress_query_stats.sql upgrade script migrates data in batches of 10,000 rows with a 5-minute command timeout. On large databases (your 240GB case), this timeout is far too short. When it times out, the installer logs the failure but continues to the full install path — which hits 00_uninstall.sql and drops the database.

The version detection issue (#539 duplicate)

The installation_history table was empty on some servers (likely from a prior GUI installer bug that didn't write history rows). When the 2.2.0 installer can't find a version, it treats it as a fresh install, skips all upgrades, and goes straight to the install path — which again starts with 00_uninstall.sql.

Fixes in progress

1. Both installers now exclude 00_ files from the install file list (same as 97_ test and 99_ troubleshooting scripts are already excluded)
2. Abort installation when upgrade scripts fail instead of continuing to the full install
3. Fall back to database/table existence when installation_history is empty, so we don't treat an existing database as a fresh install
4. Increase timeout for upgrade scripts that do large data migrations

Pre-release process improvements

Our testing covered fresh installs and upgrades on small test databases, but never verified that existing data survives the upgrade (row count comparison before/after). We've added:

• Data survival verification as a mandatory step for every upgrade test
• Multi-hop upgrade testing (install 3 most recent releases in sequence)
• installation_history verification after every install
• Uninstall path testing

Again, I'm truly sorry about the data loss. This was a preventable bug and we're making sure it can't happen again. I'll update this issue when the fix PR is ready.

[ghauan]

No worries. I have backups :).
Just glad I can give som feedback to this awsome tool. Thanks for the work you and the other contributors do.

[erikdarlingdata]

wow, you actually backed up a monitoring database!

would you mind trying the installer fixes from the nightly build and seeing if you still hit the timeout errors, etc.? i don't have a large enough data set to test against.

[ghauan]

I can try it, but probably won't be able to until Sunday. Preparing for migration to a new staging-server tomorrow.

[holidasa]

Hi Erik, I had the same issue with the timeout and database being dropped (2.1.0 to 2.2.0 upgrade) on six servers. I did not have as large of a database (only 8-12 GB), but I can restore the PerformanceMonitor database and try the nightly build on one of them, if that helps.