Orphan VMs and volumes remain after Shared File System removal (cannot delete via UI)

[joan-911]

After removing a Shared File System (SFS) in CloudStack, the associated backend VM instances and volumes were not removed. The Shared File System entry itself has been deleted, but the related VMs and volumes remain in the database.

These orphan resources cannot be removed through the CloudStack UI or API

Please advise the recommended safe cleanup procedure, via Database

[DaanHoogland]

@joan-911 ,

You say , "the associated backend VM instances and volumes were not removed” , but then , "but the related VMs and volumes remain in the database” , which makes me wonder if the actual files remain on disk. Can you check?

[joan-911]

Yes. The Shared filesystem VM, and its associated disks were not deleted on removing the shared file system. Since cloudstack is not allowing me to remove the VM and volume from UI/API, those need to me marked as removed in DB and at hypervisor I need to delete them directly

[DaanHoogland]

ok, that sounds like a bug :(

[abh1sar]

@joan-911 you have to select the expunge option when deleting the Shared Filesystem.
If it is not expunged, it will still show up as destroyed. You can expunge it then.

[joan-911]

Yes, it was expunged and the shared filesystem has been removed from the UI. However, the backend disks and the instance still remain in a stopped state.

Can I mark them as expunged directly in the database? Is there a safe way to update the corresponding DB entries?

[abh1sar]

Which ACS version are you on? Can you share steps to recreate the issue and some screenshots? I am not able to recreate the issue on my env in 4.20.

[joan-911]

My cloudstack is v 4.21 and Xcp-ng
I tried deleting the newly created SFS, and the associated instances and volumes were cleaned up successfully. However, this particular VM under concern has not been expunged, and I am unable to remove the instance from the UI.

[abh1sar]

You can mark the VM and the volume as 'Destroyed' in the database. Storage cleanup thread will expunge those when it runs.
Another option is that you set the user_vm_type in the user_vm table as null and then expunge the instance.

Any idea how this particular VM got into this state?

[joan-911]

Thanks for the update. This helps.
Not sure, I tried to recreate the issue, all dependent resources got removed now :)