Clone a Virtual Machine by atrocitytheme · Pull Request #5216 · apache/cloudstack

atrocitytheme · 2021-07-16T17:31:41Z

Description/Report

This PR adds a Clone VM feature at the API level (as part of GSoC project #4818), which enables the creation of a fully-cloned virtual machine with ROOT / DATA disks, and the same system configuration as the original VM. (Currently Supported for KVM hypervisor only)

Steps involved:

Creation of temporary snapshots (during the clone VM operation) for both ROOT
Creation of template from the ROOT disk snapshot
Create a new VM from the template created in step 2
Automatic assignment of new network resources for the new Cloned VM
Creation of temporary snapshots for data disk
Create data disk Volume from the snapshots created in step 5
Newly created DATA disk created in step 6 is attached to the new VM
Cleanup of temporary resources (snapshots) and error handling of the clone VM process

Note: The template created in step 2 cannot be cleaned up as the newly created clone VM uses this template

Feature included:

A new CloneVmCmd API interface to use this feature
A clone button in the compute instance page supporting the related functionalities

Documentation

Documentation PR: Clone VM documentation cloudstack-documentation#236

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Major
Minor

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

This has been manually tested with a mbx KVM setup on a local machine and mbx kvm setup on GCP

This feature has been tested on a local Linux system with KVM support (5.3.0-64-generic Ubuntu) and Google Cloud instance (4.9.0-15-amd64 Debian)
Try with cloudmonkey if the management server is running with default configurations.

cloneVirtualMachine virtualmachineid=<target_vm_id>

will start the cloning process, it'll create a new cloned VM and start it (with all copied data available), network Ip will be assigned to DB instantly and the actual VM will get this IP after a while

When secondary system VM agents are not available, the cloning process will fail and clean the previously cloned resources
It'll copy all the data disk content from the target VM no matter whether the data disks are mounted in the VM system or the VM is running
Temporary resources created (snapshots) during the process will not remain whether the clone succeeds or not
New smoke test has been added in test_vm_life_cycle.py with class TestCloneVM, which tests the clone of a VM with data disk attached.
New unit test has been added as validateCloneCondition in the UserVmManagerImpl class
Manual test of Clone: in the VM page, there's a clone VM button, click it and it'll clone a new VM if with the correct hypervisor setup.

GCP environment setup

Create a GCP instance with nested virtualization enabled, for detailed requirements see: https://cloud.google.com/compute/docs/instances/nested-virtualization/overview
After creating the instance, deploy a KVM host using mbx: https://github.com/shapeblue/mbx (centos7)
Before deploying and configuring the cloudstack agent, enable the system nested kvm by:

cat /sys/module/kvm_intel/parameters/nested
// if return N
sudo modprobe -r kvm_intel
sudo modprobe kvm_intel nested=1

…to full-clone

sureshanaparti · 2022-07-28T13:24:04Z

@blueorangutan package

blueorangutan · 2022-07-28T13:25:02Z

@sureshanaparti a Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

acs-robot · 2022-07-28T13:25:10Z

Found UI changes, kicking a new UI QA build
@blueorangutan ui

blueorangutan · 2022-07-28T13:26:02Z

@acs-robot a Jenkins job has been kicked to build UI QA env. I'll keep you posted as I make progress.

blueorangutan · 2022-07-28T13:28:32Z

Packaging result: ✖️ el7 ✖️ el8 ✖️ debian ✖️ suse15. SL-JID 3847

blueorangutan · 2022-07-28T13:32:15Z

UI build: ✔️
Live QA URL: http://qa.cloudstack.cloud:8080/client/pr/5216 (SL-JID-2022)

github-actions · 2022-08-02T12:37:03Z

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

DaanHoogland · 2022-08-03T13:34:53Z

@atrocitytheme can you resolve the conflict?

JoaoJandre · 2022-10-17T16:41:27Z

api/src/main/java/org/apache/cloudstack/api/command/user/vm/CloneVMCmd.java

+
+import java.util.Optional;
+
+@APICommand(name = "cloneVirtualMachine", responseObject = UserVmResponse.class, description = "clone a virtual VM",


Suggested change

@APICommand(name = "cloneVirtualMachine", responseObject = UserVmResponse.class, description = "clone a virtual VM",

@APICommand(name = "cloneVirtualMachine", responseObject = UserVmResponse.class, description = "clone a virtual machine",

or

Suggested change

@APICommand(name = "cloneVirtualMachine", responseObject = UserVmResponse.class, description = "clone a virtual VM",

@APICommand(name = "cloneVirtualMachine", responseObject = UserVmResponse.class, description = "clone a VM",

JoaoJandre · 2022-10-17T16:54:56Z

server/src/main/java/com/cloud/template/TemplateManagerImpl.java

+        VMTemplateVO finalTmpProduct = null;
+        SnapshotVO snapshot = null;
+        try {
+            TemplateInfo cloneTempalateInfp = _tmplFactory.getTemplate(templateId, DataStoreRole.Image);


Suggested change

TemplateInfo cloneTempalateInfp = _tmplFactory.getTemplate(templateId, DataStoreRole.Image);

TemplateInfo cloneTemplateInfo = _tmplFactory.getTemplate(templateId, DataStoreRole.Image);

JoaoJandre · 2022-10-17T16:55:23Z

server/src/main/java/com/cloud/template/TemplateManagerImpl.java

+                    store = snapStore; // pick snapshot image store to create template
+                }
+            }
+            future = _tmpltSvr.createTemplateFromSnapshotAsync(snapInfo, cloneTempalateInfp, store);


Suggested change

future = _tmpltSvr.createTemplateFromSnapshotAsync(snapInfo, cloneTempalateInfp, store);

future = _tmpltSvr.createTemplateFromSnapshotAsync(snapInfo, cloneTemplateInfo, store);

JoaoJandre · 2022-10-17T17:01:18Z

server/src/main/java/com/cloud/template/TemplateManagerImpl.java

+            } catch (InterruptedException e) {
+                s_logger.debug("Failed to create template for id: " + templateId, e);
+                throw new CloudRuntimeException("Failed to create template" , e);
+            } catch (ExecutionException e) {
+                s_logger.debug("Failed to create template for id: " + templateId, e);
+                throw new CloudRuntimeException("Failed to create template ", e);
+            }


Suggested change

} catch (InterruptedException e) {

s_logger.debug("Failed to create template for id: " + templateId, e);

throw new CloudRuntimeException("Failed to create template" , e);

} catch (ExecutionException e) {

s_logger.debug("Failed to create template for id: " + templateId, e);

throw new CloudRuntimeException("Failed to create template ", e);

}

} catch (InterruptedException | ExecutionException e) {

s_logger.debug("Failed to create template for id: " + templateId, e);

throw new CloudRuntimeException("Failed to create template" , e);

}

DaanHoogland · 2022-12-13T09:06:44Z

@atrocitytheme are you taking care of this? can you resolve the conflicts?

DaanHoogland · 2023-01-12T14:06:50Z

@atrocitytheme are you still having this in your scope? shall I move it to 4.19?

github-actions · 2023-01-27T10:24:40Z

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

shwstppr · 2023-05-09T11:08:26Z

Discussed it with GSoC mentors at the time. Considering the outstanding changes and manual testing, closing this for now. We will revisit this at a later stage. cc @rohityadavcloud @borisstoyanov @DaanHoogland
Feel free to add anything @atrocitytheme

cron-ns added 30 commits June 7, 2021 02:58

API command initialization

d3b8e74

first step to create the clone api execution

e2f7228

Merge branch 'main' of https://github.com/atrocitytheme/cloudstack in…

6a2237c

…to full-clone

add findISO and findVO steps

6b2604d

add creation of root-disk -> template -> vm step

8c815ab

prepare the template impl

fb6cb0a

clear the dot record for later test

77c39c2

checks for VM are implemented

1971a39

fix checkstyle

3687110

compilation fixed

fbb886c

checkClone debug

16d75da

server state update

779681c

add and statement

5a81d2b

fix null error

6f4fca9

test hit the current vm information

896b033

fix typo of star import

cfd131c

fix the null pt except

9677858

add vm record creation code

85129ee

add the user vm db creation code

b6607f4

add vm creation typo fix

afae560

change ip to available

0a1405d

fix ip issue

64c0fd5

fix the vm creation detail

e04dc22

template zone addition code

7bef87f

trail-issue-1

d15f620

change test ip address

fee4a95

add template download record

eecacc0

get the template created and prepare for the vm start

31824f2

fix unknown symbol

e3bf894

finish the final private creation

99e5043

boring-cyborg bot added component:networking component:storage component:UI component:usage-server Python Warning... Python code Ahead! labels Jul 28, 2022

github-actions bot removed the status:has-conflicts label Jul 28, 2022

github-actions bot added the status:has-conflicts label Aug 2, 2022

JoaoJandre reviewed Oct 17, 2022

View reviewed changes

Merge branch 'main' into full-clone

733b903

github-actions bot removed the status:has-conflicts label Jan 9, 2023

DaanHoogland changed the title ~~Support for new Feature: Clone a Virtual Machine (#4818)~~ Clone a Virtual Machine Jan 12, 2023

DaanHoogland closed this Jan 12, 2023

DaanHoogland reopened this Jan 12, 2023

DaanHoogland modified the milestones: 4.18.0.0, 4.19.0.0 Jan 16, 2023

github-actions bot added the status:has-conflicts label Jan 27, 2023

shwstppr closed this May 9, 2023

DaanHoogland removed the status:needs-testing label Jan 28, 2026


		import java.util.Optional;

		@APICommand(name = "cloneVirtualMachine", responseObject = UserVmResponse.class, description = "clone a virtual VM",

	TemplateInfo cloneTempalateInfp = _tmplFactory.getTemplate(templateId, DataStoreRole.Image);
	TemplateInfo cloneTemplateInfo = _tmplFactory.getTemplate(templateId, DataStoreRole.Image);

	future = _tmpltSvr.createTemplateFromSnapshotAsync(snapInfo, cloneTempalateInfp, store);
	future = _tmpltSvr.createTemplateFromSnapshotAsync(snapInfo, cloneTemplateInfo, store);

Conversation

atrocitytheme commented Jul 16, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description/Report

Documentation

Types of changes

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

GCP environment setup

Uh oh!

sureshanaparti commented Jul 28, 2022

Uh oh!

blueorangutan commented Jul 28, 2022

Uh oh!

acs-robot commented Jul 28, 2022

Uh oh!

blueorangutan commented Jul 28, 2022

Uh oh!

blueorangutan commented Jul 28, 2022

Uh oh!

blueorangutan commented Jul 28, 2022

Uh oh!

github-actions bot commented Aug 2, 2022

Uh oh!

DaanHoogland commented Aug 3, 2022

Uh oh!

JoaoJandre Oct 17, 2022

Choose a reason for hiding this comment

Uh oh!

JoaoJandre Oct 17, 2022

Choose a reason for hiding this comment

Uh oh!

JoaoJandre Oct 17, 2022

Choose a reason for hiding this comment

Uh oh!

JoaoJandre Oct 17, 2022

Choose a reason for hiding this comment

Uh oh!

DaanHoogland commented Dec 13, 2022

Uh oh!

DaanHoogland commented Jan 12, 2023

Uh oh!

github-actions bot commented Jan 27, 2023

Uh oh!

shwstppr commented May 9, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

11 participants

atrocitytheme commented Jul 16, 2021 •

edited

Loading