Cannot start cloudstack-agent after upgrade from 4.16.1 to 4.17.1

[levindecaro]

ISSUE TYPE

• Bug Report

COMPONENT NAME

cloudstack-agent

CLOUDSTACK VERSION

4.17.1

CONFIGURATION

kvm
SharedMountPoint (CephFS)

OS / ENVIRONMENT

CentOS Linux release 7.9.2009 (Core) 3.10.0-1160.45.1.el7.x86_64
libvirt-4.5.0-36.el7_9.5.x86_64
qemu-kvm-ev-2.12.0-44.1.el7_8.1.x86_64
qemu-img-ev-2.12.0-44.1.el7_8.1.x86_64
qemu-kvm-common-ev-2.12.0-44.1.el7_8.1.x86_64
libvirt-daemon-driver-qemu-4.5.0-36.el7_9.5.x86_64

SUMMARY

we have 10 cloudstack node are same configuration, however 2 of 10 node was failed to start cloudstack-agent after ran yum update cloudstack-agent to cloudstack-common-4.17.1.0-1.el7.x86_64, cloudstack-agent-4.17.1.0-1.el7.x86_64,
cloudstack-agent fail to start with reporting Unable to start agent: java.lang.InstantiationException: com.cloud.hypervisor.kvm.storage.ScaleIOStorageAdaptor

Resolve by

1. force remove node from cloudstack-management
2. yum remove cloudstack-agent
3. rm -rf /etc/cloudstack/agent/
4. yum install cloudstack-agent
5. re-add same node from cloudstack-management

Although this is a workaround from this situation, but it involve quite much cleanup task when it occurs . we dont ever configure any scaleIO storage before.

STEPS TO REPRODUCE

systemctl

EXPECTED RESULTS

cloudstack-agent start without problem after upgrade.

ACTUAL RESULTS

-- Logs begin at Sun 2022-08-28 00:13:44 HKT. --
Oct 01 02:31:46 compute-20.lab.example.com java[2743357]: at com.cloud.agent.Agent.<init>(Agent.java:189)
Oct 01 02:31:46 compute-20.lab.example.com java[2743357]: at com.cloud.agent.AgentShell.launchNewAgent(AgentShell.java:451)
Oct 01 02:31:46 compute-20.lab.example.com java[2743357]: at com.cloud.agent.AgentShell.launchAgentFromClassInfo(AgentShell.java:420)
Oct 01 02:31:46 compute-20.lab.example.com java[2743357]: at com.cloud.agent.AgentShell.launchAgent(AgentShell.java:404)
Oct 01 02:31:46 compute-20.lab.example.com java[2743357]: at com.cloud.agent.AgentShell.start(AgentShell.java:510)
Oct 01 02:31:46 compute-20.lab.example.com java[2743357]: at com.cloud.agent.AgentShell.main(AgentShell.java:545)
Oct 01 02:31:46 compute-20.lab.example.com java[2743357]: Unable to start agent: java.lang.InstantiationException: com.cloud.hypervisor.kvm.storage.ScaleIOStorageAdaptor
Oct 01 02:31:46 compute-20.lab.example.com systemd[1]: cloudstack-agent.service: main process exited, code=exited, status=67/n/a
Oct 01 02:31:46 compute-20.lab.example.com systemd[1]: Unit cloudstack-agent.service entered failed state.
Oct 01 02:31:46 compute-20.lab.example.com systemd[1]: cloudstack-agent.service failed.
Oct 01 02:31:56 compute-20.lab.example.com systemd[1]: cloudstack-agent.service holdoff time over, scheduling restart.
Oct 01 02:31:56 compute-20.lab.example.com systemd[1]: Stopped CloudStack Agent.
Oct 01 02:31:56 compute-20.lab.example.com systemd[1]: Started CloudStack Agent.
Oct 01 02:31:56 compute-20.lab.example.com java[2743451]: log4j:WARN No appenders could be found for logger (com.cloud.agent.AgentShell).
Oct 01 02:31:56 compute-20.lab.example.com java[2743451]: log4j:WARN Please initialize the log4j system properly.
Oct 01 02:31:56 compute-20.lab.example.com java[2743451]: log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: INFO  [cloud.agent.AgentShell] (main:) (logid:) Agent started
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: INFO  [cloud.agent.AgentShell] (main:) (logid:) Implementation Version is 4.17.1.0
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: INFO  [cloud.agent.AgentShell] (main:) (logid:) agent.properties found at /etc/cloudstack/agent/agent.properties
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [cloud.agent.AgentShell] (main:) (logid:) Found property: vm.rng.rate.period
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [cloud.agent.AgentShell] (main:) (logid:) Found property: vm.rng.enable
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [cloud.agent.AgentShell] (main:) (logid:) Found property: resource
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [cloud.agent.AgentShell] (main:) (logid:) Found property: vm.rng.model
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [cloud.agent.AgentShell] (main:) (logid:) Found property: guest.network.device
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [cloud.agent.AgentShell] (main:) (logid:) Found property: local.storage.uuid
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [cloud.agent.AgentShell] (main:) (logid:) Found property: router.aggregation.command.each.timeout
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [cloud.agent.AgentShell] (main:) (logid:) Found property: port
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [cloud.agent.AgentShell] (main:) (logid:) Found property: iscsi.session.cleanup.enabled
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [cloud.agent.AgentShell] (main:) (logid:) Found property: host
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [cloud.agent.AgentShell] (main:) (logid:) Found property: pod
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [cloud.agent.AgentShell] (main:) (logid:) Found property: guid
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [cloud.agent.AgentShell] (main:) (logid:) Found property: vm.rng.path
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [cloud.agent.AgentShell] (main:) (logid:) Found property: LibvirtComputingResource.id
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [cloud.agent.AgentShell] (main:) (logid:) Found property: cluster
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [cloud.agent.AgentShell] (main:) (logid:) Found property: public.network.device
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [cloud.agent.AgentShell] (main:) (logid:) Found property: private.network.device
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [cloud.agent.AgentShell] (main:) (logid:) Found property: zone
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [cloud.agent.AgentShell] (main:) (logid:) Found property: vm.rng.rate.bytes
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [cloud.agent.AgentShell] (main:) (logid:) Found property: domr.scripts.dir
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [cloud.agent.AgentShell] (main:) (logid:) Found property: keystore.passphrase
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [cloud.agent.AgentShell] (main:) (logid:) Found property: guest.cpu.mode
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [cloud.agent.AgentShell] (main:) (logid:) Found property: vm.migrate.wait
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [cloud.agent.AgentShell] (main:) (logid:) Found property: workers
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [cloud.agent.AgentShell] (main:) (logid:) Found property: hypervisor.type
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: INFO  [cloud.agent.AgentShell] (main:) (logid:) Defaulting to using properties file for storage
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: INFO  [cloud.agent.AgentShell] (main:) (logid:) Defaulting to the constant time backoff algorithm
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: INFO  [cloud.utils.LogUtils] (main:) (logid:) log4j configuration found at /etc/cloudstack/agent/log4j-cloud.xml
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: INFO  [cloud.agent.AgentShell] (main:) (logid:) Using default Java settings for IPv6 preference for agent connection
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [cloud.agent.AgentShell] (main:) (logid:) Checking to see if agent.pid exists.
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [cloud.utils.ProcessUtil] (main:) (logid:) Executing: bash -c echo $PPID
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [cloud.utils.ProcessUtil] (main:) (logid:) Executing while with timeout : 120000
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [cloud.utils.ProcessUtil] (main:) (logid:) Execution is successful.
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: INFO  [cloud.agent.Agent] (main:) (logid:) id is 75
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [cloud.resource.ServerResourceBase] (main:) (logid:) Retrieving network interface: cloudbr0
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [cloud.resource.ServerResourceBase] (main:) (logid:) Retrieving network interface: cloudbr0
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [cloud.resource.ServerResourceBase] (main:) (logid:) Retrieving network interface: null
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [cloud.resource.ServerResourceBase] (main:) (logid:) Retrieving network interface: null
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: INFO  [kvm.resource.LibvirtComputingResource] (main:) (logid:) uefi.properties file found at /etc/cloudstack/agent/uefi.properties
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: INFO  [kvm.resource.LibvirtComputingResource] (main:) (logid:) guest.nvram.template.legacy = /usr/share/OVMF/OVMF_VARS.fd
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: INFO  [kvm.resource.LibvirtComputingResource] (main:) (logid:) guest.loader.legacy = /usr/share/OVMF/OVMF_CODE.secboot.fd
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: INFO  [kvm.resource.LibvirtComputingResource] (main:) (logid:) guest.nvram.template.secure = /usr/share/OVMF/OVMF_VARS.secboot.fd
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: INFO  [kvm.resource.LibvirtComputingResource] (main:) (logid:) guest.loader.secure =/usr/share/OVMF/OVMF_CODE.secboot.fd
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: INFO  [kvm.resource.LibvirtComputingResource] (main:) (logid:) guest.nvram.path = /var/lib/libvirt/qemu/nvram/
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [resource.virtualnetwork.VirtualRoutingResource] (main:) (logid:) The router.aggregation.command.each.timeout in seconds is set to 600
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for update_host_passwd.sh in the classpath
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) System resource: null
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Classpath resource: null
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for update_host_passwd.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Current binaries reside at /usr/share/cloudstack-agent/lib
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for update_host_passwd.sh in /usr/share/cloudstack-agent/lib/scripts/vm/hypervisor/update_host_passwd.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for update_host_passwd.sh in /usr/share/cloudstack-agent/lib/scripts/vm/update_host_passwd.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for update_host_passwd.sh in /usr/share/cloudstack-agent/lib/scripts/update_host_passwd.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for update_host_passwd.sh in /usr/share/cloudstack-agent/lib/update_host_passwd.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for update_host_passwd.sh in /usr/share/cloudstack-agent/update_host_passwd.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for update_host_passwd.sh in /usr/share/update_host_passwd.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for update_host_passwd.sh in /usr/update_host_passwd.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for update_host_passwd.sh in /update_host_passwd.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Searching in environment.properties
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) environment.properties says scripts should be in /usr/share/cloudstack-common
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for update_host_passwd.sh in /usr/share/cloudstack-common/scripts/vm/hypervisor/update_host_passwd.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for modifyvlan.sh in the classpath
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) System resource: null
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Classpath resource: null
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for modifyvlan.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Current binaries reside at /usr/share/cloudstack-agent/lib
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for modifyvlan.sh in /usr/share/cloudstack-agent/lib/scripts/vm/network/vnet/modifyvlan.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for modifyvlan.sh in /usr/share/cloudstack-agent/lib/scripts/vm/network/modifyvlan.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for modifyvlan.sh in /usr/share/cloudstack-agent/lib/scripts/vm/modifyvlan.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for modifyvlan.sh in /usr/share/cloudstack-agent/lib/scripts/modifyvlan.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for modifyvlan.sh in /usr/share/cloudstack-agent/lib/modifyvlan.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for modifyvlan.sh in /usr/share/cloudstack-agent/modifyvlan.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for modifyvlan.sh in /usr/share/modifyvlan.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for modifyvlan.sh in /usr/modifyvlan.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for modifyvlan.sh in /modifyvlan.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Searching in environment.properties
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) environment.properties says scripts should be in /usr/share/cloudstack-common
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for modifyvlan.sh in /usr/share/cloudstack-common/scripts/vm/network/vnet/modifyvlan.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for versions.sh in the classpath
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) System resource: null
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Classpath resource: null
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for versions.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Current binaries reside at /usr/share/cloudstack-agent/lib
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for versions.sh in /usr/share/cloudstack-agent/lib/scripts/vm/hypervisor/kvm/versions.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for versions.sh in /usr/share/cloudstack-agent/lib/scripts/vm/hypervisor/versions.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for versions.sh in /usr/share/cloudstack-agent/lib/scripts/vm/versions.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for versions.sh in /usr/share/cloudstack-agent/lib/scripts/versions.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for versions.sh in /usr/share/cloudstack-agent/lib/versions.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for versions.sh in /usr/share/cloudstack-agent/versions.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for versions.sh in /usr/share/versions.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for versions.sh in /usr/versions.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for versions.sh in /versions.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Searching in environment.properties
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) environment.properties says scripts should be in /usr/share/cloudstack-common
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for versions.sh in /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/versions.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for versions.sh in /usr/share/cloudstack-common/scripts/vm/hypervisor/versions.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for patch.sh in the classpath
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) System resource: null
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Classpath resource: null
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for patch.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Current binaries reside at /usr/share/cloudstack-agent/lib
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for patch.sh in /usr/share/cloudstack-agent/lib/scripts/vm/hypervisor/kvm/patch.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for patch.sh in /usr/share/cloudstack-agent/lib/scripts/vm/hypervisor/patch.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for patch.sh in /usr/share/cloudstack-agent/lib/scripts/vm/patch.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for patch.sh in /usr/share/cloudstack-agent/lib/scripts/patch.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for patch.sh in /usr/share/cloudstack-agent/lib/patch.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for patch.sh in /usr/share/cloudstack-agent/patch.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for patch.sh in /usr/share/patch.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for patch.sh in /usr/patch.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for patch.sh in /patch.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Searching in environment.properties
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) environment.properties says scripts should be in /usr/share/cloudstack-common
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for patch.sh in /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/patch.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for kvmheartbeat.sh in the classpath
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) System resource: null
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Classpath resource: null
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for kvmheartbeat.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Current binaries reside at /usr/share/cloudstack-agent/lib
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for kvmheartbeat.sh in /usr/share/cloudstack-agent/lib/scripts/vm/hypervisor/kvm/kvmheartbeat.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for kvmheartbeat.sh in /usr/share/cloudstack-agent/lib/scripts/vm/hypervisor/kvmheartbeat.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for kvmheartbeat.sh in /usr/share/cloudstack-agent/lib/scripts/vm/kvmheartbeat.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for kvmheartbeat.sh in /usr/share/cloudstack-agent/lib/scripts/kvmheartbeat.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for kvmheartbeat.sh in /usr/share/cloudstack-agent/lib/kvmheartbeat.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for kvmheartbeat.sh in /usr/share/cloudstack-agent/kvmheartbeat.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for kvmheartbeat.sh in /usr/share/kvmheartbeat.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for kvmheartbeat.sh in /usr/kvmheartbeat.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for kvmheartbeat.sh in /kvmheartbeat.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Searching in environment.properties
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) environment.properties says scripts should be in /usr/share/cloudstack-common
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for kvmheartbeat.sh in /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/kvmheartbeat.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for createvm.sh in the classpath
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) System resource: null
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Classpath resource: null
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for createvm.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Current binaries reside at /usr/share/cloudstack-agent/lib
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for createvm.sh in /usr/share/cloudstack-agent/lib/scripts/storage/qcow2/createvm.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for createvm.sh in /usr/share/cloudstack-agent/lib/scripts/storage/createvm.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for createvm.sh in /usr/share/cloudstack-agent/lib/scripts/createvm.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for createvm.sh in /usr/share/cloudstack-agent/lib/createvm.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for createvm.sh in /usr/share/cloudstack-agent/createvm.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for createvm.sh in /usr/share/createvm.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for createvm.sh in /usr/createvm.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for createvm.sh in /createvm.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Searching in environment.properties
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) environment.properties says scripts should be in /usr/share/cloudstack-common
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for createvm.sh in /usr/share/cloudstack-common/scripts/storage/qcow2/createvm.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for managesnapshot.sh in the classpath
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) System resource: null
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Classpath resource: null
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for managesnapshot.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Current binaries reside at /usr/share/cloudstack-agent/lib
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for managesnapshot.sh in /usr/share/cloudstack-agent/lib/scripts/storage/qcow2/managesnapshot.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for managesnapshot.sh in /usr/share/cloudstack-agent/lib/scripts/storage/managesnapshot.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for managesnapshot.sh in /usr/share/cloudstack-agent/lib/scripts/managesnapshot.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for managesnapshot.sh in /usr/share/cloudstack-agent/lib/managesnapshot.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for managesnapshot.sh in /usr/share/cloudstack-agent/managesnapshot.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for managesnapshot.sh in /usr/share/managesnapshot.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for managesnapshot.sh in /usr/managesnapshot.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for managesnapshot.sh in /managesnapshot.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Searching in environment.properties
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) environment.properties says scripts should be in /usr/share/cloudstack-common
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for managesnapshot.sh in /usr/share/cloudstack-common/scripts/storage/qcow2/managesnapshot.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for resizevolume.sh in the classpath
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) System resource: null
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Classpath resource: null
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for resizevolume.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Current binaries reside at /usr/share/cloudstack-agent/lib
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for resizevolume.sh in /usr/share/cloudstack-agent/lib/scripts/storage/qcow2/resizevolume.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for resizevolume.sh in /usr/share/cloudstack-agent/lib/scripts/storage/resizevolume.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for resizevolume.sh in /usr/share/cloudstack-agent/lib/scripts/resizevolume.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for resizevolume.sh in /usr/share/cloudstack-agent/lib/resizevolume.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for resizevolume.sh in /usr/share/cloudstack-agent/resizevolume.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for resizevolume.sh in /usr/share/resizevolume.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for resizevolume.sh in /usr/resizevolume.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for resizevolume.sh in /resizevolume.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Searching in environment.properties
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) environment.properties says scripts should be in /usr/share/cloudstack-common
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for resizevolume.sh in /usr/share/cloudstack-common/scripts/storage/qcow2/resizevolume.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for kvmvmactivity.sh in the classpath
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) System resource: null
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Classpath resource: null
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for kvmvmactivity.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Current binaries reside at /usr/share/cloudstack-agent/lib
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for kvmvmactivity.sh in /usr/share/cloudstack-agent/lib/scripts/vm/hypervisor/kvm/kvmvmactivity.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for kvmvmactivity.sh in /usr/share/cloudstack-agent/lib/scripts/vm/hypervisor/kvmvmactivity.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for kvmvmactivity.sh in /usr/share/cloudstack-agent/lib/scripts/vm/kvmvmactivity.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for kvmvmactivity.sh in /usr/share/cloudstack-agent/lib/scripts/kvmvmactivity.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for kvmvmactivity.sh in /usr/share/cloudstack-agent/lib/kvmvmactivity.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for kvmvmactivity.sh in /usr/share/cloudstack-agent/kvmvmactivity.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for kvmvmactivity.sh in /usr/share/kvmvmactivity.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for kvmvmactivity.sh in /usr/kvmvmactivity.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for kvmvmactivity.sh in /kvmvmactivity.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Searching in environment.properties
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) environment.properties says scripts should be in /usr/share/cloudstack-common
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for kvmvmactivity.sh in /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/kvmvmactivity.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for createtmplt.sh in the classpath
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) System resource: null
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Classpath resource: null
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for createtmplt.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Current binaries reside at /usr/share/cloudstack-agent/lib
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for createtmplt.sh in /usr/share/cloudstack-agent/lib/scripts/storage/qcow2/createtmplt.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for createtmplt.sh in /usr/share/cloudstack-agent/lib/scripts/storage/createtmplt.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for createtmplt.sh in /usr/share/cloudstack-agent/lib/scripts/createtmplt.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for createtmplt.sh in /usr/share/cloudstack-agent/lib/createtmplt.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for createtmplt.sh in /usr/share/cloudstack-agent/createtmplt.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for createtmplt.sh in /usr/share/createtmplt.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for createtmplt.sh in /usr/createtmplt.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for createtmplt.sh in /createtmplt.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Searching in environment.properties
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) environment.properties says scripts should be in /usr/share/cloudstack-common
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for createtmplt.sh in /usr/share/cloudstack-common/scripts/storage/qcow2/createtmplt.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for security_group.py in the classpath
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) System resource: null
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Classpath resource: null
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for security_group.py
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Current binaries reside at /usr/share/cloudstack-agent/lib
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for security_group.py in /usr/share/cloudstack-agent/lib/scripts/vm/network/vnet/security_group.py
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for security_group.py in /usr/share/cloudstack-agent/lib/scripts/vm/network/security_group.py
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for security_group.py in /usr/share/cloudstack-agent/lib/scripts/vm/security_group.py
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for security_group.py in /usr/share/cloudstack-agent/lib/scripts/security_group.py
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for security_group.py in /usr/share/cloudstack-agent/lib/security_group.py
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for security_group.py in /usr/share/cloudstack-agent/security_group.py
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for security_group.py in /usr/share/security_group.py
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for security_group.py in /usr/security_group.py
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for security_group.py in /security_group.py
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Searching in environment.properties
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) environment.properties says scripts should be in /usr/share/cloudstack-common
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for security_group.py in /usr/share/cloudstack-common/scripts/vm/network/vnet/security_group.py
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for security_group.py in /usr/share/cloudstack-common/scripts/vm/network/security_group.py
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for ovstunnel.py in the classpath
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) System resource: null
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Classpath resource: null
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for ovstunnel.py
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Current binaries reside at /usr/share/cloudstack-agent/lib
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for ovstunnel.py in /usr/share/cloudstack-agent/lib/scripts/vm/network/vnet/ovstunnel.py
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for ovstunnel.py in /usr/share/cloudstack-agent/lib/scripts/vm/network/ovstunnel.py
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for ovstunnel.py in /usr/share/cloudstack-agent/lib/scripts/vm/ovstunnel.py
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for ovstunnel.py in /usr/share/cloudstack-agent/lib/scripts/ovstunnel.py
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for ovstunnel.py in /usr/share/cloudstack-agent/lib/ovstunnel.py
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for ovstunnel.py in /usr/share/cloudstack-agent/ovstunnel.py
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for ovstunnel.py in /usr/share/ovstunnel.py
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for ovstunnel.py in /usr/ovstunnel.py
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for ovstunnel.py in /ovstunnel.py
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Searching in environment.properties
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) environment.properties says scripts should be in /usr/share/cloudstack-common
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for ovstunnel.py in /usr/share/cloudstack-common/scripts/vm/network/vnet/ovstunnel.py
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for router_proxy.sh in the classpath
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) System resource: null
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Classpath resource: null
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for router_proxy.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Current binaries reside at /usr/share/cloudstack-agent/lib
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for router_proxy.sh in /usr/share/cloudstack-agent/lib/scripts/network/domr/router_proxy.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for router_proxy.sh in /usr/share/cloudstack-agent/lib/scripts/network/router_proxy.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for router_proxy.sh in /usr/share/cloudstack-agent/lib/scripts/router_proxy.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for router_proxy.sh in /usr/share/cloudstack-agent/lib/router_proxy.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for router_proxy.sh in /usr/share/cloudstack-agent/router_proxy.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for router_proxy.sh in /usr/share/router_proxy.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for router_proxy.sh in /usr/router_proxy.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for router_proxy.sh in /router_proxy.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Searching in environment.properties
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) environment.properties says scripts should be in /usr/share/cloudstack-common
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for router_proxy.sh in /usr/share/cloudstack-common/scripts/network/domr/router_proxy.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for ovs-pvlan-kvm-dhcp-host.sh in the classpath
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) System resource: null
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Classpath resource: null
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for ovs-pvlan-kvm-dhcp-host.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Current binaries reside at /usr/share/cloudstack-agent/lib
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for ovs-pvlan-kvm-dhcp-host.sh in /usr/share/cloudstack-agent/lib/scripts/vm/network/vnet/ovs-pvlan-kvm-dhcp-host.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for ovs-pvlan-kvm-dhcp-host.sh in /usr/share/cloudstack-agent/lib/scripts/vm/network/ovs-pvlan-kvm-dhcp-host.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for ovs-pvlan-kvm-dhcp-host.sh in /usr/share/cloudstack-agent/lib/scripts/vm/ovs-pvlan-kvm-dhcp-host.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for ovs-pvlan-kvm-dhcp-host.sh in /usr/share/cloudstack-agent/lib/scripts/ovs-pvlan-kvm-dhcp-host.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for ovs-pvlan-kvm-dhcp-host.sh in /usr/share/cloudstack-agent/lib/ovs-pvlan-kvm-dhcp-host.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for ovs-pvlan-kvm-dhcp-host.sh in /usr/share/cloudstack-agent/ovs-pvlan-kvm-dhcp-host.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for ovs-pvlan-kvm-dhcp-host.sh in /usr/share/ovs-pvlan-kvm-dhcp-host.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for ovs-pvlan-kvm-dhcp-host.sh in /usr/ovs-pvlan-kvm-dhcp-host.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for ovs-pvlan-kvm-dhcp-host.sh in /ovs-pvlan-kvm-dhcp-host.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Searching in environment.properties
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) environment.properties says scripts should be in /usr/share/cloudstack-common
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for ovs-pvlan-kvm-dhcp-host.sh in /usr/share/cloudstack-common/scripts/vm/network/vnet/ovs-pvlan-kvm-dhcp-host.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for ovs-pvlan-kvm-dhcp-host.sh in /usr/share/cloudstack-common/scripts/vm/network/ovs-pvlan-kvm-dhcp-host.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for ovs-pvlan-kvm-vm.sh in the classpath
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) System resource: null
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Classpath resource: null
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for ovs-pvlan-kvm-vm.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Current binaries reside at /usr/share/cloudstack-agent/lib
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for ovs-pvlan-kvm-vm.sh in /usr/share/cloudstack-agent/lib/scripts/vm/network/vnet/ovs-pvlan-kvm-vm.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for ovs-pvlan-kvm-vm.sh in /usr/share/cloudstack-agent/lib/scripts/vm/network/ovs-pvlan-kvm-vm.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for ovs-pvlan-kvm-vm.sh in /usr/share/cloudstack-agent/lib/scripts/vm/ovs-pvlan-kvm-vm.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for ovs-pvlan-kvm-vm.sh in /usr/share/cloudstack-agent/lib/scripts/ovs-pvlan-kvm-vm.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for ovs-pvlan-kvm-vm.sh in /usr/share/cloudstack-agent/lib/ovs-pvlan-kvm-vm.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for ovs-pvlan-kvm-vm.sh in /usr/share/cloudstack-agent/ovs-pvlan-kvm-vm.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for ovs-pvlan-kvm-vm.sh in /usr/share/ovs-pvlan-kvm-vm.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for ovs-pvlan-kvm-vm.sh in /usr/ovs-pvlan-kvm-vm.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for ovs-pvlan-kvm-vm.sh in /ovs-pvlan-kvm-vm.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Searching in environment.properties
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) environment.properties says scripts should be in /usr/share/cloudstack-common
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for ovs-pvlan-kvm-vm.sh in /usr/share/cloudstack-common/scripts/vm/network/vnet/ovs-pvlan-kvm-vm.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for ovs-pvlan-kvm-vm.sh in /usr/share/cloudstack-common/scripts/vm/network/ovs-pvlan-kvm-vm.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for pingtest.sh in the classpath
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) System resource: null
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Classpath resource: null
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for pingtest.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Current binaries reside at /usr/share/cloudstack-agent/lib
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for pingtest.sh in /usr/share/cloudstack-agent/lib/scripts/vm/hypervisor/kvm/pingtest.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for pingtest.sh in /usr/share/cloudstack-agent/lib/scripts/vm/hypervisor/pingtest.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for pingtest.sh in /usr/share/cloudstack-agent/lib/scripts/vm/pingtest.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for pingtest.sh in /usr/share/cloudstack-agent/lib/scripts/pingtest.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for pingtest.sh in /usr/share/cloudstack-agent/lib/pingtest.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for pingtest.sh in /usr/share/cloudstack-agent/pingtest.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for pingtest.sh in /usr/share/pingtest.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for pingtest.sh in /usr/pingtest.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for pingtest.sh in /pingtest.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Searching in environment.properties
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) environment.properties says scripts should be in /usr/share/cloudstack-common
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for pingtest.sh in /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/pingtest.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for pingtest.sh in /usr/share/cloudstack-common/scripts/vm/hypervisor/pingtest.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for pingtest.sh in /usr/share/cloudstack-common/scripts/vm/pingtest.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [kvm.resource.LibvirtConnection] (main:) (logid:) Looking for libvirtd connection at: qemu:///system
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: INFO  [kvm.resource.LibvirtConnection] (main:) (logid:) No existing libvirtd connection found. Opening a new one
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [kvm.resource.LibvirtConnection] (main:) (logid:) Successfully connected to libvirt at: qemu:///system
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [kvm.resource.LibvirtComputingResource] (main:) (logid:) Found libvirt default network, destroying it and setting autostart to false
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [kvm.resource.LibvirtCapXMLParser] (main:) (logid:) Found /usr/libexec/qemu-kvm as a suiteable emulator
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [kvm.resource.LibvirtCapXMLParser] (main:) (logid:) Found /usr/libexec/qemu-kvm as a suiteable emulator
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for managesnapshot.sh in the classpath
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) System resource: null
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Classpath resource: null
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for managesnapshot.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Current binaries reside at /usr/share/cloudstack-agent/lib
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for managesnapshot.sh in /usr/share/cloudstack-agent/lib/scripts/storage/qcow2/managesnapshot.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for managesnapshot.sh in /usr/share/cloudstack-agent/lib/scripts/storage/managesnapshot.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for managesnapshot.sh in /usr/share/cloudstack-agent/lib/scripts/managesnapshot.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for managesnapshot.sh in /usr/share/cloudstack-agent/lib/managesnapshot.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for managesnapshot.sh in /usr/share/cloudstack-agent/managesnapshot.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for managesnapshot.sh in /usr/share/managesnapshot.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for managesnapshot.sh in /usr/managesnapshot.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for managesnapshot.sh in /managesnapshot.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Searching in environment.properties
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) environment.properties says scripts should be in /usr/share/cloudstack-common
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: DEBUG [utils.script.Script] (main:) (logid:) Looking for managesnapshot.sh in /usr/share/cloudstack-common/scripts/storage/qcow2/managesnapshot.sh
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: ERROR [cloud.agent.AgentShell] (main:) (logid:) Unable to start agent:
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: com.cloud.utils.exception.CloudRuntimeException: java.lang.InstantiationException: com.cloud.hypervisor.kvm.storage.ScaleIOStorageAdaptor
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: at com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.<init>(KVMStoragePoolManager.java:116)
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.configure(LibvirtComputingResource.java:1056)
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: at com.cloud.agent.Agent.<init>(Agent.java:189)
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: at com.cloud.agent.AgentShell.launchNewAgent(AgentShell.java:451)
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: at com.cloud.agent.AgentShell.launchAgentFromClassInfo(AgentShell.java:420)
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: at com.cloud.agent.AgentShell.launchAgent(AgentShell.java:404)
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: at com.cloud.agent.AgentShell.start(AgentShell.java:510)
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: at com.cloud.agent.AgentShell.main(AgentShell.java:545)
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: Unable to start agent: java.lang.InstantiationException: com.cloud.hypervisor.kvm.storage.ScaleIOStorageAdaptor
Oct 01 02:31:57 compute-20.lab.example.com systemd[1]: cloudstack-agent.service: main process exited, code=exited, status=67/n/a
Oct 01 02:31:57 compute-20.lab.example.com systemd[1]: Unit cloudstack-agent.service entered failed state.
Oct 01 02:31:57 compute-20.lab.example.com systemd[1]: cloudstack-agent.service failed.

[DaanHoogland]

The problem seems to be that a StorageAdaptor is being attempted to be created which doesn´t have a default constructor, by the looks of the stacktrace.

[weizhouapache]

The problem seems to be that a StorageAdaptor is being attempted to be created which doesn´t have a default constructor, by the looks of the stacktrace.

@DaanHoogland I think it might not be caused by java code. because @levindecaro has fixed the issue without any code changes
maybe a packaging or installation or permission issue ?

[DaanHoogland]

The problem seems to be that a StorageAdaptor is being attempted to be created which doesn´t have a default constructor, by the looks of the stacktrace.

@DaanHoogland I think it might not be caused by java code. because @levindecaro has fixed the issue without any code changes
maybe a packaging or installation or permission issue ?

Yes, but the stored configuration had to be thrown away.
As there is a workaround I wouldn't regard it critical. Not for smaller environments anyway.

[DaanHoogland]

@weizhouapache there may be a direct cause in terms of permissions but:

Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: ERROR [cloud.agent.AgentShell] (main:) (logid:) Unable to start agent:
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: com.cloud.utils.exception.CloudRuntimeException: java.lang.InstantiationException: com.cloud.hypervisor.kvm.storage.ScaleIOStorageAdaptor
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: at com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.<init>(KVMStoragePoolManager.java:116)
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.configure(LibvirtComputingResource.java:1056)
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: at com.cloud.agent.Agent.<init>(Agent.java:189)
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: at com.cloud.agent.AgentShell.launchNewAgent(AgentShell.java:451)
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: at com.cloud.agent.AgentShell.launchAgentFromClassInfo(AgentShell.java:420)
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: at com.cloud.agent.AgentShell.launchAgent(AgentShell.java:404)
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: at com.cloud.agent.AgentShell.start(AgentShell.java:510)
Oct 01 02:31:57 compute-20.lab.example.com java[2743451]: at com.cloud.agent.AgentShell.main(AgentShell.java:545)

this snippit shows that a default constructor is used in the code and it does not exist in the code. That is a bug in java code for sure.

[DaanHoogland]

@levindecaro is there any way you could reproduce this and test a solution against it? i.e. #6797
I want to move this forward but have a lack of clue atm

[levindecaro]

@DaanHoogland I'll try to validate it soon by reinstalling the agent with this revised version, I'm not sure it is 100% reproducible, but i'll report the result if got anything new. thank you

[levindecaro]

@DaanHoogland

Just tried to build 4.17.2 with this #6797, but it still fail after agent upgrade

Dec 22 22:40:08 compute-20.example.com java[1991061]: com.cloud.utils.exception.CloudRuntimeException: java.lang.InstantiationException: com.cloud.hypervisor.kvm.storage.ScaleIOStorageAdaptor
Dec 22 22:40:08 compute-20.example.com java[1991061]: at com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.<init>(KVMStoragePoolManager.java:116)
Dec 22 22:40:08 compute-20.example.com java[1991061]: at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.configure(LibvirtComputingResource.java:1056)
Dec 22 22:40:08 compute-20.example.com java[1991061]: at com.cloud.agent.Agent.<init>(Agent.java:189)
Dec 22 22:40:08 compute-20.example.com java[1991061]: at com.cloud.agent.AgentShell.launchNewAgent(AgentShell.java:451)
Dec 22 22:40:08 compute-20.example.com java[1991061]: at com.cloud.agent.AgentShell.launchAgentFromClassInfo(AgentShell.java:420)
Dec 22 22:40:08 compute-20.example.com java[1991061]: at com.cloud.agent.AgentShell.launchAgent(AgentShell.java:404)
Dec 22 22:40:08 compute-20.example.com java[1991061]: at com.cloud.agent.AgentShell.start(AgentShell.java:510)
Dec 22 22:40:08 compute-20.example.com java[1991061]: at com.cloud.agent.AgentShell.main(AgentShell.java:545)
Dec 22 22:40:08 compute-20.example.com java[1991061]: Unable to start agent: java.lang.InstantiationException: com.cloud.hypervisor.kvm.storage.ScaleIOStorageAdaptor
Dec 22 22:40:08 compute-20.example.com libvirtd[1965359]: 2022-12-22 14:40:08.496+0000: 1965359: error : virNetSocketReadWire:1806 : End of file while reading data: Input/output error