Overview
There are many 3rd-party tools which can be used to create an image file from a block device. While you can always follow the general procedure for this—create a snapshot, create a volume from that snapshot, and attach that to a VM to utilize your own tooling—Zadara also provides a tool for this via use of the zCompute Toolbox Image.
Creating a Machine Image with zCompute Toolbox
Downloading zCompute Toolbox
To start, you will need to download the zCompute Toolbox, unless you already have this.You may be able to choose a scope of Project, Account, or Public (depending on your permissions).
Creating zCompute Toolbox VM
Create a new VM Instance by deploying the zCompute Toolbox Image. Ensure that the VM has a minimum of double the image size available as free storage space. In other words, for a 10GB image, ensure that there is a minimum of 20GB of free storage space in the directory where the script is executed.
Creating an Image
-
Connect to the zCompute Toolbox linux VM:
- Begin by accessing the toolbox VM using the key pair specified during the VM creation process.
- Ensure to read the login information and run symp-update to fully install the latest symp binary which is a pre-requisite for the next steps.
-
Transfer the Image Conversion Script:
- Transfer the script provided below,
export_image_on_v2z.sh
, to the VM. Ensure that the script is placed in a partition with sufficient space to accommodate the image conversion process. Ensure that the script is set to be executable,
ie:chmod +x export_image_on_v2z.sh
- Transfer the script provided below,
-
Configure Cluster and Login Parameters as Environment Variables:
- Configure essential cluster and login parameters as environment variables. The parameters are as follows:
-
SYMP_URL
—the URL/IP of your cluster API endpoint -
SYMP_COMMAND
—the specifiedsymp
CLI command, usesymp -k
to bypass SSL certificate validation - Option1: Use Login Parameters:
-
SYMP_DOMAIN
—the Zadara Compute Account name -
SYMP_PROJECT
—the Project name
NB: if using default please set to "Default Project" -
SYMP_USERNAME
—the Username to be used for authentication -
SYMP_PASSWORD
—the Password to be used for authentication
-
- Option2: Alternatively, instead of using login parameters, you may provide a pre-existing API token, in which case all other login-related parameters are disregarded.
-
SYMP_TOKEN
—the API token to be used for authentication
-
-
- Configure essential cluster and login parameters as environment variables. The parameters are as follows:
-
Executing the Image Conversion Command:
- From the toolbox execute the following command to initiate the image conversion process, replacing
<image uuid>
with the relevant image UUID which can be found in the zCompute dashboard. - It's highly recommended not to use spaces or special characters, symbols in the source Image name to simplify the next steps.
-
./export_image_on_v2z.sh <image uuid>
-
- From the toolbox execute the following command to initiate the image conversion process, replacing
After successful execution, the converted image will be accessible as volume_<image_snapshot_name>.qcow2
within the local directory and can then be exported as needed.
Image Creation Script
export_image_on_v2z.sh
:
#!/bin/bash
# sudo yum install epel-release
# sudo yum install jq
# SYMP_URL=https://1.2.3.4
# SYMP_USERNAME=exampleuser
# SYMP_DOMAIN=exampleaccount
# SYMP_PROJECT=exampleproject
# SYMP_PASSWORD=*** symp -k
set -ex
if [ "$(whoami)" != "root" ]
then
echo "Not running as root - re-executing with sudo"
sudo "$0" $1
exit
fi
IMAGE_ID=$1
### Please configure the below ->
export SYMP_URL=https://1.2.3.4
export SYMP_USERNAME=exampleuser
export SYMP_DOMAIN=exampleaccount
export SYMP_PROJECT="Default Project"
export SYMP_PASSWORD=***
LOG="log_for_last_run"
echo `date` > $LOG
TLOG="tee -a $LOG"
SYMP="symp -k"
TOKEN=$(symp -k login $SYMP_USERNAME $SYMP_PASSWORD $SYMP_DOMAIN $SYMP_PROJECT -f value)
SYMP="symp -k --project-token=$TOKEN"
echo "Extracting own VM ID" | $TLOG
VM_INFO=$(curl http://169.254.169.254/openstack/latest/meta_data.json)
MYVM_ID=$(echo ${VM_INFO} | jq -r .uuid)
echo "Own VM ID is $VM_UUID" | $TLOG
## { "block_device_mapping": [ {"snapshot_id": uuid, asd: asd}, ...] }
echo "Getting IMAGE INFO" | $TLOG
IMAGE_INFO=$($SYMP machine-images get ${IMAGE_ID} -f json)
IMAGE_SNAPSHOTS=$(echo ${IMAGE_INFO} | jq -r .block_device_mapping[].snapshot_id)
for SN in ${IMAGE_SNAPSHOTS}
do
# Saving current disk device names
LVOLUMES=($(lsblk --output NAME -d -n -e 11 | sed -z 's/\n/\|/g'))
echo "Currently attached devices $LVOLUMES" | $TLOG
LVOLUMES=${LVOLUMES}dummyeoff
echo "Cloning image snapshot: $SN" | $TLOG
echo "running command: volume create --source-id $SN \"volume_for_image_download\"" | $TLOG
VOLUME_INFO=$($SYMP volume create --source-id $SN -f json "volume_for_image_download_${IMAGE_ID}")
VOLUME_ID=$(echo $VOLUME_INFO | jq -r .id)
SNAPSHOT_INFO=$($SYMP snapshot get $SN -f json)
SNAPSHOT_NAME=$(echo ${SNAPSHOT_INFO} | jq -r .name)
echo "Attaching new volumes and waiting for them be seen by the OS" | $TLOG
echo "running command: vm volumes attach $MYVM_ID $VOLUME_ID" | $TLOG
$SYMP vm volumes attach $MYVM_ID $VOLUME_ID | $TLOG
sleep 15 # allow kernel to recognize new voles.
$SYMP vm get -c id -c name -c bootVolume -c volumes -c networks -c vpc_id -c instanceType $MYVM_ID 2>>$LOG || $TLOG
echo "vm devices:" | $TLOG
lsblk --output NAME -d -n -e 11 | $TLOG
LOCALVOLS=(`lsblk --output NAME -d -n -e 11 | egrep -v "$LVOLUMES"`)
for ADDED_VOL in ${LOCALVOLS[*]}
do
echo "Detected added volume: $ADDED_VOL" | $TLOG
qemu-img convert -f raw -O qcow2 /dev/$ADDED_VOL volume_$SNAPSHOT_NAME.qcow2
done
done
Once the export image is available you can transfer this from the Toolbox using tools such as WinSCP, SCP, FileZilla or even transfer this to an Object Instance target for backup purposes.
The Toolbox VM already has the aws cli and s3cmd binaries available, which (once configured) will allow you to upload the converted image from the Toolbox VM externally to your own object store .
eg.
aws s3 --endpoint-url="https://s3.myobjectstore.com" cp volume_<image_snapshot_name>.qcow2
s3://examplebucket/examplefolder
aws s3 cp volume_<image_snapshot_name>.qcow2
s3://examplebucket/examplefolder