Backlog #2767
Qcow2 preallocation features
| Status: | Pending | Start date: | 02/28/2014 | |
|---|---|---|---|---|
| Priority: | High | Due date: | ||
| Assignee: | - | % Done: | 0% | |
| Category: | Drivers - Storage | |||
| Target version: | - |
Description
Hello,
Recently we set up a new OpenNebula environnement for the need of system architects.
We use this platform as a mirror of the production (only physical platform) for testing configurations, kickstart, etc...
But physical drives have a lot of space avalaible contrary to vm. For this reason we use qcow preallocation metadata, to allow storage overallocation.
To integrate this into opennebula we change the way the driver works, but this broke the creation of other image type.
Currently the way OpenNebula create image isn't compatible with preallocation because it use "dd" or a pipe with "cat". We replace all of this command by a dirty qcow preallocation... For example :
Datastore/fs/mkfs :
Original : exec_and_log "$DD if=/dev/zero of=$DST bs=1 count=1 seek=${SIZE}M"
Qcow Preallocation : exec_and_log "/usr/bin/qemu-img create -f qcow2 -o preallocation=metadata $DST ${SIZE}M"
Datastore/fs/cp
Original : COPY_COMMAND="$UTILS_PATH/downloader.sh $DOWNLOADER_ARGS"
COPY_COMMAND="/bin/cp --sparse=always $SRC $DST"
Currently this works, but we have to be sure that when creating a image, qcow2 type is used, otherwise the vm won't start.
It would be great to upgrade the fs driver to allow qcow preallocation (and probably some transfer driver to manage the volatile mkimage or cloning).
History
#1
Updated by Ruben S. Montero over 7 years ago
I think this is already covered:
- When creating a datablock, simply use FSTYPE=qcow2, either for datastore images http://docs.opennebula.org/stable/user/references/img_template.html or for volatile disks http://docs.opennebula.org/stable/user/references/template.html#volatile-disks with FORMAT
- To clone using qcow features, just configure your datastore to use the qcow2 drivers http://docs.opennebula.org/stable/administration/storage/fs_ds.html#using-the-qcow2-transfer-driver
#2
Updated by Bastien Cadiot over 7 years ago
Hi Ruben,
I'm reading the docs and qcow2 driver code, but I'm always thinking that OpenNebula isn't designed for the moment to allow sparse in storage.
Qcow2 tm drivers allow to create qcow2 images but without any specific options. For example (CLONE_CMD="cd $DST_DIR; $QEMU_IMG create -b $SRC_PATH -f qcow2 $DST_PATH"), this do not allow to add "-o preallocation" feature.
In the same way QCOW2 fstype, and qcow2 TM clone has a strange behaviour. The driver create the image with a DD, then qemu-img replace this file...
Example Datastore/fs/mkfs :
exec_and_log "$DD if=/dev/zero of=$DST bs=1 count=1 seek=${SIZE}M" \
"Could not create image $DST"
exec_and_log "$MKFS_CMD" \
"Unable to create filesystem $FSTYPE in $DST"
I still don't know what is the best way to achieve this. Patch current ds driver (fs) and tm driver (qcow2) to allow preallocation feature, or create a new DS driver based on the FS one ?
#3
Updated by Ruben S. Montero over 7 years ago
- Tracker changed from Request to Backlog
- Priority changed from Normal to High
You are right , for some reason I over-read the preallocation part, and just though the issue was tu use qcow specific actions :(
Maybe we can add a QEMU_OPTIONS variable to scripts_common.sh and then use it for QEMU_IMG. Another alternative would be to apply this just to the fs drivers, like we did for ceph (e.g. take a look to ceph/ceph.conf, we have QEMU_IMG_CONVERT_ARGS there)
Moving this issue to our backlog to implement it in future releases. In the meantime just be careful when upgrading to preserve your modifications (we are also looking to the best way to keep driver modifications...)
Thanks Ruben