Create Custom Cloud Image

Bhuvan

Creating a custom image using "virt-customized":

Install libguestfs-tools to use virt-customized:

sudo apt update
sudo apt-get -y install libguestfs-tools
export LIBGUESTFS_BACKEND=direct

Download base cloud image (Ubuntu 16.04 Xenial or any other as required):

wget http://cloud-images.ubuntu.com/xenial/current/xenial-server-cloudimg-amd64-disk1.img

Install packages in cloud image:

# virt-customize -a [DOWNLOADED_FILE_NAME] --install [LIST_OF_PACKAGES]
virt-customize -a xenial-server-cloudimg-amd64-disk1.img --install apache2,git,python-dev

Upload files in cloud image:

# virt-customize -a [DOWNLOADED_FILE_NAME] --upload [LOCAL_FILE_PATH]:[IMAGE_FILE_PATH]
virt-customize -a xenial-server-cloudimg-amd64-disk1.img --upload pyscript.py:/pyscript.py

Edit cloud.cfg file inside the cloud image: (MANDATORY)

# virt-customize -a [DOWNLOADED_FILE_NAME]  --run-command 'echo "manage_etc_hosts: true" >> /etc/cloud/cloud.cfg'
virt-customize -a xenial-server-cloudimg-amd64-disk1.img --run-command 'echo "manage_etc_hosts: true" >> /etc/cloud/cloud.cfg'

Create tng-bench-mp for openstack:

Taken from mpeuster/tng-bench-mp.
Install packages and wget required repos:

experiments/tree/master/vnfs/tng-bench-mp)
sudo virt-customize -a xenial-server-cloudimg-amd64-disk1.img \
	--run-command 'apt-get update' \
    --install software-properties-common,aptitude,git,wget,build-essential,python-dev,python-pip,net-tools,iproute,inetutils-ping,curl,iperf3,iperf,tcpreplay,bmon,nmap,python,python-yaml,siege,httperf,gettext,apache2-utils,hping3,polygraph,libfuse-dev,stress \
    --run-command 'apt-add-repository -y ppa:mosquitto-dev/mosquitto-ppa' \
    --run-command 'apt-get update' \
    --run-command 'apt-get install -y mosquitto-clients' \
    --run-command 'git clone https://github.com/mpeuster/mqtt-malaria.git' \
    --run-command 'cd /mqtt-malaria; python2 setup.py install' \
    --run-command 'git clone https://github.com/flok99/httping.git' \
    --run-command 'cd /httping; make install' \
    --mkdir /pcaps \
    --run-command 'cd /pcaps; wget https://s3.amazonaws.com/tcpreplay-pcap-files/smallFlows.pcap; wget https://s3.amazonaws.com/tcpreplay-pcap-files/bigFlows.pcap; wget --no-check-certificate -O /normal.pcap https://mcfp.felk.cvut.cz/publicDatasets/CTU-Normal-7/2013-12-17_capture1.pcap; wget --no-check-certificate -O /malware.pcap https://mcfp.felk.cvut.cz/publicDatasets/CTU-Malware-Capture-Botnet-3/2013-08-20_capture-win15.pcap' \
    --mkdir /tngbench_share

Uplaoding python and bash scripts:

# Assuming the files are available in current directory
virt-customize -a xenial-server-cloudimg-amd64-disk1.img \
    --upload start.sh:/start.sh \
    --upload stop.sh:/stop.sh \
    --run-command 'chmod +x start.sh; chmod +x stop.sh'
    --upload process_iperf3_results.py:/process_iperf3_results.py \
    --upload process_ab_results.py:/process_ab_results.py \
    --upload process_boom_results.py:/process_boom_results.py \
    --upload process_httping_results.py:/process_httping_results.py \
    --upload process_malaria_results.py:/process_malaria_results.py \
    --upload log_intf_statistics.py:/log_intf_statistics.py

Editing cloud.cfg file inside the cloud image: (MANDATORY)

virt-customize -a xenial-server-cloudimg-amd64-disk1.img --run-command 'echo "manage_etc_hosts: true" >> /etc/cloud/cloud.cfg'

Uploading image to OpenStack:

openstack image create --disk-format qcow2 --container-format bare --public --file ./xenial-server-cloudimg-amd64-disk1.img tng-bench-mp

Getting Detailed View on the Filesystem of a Cloud Image

guestfish --rw -a xenial-server-cloudimg-amd64-disk1.img
run
list-filesystems
# Output:
# /dev/sda1: ext4
mount /dev/sda1 /
ls /
# Now you can use basic linux commands to edit and look up files

Increasing Volume

If you get an error because of volume shortage in your cloud image:

# Checking how much space is used
sudo virt-df -a xenial-server-cloudimg-amd64-disk1.img
# If "Use%" is too high, you need to resize and follow the following steps

# Create an empty image with 5G size
qemu-img create -f qcow2 new-cloud-image.img 5G

# Get the filesystem of original cloud image which has to be resized
virt-filesystems --long --parts --blkdevs -h -a xenial-server-cloudimg-amd64-disk1.img
# Output:
# Name       Type       MBR  Size  Parent
# /dev/sda1  partition  83   2.2G  /dev/sda
# /dev/sda   device     -    2.2G  -

# Copying the partition "/dev/sda1" from original cloud image to new cloud image
virt-resize --expand /dev/sda1 xenial-server-cloudimg-amd64-disk1.img new-cloud-image.img

Note:

• MANDATORY:
  • You have to edit the cloud.cfg file always to get the images work properly and obtain their hostnames
• If you get an error while deploying the image on OpenStack, try not creating the volume at the time of instantiation.
  Launch Instance > Source > Create New Volume > Select "No"