meta-GOP: Our tool for extracting transcoding metadata

In our HPCC laboratory, my student Xiangbo Li and I, have recently developed the meta-GOP, a tool that can provide researchers in Cloud and Distributed computing with realistic workload data. Before explaining the tool, I need to describe our current research project a little bit.

Video content, either in form of on-demand streaming (e.g., YouTube or Netflix) or live-streaming (e.g., Livestream), needs to be converted based on the characteristics of the clients devices. That is, the original video has to be converted to a supported resolution, frame rate, video codec, and network bandwidth of the clients devices. The conversion is termed video transcoding, which is a computationally heavy and time-consuming process. A Video stream consists of several sequences. Each sequence is divided into multiple Group Of Pictures (GOP) with sequence header information at front. GOP is essentially a sequence of frames beginning with an
I (intra) frame, followed by a number of P (predicted) frames or B (be-directional predicted) frames. To transcode a video format to another (e.g., to another spatial resolution), each GOP should be processed independently.

Our current research is on how to execute video stream transcoding on Cloud virtual Machines (VMs). More specifically, in the HPCC lab at University of Louisiana Lafayette, we are currently researching on how to optimally schedule GOPs on Cloud VMs. We utilize Cloudsim tool to simulate the Cloud infrastructure. However, we needed to generate realistic transcoding processing information to construct our workload. Therefore, we developed a tool that receives video files in any format as input, performs different types of transcoding operations on the videos at the GOP level, and generates GOP execution information. This information can then simply be fed to Cloudsim for simulation.

The workload information generated by meta-GOP can be useful for people researching on video transcoding and those who work on scheduling independent tasks in any type of parallel or distributed system.

Meta-GOP is open-source and can be downloaded from here. Along with the source code, we provide a Jar file that can be simply imported to your Java project. Then, you can invoke a wide range of transcoding operations for a single video file and produce different types of workload. Finally, we provide a stand-alone Jar file that can be run independently. You can download the runnable Jar file from here. More information about how to utilize each of these files are accessible here.

People who use this tool are expected to cite our initial publication that yielded to generating this tool:
“Xiangbo Li, Mohsen Amini Salehi, Magdi Bayoumi, Cloud-Based Video Streaming for Energy- and Compute-Limited Thin Clients, Presented in Stream2015 Workshop at Indiana University, Indianapolis, USA, Oct. 2015.”

How to Enable TPM in Linux, Communicate with it, and check the PCR values

Tags

This post is written with the help of Henry Kironde.

Most systems these days have the Tpm build on the motherboard. However, it is not usually enabled by default. Therefore, we need to enable the TPM in the BOIS.

For that do the following, as the system starts:

1) Press F2 >> Security >> TPM >>activate

Then, reboot your system

2) Check to see if the models are supported:

ls -la /lib/modules/`uname -r`/kernel/drivers/char/tpm

drwxr-xr-x 2 root root  4096 Aug  8 10:32 .

drwxr-xr-x 8 root root  4096 Aug  8 10:32 ..

-rw-r–r– 1 root root 12852 Jul 14 23:31 tpm_atmel.ko

-rw-r–r– 1 root root 18708 Jul 14 23:31 tpm_i2c_infineon.ko

-rw-r–r– 1 root root 25468 Jul 14 23:31 tpm_infineon.ko

-rw-r–r– 1 root root 14100 Jul 14 23:31 tpm_nsc.ko

-rw-r–r– 1 root root 30948 Jul 14 23:31 tpm_tis.ko


3) To enable the modules for  TPM use the following command.

modprobe tpm_tis.ko

modprobe tpm_bios

modprobe tpm

4) Use the following to see if the models have been loaded

dmesg | grep -i tpm

Communicating with the TPM

1) Download trousers and install it. Use root so that you eliminate most of the installation problems.

sudo apt-get update

sudo apt-get install aptitude

aptitude install trousers tpm-tools


2) Start Trousers:

/etc/intit.d/trousers

root@ubuntu:/home/amini# /etc/init.d/trousers start

You should see this response:

* Starting Trusted Computing daemon tcsd                                [ OK ]

you will be notified if it fails( fail meaning that the daemon is not started)

Once the TPM is enabled, boot of the system creates a /dev/tpm0 device file.

3) Read the TPM version

amini:~$ tpm_version

This is the output on my machine:

TPM 1.2 Version Info:

Chip Version:        1.2.3.91

Spec Level:          2

Errata Revision:     3

TPM Vendor ID:       WEC

TPM Version:         01010000

Manufacturer Info:   57454300

4) Take ownership of the TPM

$tpm_takeownership

 

5) Read PCR values :

cat /sys/class/misc/tpm0/device/pcrs

PCR-00: AD B5 A1 6B F5 42 CA 9D 0F EA 7A 60 94 81 53 F8 E0 42 E6 B6

PCR-01: 3A 3F 78 0F 11 A4 B4 99 69 FC AA 80 CD 6E 39 57 C3 3B 22 75

PCR-02: 0E AA F8 1E 92 C7 84 F9 9C BB C1 D3 72 12 9D DD DA 30 6E 5A

PCR-03: 3A 3F 78 0F 11 A4 B4 99 69 FC AA 80 CD 6E 39 57 C3 3B 22 75

PCR-04: 47 0E D4 44 DE 46 2C FC 17 5E 3C 68 8D 79 A3 2B 97 30 DF 13

PCR-05: 81 96 5F 15 B0 6D 54 56 18 FA E1 51 F1 48 B3 02 D5 08 E9 21

PCR-06: 3A 3F 78 0F 11 A4 B4 99 69 FC AA 80 CD 6E 39 57 C3 3B 22 75

PCR-07: 3A 3F 78 0F 11 A4 B4 99 69 FC AA 80 CD 6E 39 57 C3 3B 22 75

PCR-08: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

PCR-09: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

PCR-10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

PCR-11: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

PCR-12: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

PCR-13: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

PCR-14: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

PCR-15: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

PCR-16: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

PCR-17: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF

PCR-18: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF

PCR-19: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF

PCR-20: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF

PCR-21: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF

PCR-22: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF

PCR-23: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

Tips on how to write a professional research paper

In this article I have compiled a few critical points that I have learnt in writing professional research papers. This post will be updated as I find/learn new tricks in future.

These points are particularly useful for the researchers in computer science/engineering area. However, some of these points can be broadly applied to any scientific paper in any research area.

  1. Pay particular attention to references. Keep them as complete and consistent as possible. Reviewer care about references. They would like to see their papers have been cited neatly. Readers also care about it. They sometimes judge the quality of the paper based on the quality of the organization in the paper.
  2. Writing is so important. there are many points regarding writing. However, the minimum is that be consistent across the paper about the terms and tenses you use. Be specific! avoid general terms such as “some” or “in this situation” instead be more specific and clearly say what is “some” and what is the situation.
  3. Care about organization of the paper. Consider a nice flow in the Section titles so that a reviewer/skimmer can easily track the paper by reading the titles.
  4. Also be careful about paragraphing each section. Do not just use paragraphs randomly or just to breaking a long column of text. Each paragraph should convey a main message. The message usually appear in the beginning of the paragraph. The rest of the paragraph just “support” that main message. Avoid one-sentence-long paragraphs. Avoid sections with one paragraph.
  5. Figures are effective. It makes the paper less boring and easier to understand. The organization of the figures should be in a way that a reviewer/reader can understand an abstract of the paper by just going across figures in a paper. Do not reserve in writing long caption (description) for the figures. Again, the reviewer should understand the story by looking at the pictures and reading their descriptions.
  6. Conclusion should have a “take-away” message. That is, the reader/reviewer should understand the main message of the work by just going over the conclusion (and possibly abstract).
  7. Write down clearly the list of contributions somewhere at the end of the Introduction section. Remember that no reviewer would have time to go over the whole text. They just look for particular points in particular spots…If you miss addressing the right thing at the right spot, it would be hard to convince the reviewer to accept your paper.
  8. Try to be clear as much as possible. This is a common illusion to make the paper vague so that the reviewer do not understand it and “accept” it. This is rare! reviewers commonly enjoy and accept, when they read something in a short time and understand it clearly.

Configuring Virtual TPM (vTPM) for Xen 4.3 Guest Virtual Machines.

Tags

, , , ,

I recently had to spend so much time to get vTPM working in XEN guest virtual machines. Here, I am going to share this experience so that others can follow this steps quickly and be able to work with vTPM (virtual trusted platform modules) as quick as possible.

*. According to the Xen 4.3 documentation, the vTPM currently just works for “para-virtualization” (PVM).

**. The following instructions are tested for Ubuntu 12.4 (as host), Xen 4.3, Linux Kernel 3.7.1 for Dom0, and Linux Kernel 3.7.9 for DomU kernel.

***. Please run all the commands in this tutorial as root (with sudo).

Prerequisites

  1. Having TPM chip on the motherboard. The TPM hardware must be activated through Bios.
  2. Having a Linux host  installed on the machine (I used ubuntu 12.4)
  3. The reader should have the basic knowledge of the Linux kernel and knows how to compile that. We have to compile linux kernel to have a customized Dom0 and Domu in Xen.
  4. Users may want to have more detailed information on the structure and architecture of vTPMs can refer to the following sources:
    IBM research:
    http://researcher.watson.ibm.com/researcher/view_project.php?id=2850
    IBM initial publication in Usenix 2006:
    https://www.usenix.org/legacy/event/sec06/tech/full_papers/berger/berger.pdf
    Xen Wiki page:
    http://xenbits.xen.org/docs/unstable/misc/vtpm.txt

High level Installation steps

Activating vTPM in Xen includes the following main steps. In the next part we explain the details of each step.

  1. Install a host operating system on the machine (we used Ubuntu 12.4)
  2. Installing Xen 4.3 hypervisor
  3. Install Dom0 kernel
  4. Install DomU kernel
  5. Configuring vTPM manager and vTPM
  6. Booting DomU with vTPM

In the following sections we describe each step in details.

1. Install a host OS

We assume that this step is already done as described in the prerequisites.

2. Installing Xen 4.3 hypervisor

Instructions of this section are based on the instructions in the Xen Guide and this blog. However, some steps has changed/added to enable vTPM in the Xen.

So, to install Xen 4.3 follow these steps:

1. Install all required packages. The list are as below. However, I suggest to install them one by one.

$sudo apt-get install bcc bin86 gawk bridge-utils iproute libcurl3 libcurl4-openssl-dev bzip2 module-init-tools transfig tgif texinfo texlive-latex-base texlive-latex-recommended texlive-fonts-extra texlive-fonts-recommended pciutils-dev mercurial build-essential make gcc libc6-dev zlib1g-dev python python-dev python-twisted libncurses5-dev patch libvncserver-dev libsdl-dev libjpeg62-dev iasl libbz2-dev e2fslibs-dev git-core uuid-dev ocaml libx11-dev bison flex xz-utils ocaml-findlib gcc-multilib checkpolicy

Install the following packages:

$ sudo apt-get install yajl*,pixman*

(by * I mean all the packages that have these words)

2. Download xen 4.3 source code:

$ wget http://bits.xensource.com/oss-xen/release/4.3.0/xen-4.3.0.tar.gz

3. Extract Xen4.3:

$ sudo tar xvf xen-4.3.0.tar.gz
$ cd xen-4.3.0
$ sudo vim Config.mk

4. Change the line in the open file as follows:

XSM_ENABLE ?=y

5. Install Xen:

$ sudo make xen
$ sudo ./configure
$ cd tools
$ sudo ./configure
$ cd ..
$ sudo make tools
$ sudo make stubdom
$ cd stubdom
$ sudo make
$ cd ..
$ sudo make install-xen
$ sudo make install-tools PYTHON_PREFIX_ARG=
$ sudo make install-stubdom

6. Since we use XSM/Flask security framework in Xen, and we need xenpolicy.24 later. You should run the following command in the xen4.3 directory:

$sudo make -C tools/flask/policy

7. If everything has been done correctly, you should see the following files in /boot directory:

/boot/xen.gz
/boot/xen-4.gz
/boot/xen-4.3.gz
/boot/xen-4.3.0.gz

8. Also check the existance of the following files:

/usr/local/lib/xen/boot/vtpmmgr-stubdom.gz
/usr/local/lib/xn/boot/vtpm-stubdom.gz

*. if you cannot see these files/path, please check the existence of “local” directory in your system. You may not have it based on your host installation. In that case check /usr/lib/…

9. Edit “/etc/xen/xend-config.sxp” file and change it as follows:

(xend-unix-server yes)

3. Install Dom0 kernel

After installing Xen, we should install Dom0 which is, in fact, a default virtual machine created by Xen that takes care of other VMs in the system. To support vTPM we need to change some installation configuration of the Linux kernel and then compile the kernel. Please follow the below steps to install and configure that:

1. Download the kernel:

$ wget http://www.kernel.org/pub/linux/kernel/v3.0/linux-3.7.1.tar.gz

2. Extract the kernel

$ tar xvf linux-3.7.1.tar.gz

3. Configure/customize the kernel

$cd linux-3.7.1
$ sudo make menuconfig

4. Assure about the following kernel configurations

Processor type and features →
    High memory support (64GB)
       PAE (Physical Address Extension) Support - enabled

Processor type and features →
    Allocate 2nd-level pagetables from highmem - disabled

ACPI (Advanced Configuration and Power Interface) Support - enabled

Processor type and features →
     Paravirtualized guest support [y] →
          Xen guest support – enabled

Bus oprions-
     Xen PCI frontend – enabled

Device Drivers → 
     Block Devices [*] → 
           Xen virtual block device support – enabled 
           Block-device backend driver – enabled 

     Network device support [*] → 
           Xen network device frontend driver – enabled 
           Xen backend network device – enabled

     Input device support →
           Miscellaneous devices →
                Xen virtual keyboard and mouse support – enabled

     Character devices →
           Xen Hypervisor console support – enabled

     Xen driver support →
           Xen memory balloon driver – enabled
           Scrub pages before returning them to system – enabled
           Xen /dev/xen/evtchn device Backend driver support – enabled
           Xen filesystem – enabled
           Create compatibility mount point /proc/xen – enabled
           Create xen entries under /sys/hypervisor – enabled
           userspace grant access device driver – enabled
           User-space grant reference allocator driver – enabled
           xen platform pci device driver – enabled

5.DISABLE TPM SUPPORT (this is a critical step- dom0 should not have access to the TPM).

For that purpose, we exclude the TPM driver in the dom0 kernel:

Device Drivers —> Character devices —><  > TPM Hardware Support

6. After the above modifications, make sure about the following configurations in .config file in the Linux-3.7.1 directory:

CONFIG_ACPI_PROCFS=y 
CONFIG_XEN=y 
CONFIG_XEN_MAX_DOMAIN_MEMORY=32 
CONFIG_XEN_SAVE_RESTORE=y 
CONFIG_XEN_DOM0=y 
CONFIG_XEN_PRIVILEGED_GUEST=y 
CONFIG_XEN_PCI=y 
CONFIG_PCI_XEN=y 
CONFIG_XEN_BLKDEV_FRONTEND=y 
CONFIG_XEN_NETDEV_FRONTEND=y 
CONFIG_XEN_KBDDEV_FRONTEND=y 
CONFIG_HVC_XEN=y 
CONFIG_XEN_FBDEV_FRONTEND=y 
CONFIG_XEN_BALLOON=y 
CONFIG_XEN_SCRUB_PAGES=y 
CONFIG_XEN_DEV_EVTCHN=y 
CONFIG_XEN_GNTDEV=y 
CONFIG_XEN_BACKEND=y 
CONFIG_XEN_BLKDEV_BACKEND=y 
CONFIG_XEN_NETDEV_BACKEND=y 
CONFIG_XENFS=y 
CONFIG_XEN_COMPAT_XENFS=y 
CONFIG_XEN_XENBUS_FRONTEND=y 
CONFIG_XEN_PCIDEV_FRONTEND=y

7. Start installation of the kernel.

$ sudo make modules_prepare
$ sudo make
$ sudo make modules_install
$ sudo make install
$ cd /boot
$ sudo mkinitramfs -o initrd.img-3.7.1 3.7.1
$ sudo update-grub

8. Reboot the system and you should be able to start the Xen as follows:

$ sudo service xencommons start

9. If Xen is started correctly, you should be able to run the following command and see dom0 running:

$ sudo xl list

Also, if you execute “cat /proc/xen/capabilities”, the output must be: “control_d”.

*. If the Xen service cannot be run, then you may fix the problem by installing “blktab-dkms”:

$ sudo apt-get install blktab-dkms

Then append “blktab” to the /etc/modules file. It may solve the problem.

10. copy the xenpolicy.24 file to the /boot directory:

$ sudo cp tools/flask/policy/xenpolicy.24 /boot

11. modify /boot/grub/grub.cfg (this file is read only even for the owner.  So, first change the permissions: $ sudo chmod 744  /boot/grub/grub.cfg). Find the menuentry that you select at boot time (it should be “xen xen and linux 3.7.1”) and add the following lines to the end of it (after initrd.img-3.7.1):

echo    ‘Loading xenpolicy.24…’
module  /boot/xenpolicy.24

*. It is better to change the permissions of the grub.cfg after saving the changes (to 444).

12. Reboot the system and now you should be able to run the Xen service.

*. if you again face a problem, with this error message:

“Starting oxenstored…/usr/local/sbin/oxenstored: error while loading shared libraries: libxenctrl.so.4.3: cannot open shared object file: No such file or directory”

then, the following fix may help you:

sudo vim /etc/ld.so.conf
include /usr/lib64
sudo ldconfig
sudo service xencommons start

13. Configure networking in Xen:
This section describes how to set up linux bridging in Xen. It assumes eth0 is both your primary interface to dom0 and the interface you want your VMs to use. It also assumes you’re using DHCP.

sudo apt-get install bridge-utils

Edit /etc/network/interfaces, and make it look like this:

auto lo
iface lo inet loopback

auto xenbr0
iface xenbr0 inet dhcp
bridge_ports eth0

auto eth0
iface eth0 inet manual

Restart networking to enable xenbr0 bridge:

sudo /etc/init.d/networking restart

4. Install DomU kernel

2 main things should be done:

A) Kernel for new domain
B) File system for new domain (creating an image)

1. Download rectified kernel from github. Unfortunately, the current Linux kernels are not yet entirely adapted for the vTPM.
Quan Xu, a friend from Intel who helped me in this project, has modified the kernel code and adapted that to support vTPM. You can download the modified kernel, from the below address:
https://github.com/virt-cloud/domuKernel

3. Use “sudo make menuconfig” and modify the configuration. In the final  form of .config file must look like this:

CONFIG_XEN=y
CONFIG_PARAVIRT_GUEST=y
CONFIG_PARAVIRT=y
CONFIG_XEN_PVHVM=y
CONFIG_XEN_MAX_DOMAIN_MEMORY=128
CONFIG_XEN_SAVE_RESTORE=y
CONFIG_PCI_XEN=y
CONFIG_XEN_PCIDEV_FRONTEND=y
CONFIG_XEN_BLKDEV_FRONTEND=y
CONFIG_XEN_NETDEV_FRONTEND=y
CONFIG_INPUT_XEN_KBDDEV_FRONTEND=y
CONFIG_HVC_XEN=y
CONFIG_XEN_FBDEV_FRONTEND=y
CONFIG_XEN_DEV_EVTCHN=y
CONFIG_XEN_XENBUS_FRONTEND=y
CONFIG_TCG_TPM=y
CONFIG_TCG_XEN=y

4. Enable IMA (security options–> Integrity Measurement Architecture(IMA)) and TPM (Device Drivers–>Character Devices–>TPM hardware support) in the menuconfig

5. compile the kernel

$ sudo make

6. Create empty disk of 10 GB with following commands

$ sudo dd if=/dev/zero of=/root/domu.img bs=1024K count=10240

7. Format this file system

$ sudo /sbin/mkfs.ext4 /root/domu.img

8. Mount the file system

$ sudo mount –o loop /root/domu.img /mnt/

9.  Install a base system. Here I am going with Debian Squeeze. For this you need to mount disk image to any directory and then use “debootstrap” command for populating image with Debian Squeeze base system. Since my system is 64 bit the command is:

$ sudo debootstrap --arch amd64 squeeze /mnt/

*. after mount and debootstrap operations completed, umount the created file system and mount it again to make sure that the file system is copied correctly.

10. At this point you should introduce the file systems to the system. For this purpose Edit /mnt/etc/fstab and copy the following lines in that

/dev/xvda1 / ext4 defaults 0 1
proc /proc proc defaults 0 0

11. Choose your guest name by editing /mnt/etc/hostname

12. Edit /mnt/etc/network/interfaces and make it look like below

auto lo
iface lo inet loopback
auto eth0
iface eth0 inet dhcp

13. Edit /mnt/etc/securetty and append

hvc0

14. Do the following steps to create xvda and hvc0:

chroot /mnt
mknod /dev/xvda1 b 202 1
mknod /dev/hvc0 c 229 0
chown root:disk /dev/xvda1
exit

15. Edit /mnt/etc/inittab and comment out the following lines

# 1:23:respawn:/sbin/getty 38400 tty1
# 2:23:respawn:/sbin/getty 38400 tty2 
# 3:23:respawn:/sbin/getty 38400 tty3 
# 4:23:respawn:/sbin/getty 38400 tty4 
# 5:23:respawn:/sbin/getty 38400 tty5 
# 6:23:respawn:/sbin/getty 38400 tty6

and add the following line:

hvc0:2345:respawn:/sbin/agetty -L 9600 hvc0

16. Create user and set password for root

$ sudo chroot /mnt
# adduser xen
# passwd root
# exit

17.Now we need to install kernel modules in newly created file system. We have downloaded and compiled kernel in step 1, at path /root/linux-3.9.1

$ cd YOUR PATH/linux-3.9.1
$ sudo make modules_install INSTALL_MOD_PATH=/mnt
$ sudo cp /root/linux-3.9.1/.config /mnt/boot/config-3.9.1

18. We need initrd file which is initial ram disk

$ sudo chroot /mnt
$ apt-get install initramfs-tools
$ mkinitramfs -o initrd.img-3.9.1-domU 3.9.1
$ exit
$ mv /mnt/initrd.img-3.9.1-domU /root/

19. Unmount the file system of DomU

sudo umount /mnt

5. Configuring vTPM Manager and vTPM

At this point we have all the necessary platforms ready. To use  vTPM in the guest VMs we have to first create a vTPM Manager.
The vTPM Manager is, in fact, a domain for itself and creates and manages vTPMs (TPMs for each VM). After successfully creating vTPM Manager, we can create vTPM instances that will serve as TPM for each VM. Finally, the guest VM is created and is connected to the vTPM. More detailed description on this structure can be found here. Below, we list the operations to achieve these steps.

5.1 Creating vTPM Manager

1. Creating disk image for vTPM Manager domain. The vTPM Manager requires a disk image to store its encrypted data. The image does not require a filesystem and can live anywhere on the host disk. The image does not need to be large. 16 Mb should be sufficient.

$ sudo dd if=/dev/zero of=/var/vtpmmgr-stubdom.img bs=16M count=1

2. Creating config file for the vTPM Manager domain. The vTPM Manager domain (vtpmmgr-stubdom) is launched like any other Xen VM and requires a config file. The manager requires a disk image for storage and permission to access the hardware memory pages for the TPM. An example configuration is as follows.

kernel="/usr/local/lib/xen/boot/vtpmmgr-stubdom.gz"
memory=16
disk=["file:/var/vtpmmgr-stubdom.img,hda,w"]
name="vtpmmgr"
iomem=["fed40,5"]

*. iomem line in the above config file says that the vTPM Manager should be able to access the IO memory pages of hardware TPM (that are 5 pages starting from 0xfed40000)

3. Launching the vTPM Manager. Like any other VM, the vTPM Manager can be launched:

$ sudo xl create -c <vTPM_Conf_File>

*. Assuming that vTPM_Conf is the name of the created config file.

**. If everything works well you should see the following line after the vTPM Manager booted correctly.

INFO[VTPM]: Waiting for commands from vTPM's:

***. It seems that the vTPM implementation is not very stable yet and it looks very fragile. Therefore, to shut down the vTPM Manager at any time, it is suggested that the vTPM Manager is destroyed carefully after it is truly booted  to prevent any disk (img) corruption. You can do that in xl toolstack: “sudo xl destroy <vTPM_ID>”.

5.2 Creating vTPM Manager

After creating vTPM Manager, we can create vTPM for a guest VM.

1. Creating a disk image for vTPM. Similar to vTPM Manager, vTPM is also a VM and therefore, it requires a disk image in the first place.

$ sudo dd if=/dev/zero of=/home/user/domu/vtpm.img bs=8M count=1

2. Creating vTPM config file:

kernel="/usr/local/lib/xen/boot/vtpm-stubdom.gz"
memory=8
disk=["file:/home/user/domu/vtpm.img,hda,w"]
name="domu-vtpm"
vtpm=["backend=vtpmmgr,uuid=XXXXXXX"]

*. You should replace UUID with your own, by running “uuidgen” in Linux.

**. The line vtpm=[…] is the front end TPM driver. It mentions that the vTPM is using the vtpmmgr in the background as the vTPM Manager.

3. Launching a VM based on the constructed config file.

$ sudo xl create -c <TPM_Conf_File>

*. After booting the vTPM, you should see the vTPM instance stops on the following line:

<pre>

Info: TPM_Startup(1)

</pre>

6 Booting DOmU (Guest VM) with vTPM Support

After starting vTPM Manager and VTPM, it is the time to fire a VM. We create a VM from the image we created with kernel 3.9.1.

1. In the first place we should create a VM config file. The config file looks like any normal VM config file. However, in the last line, we should include the connection to the vTPM. Here is an example.

kernel = '/home/user/domuKernel-master/vmlinux'
ramdisk = '/root/initrd.img-3.9.1-domU'
vcpus = '1'
memory = '1024'
root = '/dev/xvda1 ro'
disk=['tap:aio:/root/domu9.img,xvda1,w']
name = 'storage'
vif = [ '','bridge=xenbr0']
dhcp = "dhcp"
on_poweroff = 'destroy'
on_reboot = 'restart'
on_crash = 'restart'
extra = 'console=hvc0 xencons=tty'
vtpm=["backend=domu-vtpm"]

*. If you are copy/pasting the config file, please remember that you should modify the addresses based the locaiton of images etc. on your computer.
**. the last line (vtpm=[…]) shows the connection to the vTPM.

2. In this step, you can create the VM:

$ sudo xl create -c VM_CONFIG_FILE

*. during the boot process you should see the interaction of the VM with vTPM

3. After booting, make sure that the xen-tpmfront has been loaded in the guest VM. For that purpose, run the following command:

# modprobe xen-tpmfront

*. After this module is loaded, you should see in the vTPM that the front end is connected.

4. To test the TPM on the guest you should install “trousers” and “tpm-tools” using apt-get.

5. On the guest VM run the following

# tcsd
# tpm_version

Here, is the output on my computer:

TPM 1.2 Version Info:
Chip Version:        1.2.0.7
Spec Level:          2
Errata Revision:     1
TPM Vendor ID:       ETHZ
TPM Version:         01010000
Manufacturer Info:   4554485a

And everything works fine now!

JUnit slides and code

Tags

,

Today, I taught hands on course on JUnit.

The hands on part was quite cool! I incorporated many issues into the demo.

We covered specifically the following issues:

  • performance evaluation
  • exception handling
  • how to test methods that return an object (not primitives).

You can find the slides in this address:

http://goo.gl/HPxqmK

The JUnit demo is available here:

http://goo.gl/vpvNUR

Enjoy!

C/C++ in eclipse in Mac os X

Tags

, ,

Today, I was trying to develop C/C++ code in Mac OS using eclipse. However, I could not find any resource that properly tells me how to install gcc in mac as a prerequisite for developing C/C++ code in eclipse. Therefore, I am leaving this piece of note here for other people to use and describe the easy! problem of making GCC and eclipse work together in Mac OSX. So, please follow the following steps if you have this problem:

1- easiest way to install C/C++ is installing gcc through Xcode in Mac. Do that through “App store”. Search Xcode in “App Store” and follow steps to install it.

2- However, installing Xcode does not necessarily mean gcc is installed. So, do the following to make sure that gcc is installed. In Xcode menu bar, go to Xcode>preferences. Install “Command Line Tools” (click the install button in front of this choice). 

3- To make sure that gcc is installed, open a Terminal and key in: gcc -v

if everything is installed correctly, you should get some lines that shows the version and other details of the installed gcc.

4- Install eclipse from http://www.eclipse.org/downloads/ (preferably select the version for C/C++).

5- Open eclipse and set up a workspace directory somewhere on your disk.

6- Create a new project in eclipse: file>new>C Project. Key in a “project name”.

Under the “project name” and “location”, there is a box called “Project Type”. Make sure that for “executable”, you select “Empty Project” and in the right side box, make sure that you select “MacOSX GCC”. I experienced that selecting the other option (i.e., Cross GCC) does not allow eclipse to run your project. In fact, by choosing this wrong option you’ll get the annoying message of “Launch fail. Binaries are missing” when you try to run your project.

I hope this piece of note helps.

Phone interview experience

Today, I had a phone interview experience. I would like to share with you questions I was asked.

The interviewer called me with 10 minutes delay and started by introducing himself, his job and then he asked me to do the same…

After that, the technical part started. A GoogleDoc file was shared between the interviewer and me and I had to do coding (Java in my case) within that document.

For the first question, he wrote an the below interface in the Google doc file:

interface Counter {

public void increment();

public int lastSecond();

public int lastMinute();

public int lastHour();

public int lastDay();

}

Then he asked me to implement that. He explained that the code is suppose to increase the number of hit per visiting a web page (this is increment() function). However, other functions e.g., lastsecond(), lastMinute() suppose to count number of hits occurred during  the last second, minute and so on…

He also asked about the type of data structure I am going to use.

After sometime, we moved to the second question which was easier for me. The second question was adding numbers that are stored in an array. I think the following example clarifies what I mean:

2179: [ 2, 1, 7, 9 ] => [ 2, 1, 8, 0 ]

 So, as we see 2179+1=2180

This one easier and I think I did a good job on this within that limited time.

Overriding Opennebula scheduler (mm_sched)

Today, I was trying to enforce my scheduling policies of Haizea when it is connected to Opennebula. However, surprisingly, it was not working. In fact, Haizea was scheduling the leases, but another scheduler was overriding the scheduling.

 

I found out that the reason is the Opennebula built-in scheduler which is called mm_sched. However, after googling a lot I could not find a proper way to disable the scheduler. Therefore, I paused the scheduling process manually.

If you have the same problem, follow these steps (in Ubuntu 12.4):

  1. ps aux|grep mm_sched
  2. You should see the mm_sched as a process as well as its process id (pid)
  3. kill -STOP pid

Now you can run the Haizea again and this time it will “hopefully” works!

Fixing a problem in Opennebula 3.4

As I mentioned recently, I have installed Opennebula and KVM on my local host (Ubuntu 10.4) for some testing purposes.

After instantiating a new VM, it was going through pend, prolog, boot but then suddenly FAIL with the following error in the oned.log file:

“/var/tmp/one/vmm/kvm/deploy /var/lib/one//datastores/0/2/deployment.0 localhost 2 localhost”

I fixed the problem in this way:

  1. sudo vim /etc/libvirt/qemu.conf (it can be with oneadmin user or any sudo enabled user)
  2. change: user = “oneadmin”—>user=”root” and group = “oneadmin”–>group = “root”
  3. save and exit the file.
  4. Restart libvirt: sudo service libvirt-bin stop and then sudo service libvirt-bin start
  5. stop/start opennebula: one stop then one start

Now, it should work fine.

Hope this helps…

onevm shutdown does not work

For few days I was struggling with making “onevm shutdown” command works in OpenNebula3.4.

It is a bug in OpenNebula or KVM I believe and  specifically happens when we have combination of:

Ubuntu+KVM+Openebula.

Anway, I could not solve it 😦 and therefore resorted to a kind of hack! In fact, I replaced the command with with “onevm cancel”  which is similar to shutdown (although it is not the same!) but it worked for me!

Particularly, I debugged Haizea with replacing

return self.vm_action(“shutdown”, vid)

In haizea/common/opennebula_xmlrpc.py with:

return self.vm_action(“cancel”, vid)

 i wish it will also work for you.