Posts Tagged ‘virtualbox’

Share folders with Virtual machine using Virtualbox

Wednesday, January 4th, 2012

Since we use Windows Vista as default, we are used to load virtual machines to try linux stuff, and it will be nice if we can share folders between host (windows) and guest (linux) machines.

The document “shared folders” had some traps, and I can’t make it work, below are list some important steps to get it work

Envrionment

Ubuntu 11.04 (Natty) as guest OS and Virtualbox as virtual machine management tools

Install Guest Additions

Virtualbox Guest Additions should be installed for shared folders, it is a .iso files to be mounted, when it is mounted in guest OS (Ubuntu), it can be executed

image

Install linux kernel

I met the following error when I try to install the guest additional

[guest] # cat /var/log/vboxadd-install.log
Makefile:23: *** Error: unable to find the sources of your current Linux kernel. Specify KERN_DIR=<directory> and run Make again..  Stop.

After googled, I found the good article http://blog.oracle48.nl/adding-shared-folders-to-virtual-box-guest-using-the-command-line/, later I noticed it existed inside chapter 4 as well.

Therefore I need to install dkms package and related headers (it depends on which kernel you use)

apt-get install dkms
apt-get install linux-headers-2.6.38-11-generic
Shared folder from host to guest

it can be easily configured in virtual machines’ setting, if you set it automatically, it will exist when reboot

image

If it goes well, you will see it /media/sf_sdcamp, which works if you run in root id, and as documented, you can also mount it for normal user.

mount –t vboxsf sdcamp /mnt/sdcamp

/mnt/sdcamp should be created in advance

Anyway, now it works very well, I edited in Windows Environment, and run some specical commands under linux environment

create virtualbox on fly using veewee

Tuesday, October 25th, 2011

one headache of using vagrant is to have base box, and creating basebox from scratch is headache, and how to solve it ?

You want to create by yourself for some framework, please wait for a minute, there already had one project veewee, and I don’t want to repeat the nice blog http://www.ducea.com/2011/08/15/building-vagrant-boxes-with-veewee/ as well.

Below just some tips when I tried this morning

veewee tips

veewee directory structure

I prefer to create one seperate veewee directory for this tasks.
iso files should be download in advance, and make a softlink under created directory iso.
then you can define your own basebox like below

~/veewee$ vagrant basebox define 'ubuntu1104' 'ubuntu-11.04-server-i386'

the directory will looks like below, you type command under veewee directory

~/veewee$ find
.
./definitions
./definitions/myubuntubox
./definitions/myubuntubox/postinstall.sh
./definitions/myubuntubox/preseed.cfg
./definitions/myubuntubox/definition.rb
./definitions/ubuntu1104
./definitions/ubuntu1104/postinstall.sh
./definitions/ubuntu1104/preseed.cfg
./definitions/ubuntu1104/definition.rb
./iso
./iso/ubuntu-11.04-server-i386.iso
virtualbox headless

suppose it should work headless (without GUI), while when I create basebox, it pops error on keyboard permisson, and when I enable my X11 display (since I tried it in my local linux server), it pops up virtualbox GUI and start to install.

it is excellent feeling when you see it is under auto-installtion.

configure apt proxy

since I try this in the company, I need to setup the proxy, it is fine to add the line export http_proxy=http://proxy:8080 in the beginning of postinstall.cfg, and it works fine.

look inside of veewee

the architecture of veewee looks great, it adds subcommand in vagrant, and all the majoy information are put into definition.rb

Veewee::Session.declare({
  :cpu_count => '1', :memory_size=> '384',
  :disk_size => '10140', :disk_format => 'VDI', :hostiocache => 'off',
  :o s_type_id => 'Ubuntu',
  :iso_file => "ubuntu-11.04-server-i386.iso",
  :iso_src => "http://releases.ubuntu.com/11.04/ubuntu-11.04-server-i386.iso",
  :iso_md5 => "ce1cee108de737d7492e37069eed538e",
  :iso_download_timeout => "1000",
  :boot_wait => "10", :boot_cmd_sequence => [
    '',
    '/install/vmlinuz noapic preseed/url=http://%IP%:%PORT%/preseed.cfg ',
    'debian-installer=en_US auto locale=en_US kbd-chooser/method=us ',
    'hostname=%NAME% ',
    'fb=false debconf/frontend=noninteractive ',
    'keyboard-configuration/layout=USA keyboard-configuration/variant=USA console-setup/ask_detect=false ',
    'initrd=/install/initrd.gz -- '
  ],
  :kickstart_port => "7122", :kickstart_timeout => "10000", :kickstart_file => "preseed.cfg",
  :ssh_login_timeout => "10000", :ssh_user => "vagrant", :ssh_password => "vagrant", :ssh_key => "",
  :ssh_host_port => "7222", :ssh_guest_port => "22",
  :sudo_cmd => "echo '%p'|sudo -S sh '%f'",
  :shutdown_cmd => "shutdown -P now",
  :postinstall_files => [ "postinstall.sh"], :postinstall_timeout => "10000"
})

it will be used during boot time to trigger the auto-installation, different os will use own auto-installation method, preseed.cfg is used for ubuntu/debian, if for suse then kickstart is used. postinstall.sh is used when the os is installed and rebooted.

summary

with those tool stacks (virtualbox,vagrant,puppet, veewee), your work makes more easily, don’t reinvent the wheel, don’t DIY. Use others brain.

make ci easier with jenkins CI and Vagrant

Friday, October 21st, 2011

Last time, I mentioned the nice tool for vagrant which can control virtualbox virtual machines easily, at once I start to think how it can be used in CI system easily.

Also I noticed the question @ stackoverflow http://stackoverflow.com/questions/6941547/how-to-combine-vagrant-with-jenkins-for-the-perfect-continuous-integration-envir/7830173#7830173, and my suggestion is below

Separate virtual machines environment

Why

Mostly your jenkins server is used for lots of tasks, therefore it is run mostly by restricted user, for example it is tomcat6 if running as servlet, or normal jenkins user, and mostly they don’t have shell access to avoid problem.

Vagrant may needs lots of 3pps which may different with normal jenkins user.

Therefore it is better to separate the virtual machine environment

Jenkins master/slave mode

This is the nice solution to solve this problem to isolate the environment, just new another node in your jenkins server (which is master) as below

image

It is communicated by ssh, and it will invoke the slave.jar automatically. And here I created the vagrant user for this, why ?

Reuse the ssh public/private key

If you played with vagrant, you know vagrant can access virtual machine via ssh without password, it because it uses the ssh public/private key.

Therefore I created the vagrant user and put the private/public key into ~vagrant/.ssh directory, mostly they are under /var/lib/gems/1.8/gems/vagrant-0.8.7/keys/, which is used by vagrant, if you don’t know where it is, please type vagrant ssh_config

image

Prepare the base box under vagrant user

Follow the normal vagrant guideline, and create one vm box, for example, I put it under ~/vm/ubuntu

New the CI job to control VM

Now it is time to configure a job to experience this, in jenkins master node, create a job in freestyle

image

Label Expression is the category we defined in slave node.

In the task (Build – Execute shell), jenkins slave try to go to that directory and start vagrant, then doing some simple tasks to evaulate whether it is done in our vm box.

image

!! REMEBER, this is done in slave mode using vagrant user !!

Now you can see the job is invoked in slave vagrant node

image

In the console output, you can notice the result

image

Summary

Using Vagrant in jenkins master/slave mode is perfect match for this job, you can get lots of flexibility using virtualmachine.
veewee is another layer above vagrant to control vm box more easily.

turnkey solution – Virtual machine for applications

Monday, March 21st, 2011

Just back from vacation, and want to recommend two web sites which I used for application ready in virtual machine image.

imagehttp://www.turnkeylinux.org/ is my favorite place to download the application like redmine, since it will be difficult for me to install RoR in Ubuntu, by using turnkey virtual machine, I can load into virtualbox in seconds !!

And it has good design for the architecture, it will display the text GUI for the most important information (like ip address for different service), then I can easily to access it using web shell.

image image

imageAnother good web site I just notice is http://bitnami.org/ , I didn’t look into so much, it seems it can provide module, so you can install several applications into one, which turnkeylinux.org doesn’t provide.

And also bitnami can install your application into windows, interesting.

Both support Amazon image, it will be a hot area.

Summary

  1. If you want to do demo or some trial, you don’t need to install from scratch, go there and download the virtual machine, and load into your Virtualbox or VMWare directly
  2. We can learn it as well to build our application into virtual machine
  3. We could launch virtual machine for our environment as well, and it is easy to be version controlled as well, I can reload the old environment in fly, can we do it now ?

Will hudson be renamed ?

Monday, December 20th, 2010

image

Shanbo mentioned the new home for hudson http://hudson-labs.org/, and I spend time to look into it.

Hudson is also the de-facto CI tools, and you can take a look at http://nectar.cloudbees.com/about.cb, where you can find the simple history, Kohsuke created hudson in Sun and left (probably after oracle’s buying).

Please read http://hudson-labs.org/content/whos-driving-thing , the hudson community wants to leave oracle’s umbrella (move codes from java.net to github), and kohsuke points out some nice features in github (it is not just a git repositories management tool), lots of developers (include user) vote for the VCS’c changes. Later oracle managers comes out to reject this changes.

What will happen ?? probably hudson will be forked and renamed since the name/trademark is owned by Oracle.

This is good for open source tools, no one can control it.

So how about other open source tools from Sun (thank David for the information), opensolaris is killed, openoffice is forked and renamed to libreoffice (http://www.documentfoundation.org/), virtualbox, mysql, glassfish, let time to see it.

By the way, after google bought instantiation, they donated the excellent plugins (WindowsBuilder and CodePro profiler) to the eclipse community. ;-) , different company has different stratgy and I like google

Generate virtual machine in CI – detail steps

Tuesday, June 22nd, 2010

Let’s progress more for hudson CI, let’s show the script first

   1: user@ubuntu:~/script$ more buildps.sh

   2: #!/bin/sh

   3: echo "Parameter & Env:" $1 $2 $JAVA_HOME

   4: TEMPLATE=/home/user/template

   5: RELEASE=/home/user/release

   6: PACKAGE=/home/user/package

   7: VBoxManage import $TEMPLATE/ub9/ub9.ovf --vsys 0 --vmname "psdemo"

   8: VBoxManage startvm psdemo --type headless

   9: sleep 60

  10: ssh sogadm@192.168.56.121 "ls"

  11: echo "result:" $?

  12: echo "upload packages .."

  13: scp $PACKAGE/webmin_1.510-2_all.deb sogadm@192.168.56.121:/tmp/webmin_1.510-2_all.deb

  14: echo "install packages .."

  15: ssh sogadm@192.168.56.121 "echo 'sogadm' | sudo -S dpkg -I /tmp/webmin_1.510-2_all.deb"

  16: echo "Now it is time to poweroff"

  17: VBoxManage controlvm psdemo poweroff

  18: mkdir -p $RELEASE/psdemo

  19: VBoxManage export psdemo -o $RELEASE/psdemo/psdemo.ovf

  20: echo "clean up .."

  21: VBoxManage storageattach psdemo --storagectl "IDE Controller" --port 0 --device 0 --medium none

  22: VBoxManage unregistervm psdemo --delete

This script is executed in hudson CI, the OS and VirtualPackage name shall be passed as parameters

Preparation
  1. Virtual machine template:

    Line 7: The template virtual machine ub9 (ubuntu 9.10) shall be created in advance, in reality, the 3pp package shall be installed in advance like JAVA, Glassfish, MySQL to save time
  2. SSH auto access by public key

    Since remote access is better to be done via ssh interface, and password can’t be passed in command line, then normal procedure is to create, there are tons of link for this, for example : Pass on Passwords with scp | Linux Journal , anyway when the public key is generated, put it under ~/.ssh/authorized_keys
Procedure in detail
  • Line 7: import the virtual machine template with specific vmname “—vmname”
  • Line 8: start vm in headless mode, it can avoid to start in GUI mode, it is server mode ;-)
  • Line 9: wait for startup, normally it can be checked with some service in script, here is using lazy sleep
  • Line 12-13: upload prepared packages using scp command, it could be replaced with wget command as well, it is push/pull mode

    Also this part can be done in more hack mode, mount the vmdk as file system and copy files directly.
  • Line 14-15: install the package using remote command, “sudo –S” to get password from “echo”, this part can be done using one installation script as well
  • Line 17: poweroff the virtual machine
  • Line 19: export the new virtual machine to release directory
  • Line 21-22: delete the generated virtual machine from virtualbox registry, this part is little tricky, see my answer in VirtualBox community for detail: Need to remove a VM programatically
Result

After quick test in command line, I enabled it in hudson CI system and monitor the consolt out, Woooh, it really works

image image

Now I think I could pass the result to my colleagues to set up completely for real system

How to build virtual image automatically

Monday, May 17th, 2010

If you follow my blog, you know I am fans of automation and virtual technology, now I want to combine them together on How to generate a virtual image during dailybuild

image Benefit shall be simple

- reduce installation time for normal testing, you need not reinstall your application with latest build.
- more deployment scenary, the dailybuild can generate several images automatically with different deployment.

So let’s take a look how to do it technically, I use virtualbox as default

- Launch the small OS image (pre-created) without GUI , with the help of headless start-up in vb

- Upload the package with installation scripts (scp/sftp) the vm
- Trigger the installation via SSH interface

there is post here as well : http://stackoverflow.com/questions/176273/creating-a-virtual-machine-image-as-a-continuous-integration-artifact

See simple bat file for example under windows, once I have linux box, I will try to integrate with hudson

@echo off

set SECONDS=60
:: not set for path in batch
start VBoxHeadless -s tiger -p 8888
@Echo "Let's sleep for %SECONDS% seconds to start up the virtual machines"
:: small sleep loop
FOR /l %%a IN (%SECONDS%,-1,1) do (ECHO 1 >NULL %%as&ping -n 2 -w 1 127.0.0.1>NUL)
pscp -pw user c:\rdccaiy\download\hudson.war user@192.168.56.108:.
plink -pw user user@192.168.56.108 tar zxvf package-1.0.0.tar.gz
plink user@192.168.56.108 -pw user wget -nH http://codeslife.com:10000/anopackage-1.0.0.tar.gz
@Echo "Now the new image is available, kill the process"
VBoxManage controlvm tiger poweroff

tips for virtualbox

Monday, May 17th, 2010

imagePlay with VirtualBox a lot recently and I love it especially for its command line interface, it can give your more power to control the virtual image easily. And it follows the Unix Principle, if it can be done via GUI, there is command to help you.

With the CLI, it can be easily to manage your virtual image more effective, like clone, copy.

Here are some useful commands

   1: compact your disk of your virtual image
   2: >VBoxManage modifyhd ub1.vdi –compact
   3:  
   4: Clone my virtual machine see http://edin.no-ip.com/content/clone-virtualbox-disk-image-ms-windows-mini-howto
   5: >VBoxManage clonehd ub1.vdi ub2.vdi
   6: >or VBoxManage internalcommands setvdiuuid "CopiedDisk.vdi"
   7:  
   8: Start/stop your virtual image, it can help to create simple application to invoke your demo application
   9: >VBoxManage startvm ub1
  10: >VBoxManage controlvm ub1 poweroff 
  11:  
  12: Import/Export virtual image, from 3.1.4, it is better for importing
  13: >VBoxManage export ub1 –o c:\ub1.ovf  # it will create ub1.ovf, ub1.mf, ub1.vmdk totally, no extra compress needed since vmdk is already compressed
  14: >VBoxManage import ub1.ovf 

Other tips on how to copy your virtual image to others

In VirtualBox, when you copy/clone to another virtualbox, MAC address will be changed, so please disable the binding in the Ubuntu configuration

(2) Edit /etc/udev/rules.d/70-persistent-net-generator.rules

(2.1) comment out or delete these two lines:

ACTION=="add", SUBSYSTEM=="net", KERNEL=="eth*|ath*|wlan*|ra*|sta*" \

NAME!="?*", DRIVERS=="?*", GOTO="persistent_net_generator_do"

I also used to shrink my disk (http://h0bbel.p0ggel.org/shrinking-vmdk-files)

   1: # fill the useless disk with zero be well compressed
   2: >cat /dev/zero > zero.fill;sync;sleep 1;sync;rm -f zero.fill

Maybe you need to change hostname, then modify at least /etc/hostname and /etc/hosts

Play with Ubuntu under Virtualbox

Monday, May 17th, 2010

This is for recording, if you are ubuntu fans, it is too easy for you, one of the reason for ubuntu is easily install the package when u online, it is similar for AppStore, find the package, install it. You don’t need to think about the dependence and download package seperately

I use Virtualbox (3.1.8) and Ubuntu (9.10)

image – Download lastest ubuntu server iso http://www.ubuntu.com/getubuntu/download
- create virtual machine with two network adapter (one for NAT, one for Host-only)
- Mount iso inside and install, it shall be very smooth.
- Configure network, if changes run sudo /etc/init.d/networking restart

   1:  
   2: user@ub1:~$ more /etc/network/interfaces
   3: # The loopback network interface
   4: auto lo
   5: iface lo inet loopback
   6:  
   7: # The primary network interface
   8: auto eth0
   9: iface eth0 inet dhcp
  10:  
  11: auto eth1
  12: iface eth1 inet dhcp
  13: user@ub1:~$ more /etc/resolv.conf
  14: domain codeslife.com
  15: search codeslife.com
  16: nameserver 192.128.18.10
  17: nameserver 192.11.115.200
  18: nameserver 192.88.112.200
  19: nameserver 192.117.20.200

- Configure apt for package

   1: user@ub1:~$ more /etc/apt/apt.conf
   2: # choose your proxy in IE
   3: Acquire::http::Proxy "http://iproxy.codeslife.com:8080/";
   4: user@ub1:~$ more /etc/apt/sources.list
   5: # choose suitable mirror
   6: deb http://ubuntu.cn99.com/ubuntu/ karmic main restricted universe
   7: deb http://ubuntu.cn99.com/ubuntu/ karmic-updates main restricted universe
   8: deb http://ubuntu.cn99.com/ubuntu karmic-security main restricted universe

- If ssh is not install, install ssh “sudo apt-get install ssh”

virtualbox vs Vmware Player

Monday, May 17th, 2010

image Playing with virtual software recently, and want to give a quick review for vmware player and virtualbox.

VMware player (free for use) is my starter virtual software, and from 3.0.0, it starts to support “creating virtual machine” inside, originally you can to create it from other place (like web), this feature is available in VMware workstation ($$), when you install it on windows, you need to reboot it to make some network device available. It is really annoying for me, since I want to let managers to install it for demos, require extra restart is not good user experience. The good thing is on my laptop (CPU T5500), it can support 64bit Linux easily

imageVirtualBox  has all the features that vmware player have (especially for our server development), and it is easy to manage the virtual machine on hardware (see right), to tune the CPU, memory, therefore we can see the performance difference on different environment. The CPU and memory occupy on host (windows) is also quite good.

No restart needed for installation though it will break your network for a while during installation.

Besides normal GUI administrative work, it provides CLI options (VBoxManage), The VBoxManage series of commands offer administrators the ability to build virtual machines, launch teleportation events, connect USB devices to virtual machines, manage snapshots and perform start/stop commands

export/import to ovf has small problem, but it is acceptable.

The 64bit guest OS support is not as good as VMware player on old CPU, it is crash on my notebook always, but it works ok on new laptop (T7500?)

So I am very happy with VirtualBox, it will be a base for our new design environment