###########################################
Xeno HOWTO

University of Cambridge Computer Laboratory

http://www.cl.cam.ac.uk/netos/xen
#############################


Get Xeno Source Codes
==========================

The public master BK repository for the 1.2 release lives at:
'bk://xen.bkbits.net/xeno-1.2.bk'

To fetch a local copy, first download the BitKeeper tools at:
http://www.bitmover.com/download with username 'bitkeeper' and
password 'get bitkeeper'.

Then install the tools and then run:
# bk clone bk://xen.bkbits.net/xeno-1.2.bk

Under your current directory, a new directory named 'xeno-1.2.bk' has
been created, which contains all the source codes for Xen and
XenoLinux. 

To get newest changes to the repository, run
# cd xeno-1.2.bk
# bk pull


Build Xen
=============================

Hint: To see how to build Xen and all the control tools, inspect the
tools/misc/xen-clone script in the BK repository. This script can be
used to clone the repository and perform a full build.

To build Xen manually:

# cd xeno-1.2.bk/xen
# make clean
# make

This will (should) produce a file called 'xen' in the current
directory.  This is the ELF 32-bit LSB executable file of Xen.  You
can also find a gzip version, named 'xen.gz'.

To install the built files on your Xenoserver under /usr, type 'make
install' at the root of the BK repository. You will need to be root to
do this!

Hint: There is also a 'make dist' rule which copies built files to an
install directory just outside the BK repo; if this suits your setup,
go for it.


Build XenoLinux
==============================

This is a little more involved since the repository only contains a
"sparse" tree -- this is essentially an 'overlay' on a standard linux
kernel source tree. It contains only those files currently 'in play'
which are either modified versions of files in the vanilla linux tree,
or brand new files specific to XenoLinux.

So, first you need a vanilla linux-2.4.24 tree, which is located at:
http://www.kernel.org/pub/linux/kernel/v2.4

Then:
  # mv linux-2.4.24.tar.gz /xeno-1.2.bk
  # cd /xeno-1.2.bk
  # tar -zxvf linux-2.4.24.tar.gz

You'll find a new directory 'linux-2.4.24' which contains all
the vanilla Linux 2.4.24 kernel source codes.

Hint: You should choose the vanilla linux kernel tree that has the
same version as the "sparse" tree.

Next, you need to 'overlay' this sparse tree on the full vanilla Linux
kernel tree:

  # cd /xeno-1.2.bk/xenolinux-2.4.24-sparse
  # ./mkbuildtree ../linux-2.4.24

Finally, rename the buildtree since it is now a xenolinux buildtree. 

  # cd /xeno-1.2.bk
  # mv linux-2.4.24 xenolinux-2.4.24

Now that the buildtree is there, you can build the xenolinux kernel.
The default configuration should work fine for most people (use 'make
oldconfig') but you can customise using one of the other config tools
if you want.

  # cd /xeno-1.2.bk/xenolinux-2.4.24
  # ARCH=xeno make oldconfig   { or menuconfig, or xconfig, or config }  
  # ARCH=xeno make dep bzImage

Assuming the build works, you'll end up with
/xeno-1.2.bk/xenolinux-2.4.24/arch/xeno/boot/xenolinux.gz. This is the
gzip version of XenoLinux kernel image.


Build the Domain Control Tools
==============================

Under '/xeno-1.2.bk/tools', there are three sub-directories:
'balloon', 'xc' and 'misc', each containing
a group of tools. You can enter any of the four sub-directories
and type 'make' to compile the corresponding group of tools.
Or you can type 'make' under '/xeno-1.2.bk/tools' to compile
all the tools.

In order to compile the control-interface library in 'xc' you must
have zlib and development headers installed. Also you will need at
least Python v2.2. 

'make install' in the tools directory will place executables and
libraries in /usr/bin and /usr/lib. You will need to be root to do this!

As noted earlier, 'make dist' installs files to a local 'install'
directory just outside the BK repository. These files will then need
to be installed manually onto the Xenoserver.

The Example Scripts
===================

The scripts in tools/examples/ are generally useful for
administering a Xen-based system.  You can install them by running
'make install' in that directory.

The python scripts (*.py) are the main tools for controlling
Xen domains.

'defaults' and 'democd' are example configuration files for starting
new domains.

'xendomains' is a Sys-V style init script for starting and stopping
Xen domains when the system boots / shuts down.

These will be discussed below in more detail.


Installation
==============================

First:
# cp /xen-1.2.bk/xen/xen.gz /boot/xen.gz
# cp /xen-1.2.bk/xenolinux-2.4.24/arch/xeno/boot/xenolinux.gz /boot/xenolinux.gz

Second, you must have 'GNU Grub' installed. Then you need to edit
the Grub configuration file '/boot/grub/menu.lst'.

A typical Grub menu option might look like:

title Xen 1.2 / XenoLinux 2.4.24
        kernel /boot/xen.gz dom0_mem=131072 ser_baud=115200 noht
        module /boot/xenolinux.gz root=/dev/sda4 ro console=tty0

The first line specifies which Xen image to use, and what command line
arguments to pass to Xen. In this case we set the maximum amount of
memory to allocate to domain0, and enable serial I/O at 115200 baud.
We could also disable smp support (nosmp) or disable hyper-threading
support (noht). If you have multiple network interface you can use
ifname=ethXX to select which one to use. If your network card is
unsupported, use ifname=dummy

The second line specifies which XenoLinux image to use, and the
standard linux command line arguments to pass to the kernel. In this
case, we're configuring the root partition and stating that it should
(initially) be mounted read-only (normal practice). 

The following is a list of command line arguments to pass to Xen:

 ignorebiostables Disable parsing of BIOS-supplied tables. This may
                  help with some chipsets that aren't fully supported
                  by Xen. If you specify this option then ACPI tables are
                  also ignored, and SMP support is disabled.

 noreboot         Don't reboot the machine automatically on errors.
                  This is useful to catch debug output if you aren't
                  catching console messages via the serial line.

 nosmp            Disable SMP support.
                  This option is implied by 'ignorebiostables'.

 noacpi           Disable ACPI tables, which confuse Xen on some chipsets.
                  This option is implied by 'ignorebiostables'.

 watchdog         Enable NMI watchdog which can report certain failures.

 noht             Disable Hyperthreading.

 ifname=ethXX     Select which Ethernet interface to use.

 ifname=dummy     Don't use any network interface.

 ser_baud=xxx     Enable serial I/O and set the baud rate.

 dom0_mem=xxx     Set the maximum amount of memory for domain0.


Boot into Domain 0
==============================

Reboot your computer; After selecting the kernel to boot, stand back
and watch Xen boot, closely followed by "domain 0" running the
XenoLinux kernel.  Depending on which root partition you have assigned
to XenoLinux kernel in Grub configuration file, you can use the
corresponding username / password to log in.

Once logged in, it should look just like any regular linux box. All
the usual tools and commands should work as per usual.


Start New Domains
==============================

You must be 'root' to start new domains.

Make sure you have successfully configured at least one
physical network interface. Then:

# xen_nat_enable
# xen_read_console &

When new  domains are created and  started, they will  send output via
UDP packets to  the local virtual network. Those  packets are received
by xen_read_console running in Domain  0 and output are printed out to
the standard output.

The xc_dom_create.py program is useful for starting Xen domains.
You can specify configuration files using the -f switch on the command
line.  The default configuration is in /etc/xc/defaults.  You can
create custom versions of this to suit your local configuration.

You can override the settings in a configuration file using command
line arguments to xc_dom_create.py.  However, you may find it simplest
to create a separate configuration file for each domain you start.

When you start domains, you should be able to see XenoLinux boot
message on standard output with each line prepended with [domain_id].


Manage Running Domains
==============================

You can see a list of existing domains with:
# xc_dom_control.py list

In order to stop a domain, you use:
# xc_dom_control.py stop <domain_id>

To shutdown a domain cleanly use:
# xc_dom_control.py shutdown <domain_id>

To destroy a domain immediately:
# xc_dom_control.py destroy <domain_id>

There are other more advanced options, including pinning domains to
specific CPUs and saving / resuming domains to / from disk files.  To
get more information, run the tool without any arguments:
# xc_dom_control.py

There is more information available in the Xen README files, the
VBD-HOWTO and the contributed FAQ / HOWTO documents on the web.


Other Control Tasks using Python
================================

A Python module 'Xc' is installed as part of the tools-install
process. This can be imported, and an 'xc object' instantiated, to
provide access to privileged command operations:

# import Xc
# xc = Xc.new()
# dir(xc)
# help(xc.domain_create)

In this way you can see that the class 'xc' contains useful
documentation for you to consult.

A further module of useful routines (XenoUtil) is also installed:

# import XenoUtil
# help(XenoUtil)

You can use these modules to write your own custom scripts or you can
customise the scripts supplied in the Xen distribution.


Automatically start / stop domains at boot / shutdown
=====================================================

A Sys-V style init script for RedHat systems is provided in
tools/examples/xendomains.  When you run 'make install' in that
directory, it should be automatically copied to /etc/init.d/.  You can
then enable it using the chkconfig command, e.g.:

# chkconfig --add xendomains

By default, this will start the boot-time domains in runlevels 3, 4
and 5.  To specify a domain is to start at boot-time, place its
configuration file (or a link to it) under /etc/xc/auto/.

The script will also stop ALL domains when the system is shut down,
even domains that it did not start originally.

You can also use the "service" command (part of the RedHat standard
distribution) to run this script manually, e.g:

# service xendomains start

Starts all the domains with config files under /etc/xc/auto/.

# service xendomains stop

Shuts down ALL running Xen domains.
