This directory contains the 3.0 release of `OpenSAF'.

`OpenSAF' is an open source project designed to implement a complete highly
available operating environment based on Service Availability Forum (SA Forum)
standards. The objective of the `OpenSAF' project is to accelerate broad adoption
of a SA Forum compliant operating environment. OpenSAF also offers
complementary services which are required in a complete high-availability
system solution.

`OpenSAF' has been originally contributed by Motorola ECC, by open sourcing
its HA middleware suite Netplane Core Services (NCS).

`OpenSAF' has a web site at http://www.opensaf.org/

See file `COPYING.LIB' for copying conditions.
See file `INSTALL' for compilation and installation instructions.
See file `NEWS' for a list of major changes in the current release.
See file `ChangeLog' for a list of detailed changes.
See file `AUTHORS' for the names of maintainers.
See file `THANKS' for a list of contributors.

Please check the `Linux' distribution-specific notes below for any caveats
related to your operating system.


`OpenSAF' Architecture Overview
===============================

`OpenSAF' software is distributed in nature. There is a Controller node, which is
the central entity in the cluster which will control all the other Payload nodes
in the HA cluster. In the HA cluster there are two System Controllers running in
Active/Standby mode.

The `OpenSAF' software is divided in to following classes of Modules:

    * Directors:
      The Service directors run on the Controller node. They interact with
      Node Directors running on the Payload nodes and provide service specific
      functionality.

    * Node Directors:
      Node directors distribute the service responsibilities with directors.
      Node directors interact with Agents to provide service functionality to
      the user applications. Node directors run both on System Controller node
      and Payload nodes.

    * Servers:
      Servers run on the System Controller and they talk to Agents to provide
      service functionality to the users. If the service is having servers then
      it doesn't have directors, node director pieces. If the service doesn't
      have node local functionality then "Server - Agent" architectural approach
      is used.

    * Agents: Agents are the service libraries which are linked with user
      applications and provide service to the user applications by interacting
      with other service parts.

User applications can run on either System Controller node or Payload node. But
it is advisable to run the user applications on the Payload nodes, so your
applications are not impacted when System Controller failover happens due to
any failures in `OpenSAF' Modules.

The OpenSAF infrastructure is made highly available by modeling the individual
services as AMF components. All the Node Directors use no redundancy model and
are restart capable with component capability 1_ACTIVE. All the Directors and
Servers use 2N redundancy model with "1_ACTIVE_OR_1_STANDBY".


Downloading
===========

`OpenSAF' release archives can be obtained from here:

    http://devel.opensaf.org/wiki/downloads/


Documentation
=============

`OpenSAF' implements various `SAF AIS' services. The Service Availability
Forum Specifications can be downloaded from here:

    http://www.saforum.org/specification/download/

The `OpenSAF' Programmer's Reference Manuals can be obtained from here:

    http://devel.opensaf.org/wiki/downloads/

    * OpenSAF Availability Service
    * OpenSAF Command Line Interface
    * OpenSAF Checkpoint Service
    * OpenSAF Availability Service
    * OpenSAF Event Distribution Service
    * OpenSAF Global Lock Service
    * OpenSAF HPI Integration Service
    * OpenSAF Information Model Management Service
    * OpenSAF Interface Service
    * OpenSAF LEAP
    * OpenSAF LOG Service
    * OpenSAF Management Access Service
    * OpenSAF Message Based Checkpointing Service
    * OpenSAF Message Distribution Service
    * OpenSAF Message Queue Service
    * OpenSAF Overview
    * OpenSAF Persistent Store Restore Service
    * OpenSAF System Description
    * OpenSAF SMIDUMP Tool
    * OpenSAF SNMP SubAgent
    * OpenSAF System Resource Monitoring Service
    * OpenSAF Platform Control Services


Development
===========

`OpenSAF' development is managed using `Trac', the main project portal can
be found here:

  http://devel.opensaf.org/

You can find most information concerning the development of `OpenSAF' at
this site.

`OpenSAF' is using `Mercurial' (hg) as its Source Control Management system
tool, the repositories index can be found here:

    http://devel.opensaf.org/hg/

`Mercurial' is a fast, lightweight Source Control Management system designed
for efficient handling of very large distributed projects and can be obtained
from here:

    http://www.selenic.com/mercurial/wiki/


Building from `Mercurial'
=========================

If you don't want to use one of the `OpenSAF' release archive, you can get a
clone and bootstrap it by doing the following steps:

    % hg clone http://devel.opensaf.org/hg/<repo>/
    % cd <repo>
    % ./bootstrap.sh

Where <repo> is one of the tree from the repositories index listed here:

    http://devel.opensaf.org/hg/

After bootsraping follow the standard build instructions.


Bug Reporting
=============

You can send `OpenSAF' bug reports to <devel@list.opensaf.org>. You can
subscribe to the development mailing list from here:

    http://list.opensaf.org/maillist/listinfo/devel/

If you need help using `OpenSAF', try <users@list.opensaf.org> instead. You can
subscribe to the user mailing list from here:

    http://list.opensaf.org/maillist/listinfo/users/

You can also use the online bug tracking system in the `Trac' `OpenSAF'
project to submit new problem reports or search for existing ones:

    * Create New Ticket: http://devel.opensaf.org/newticket/
    * Available Reports: http://devel.opensaf.org/report/

When reporting bug, make sure you provide various useful informations about
your Linux environment:

    * Distribution used (e.g. Fedora, Ubuntu, SUSE, PNE-LE etc.)
    * The revision of your Linux distribution
    * Package revisions of the prerequisites (e.g. net-snmp, tipc etc.)
    * If using the package prerequisites from the Linux distribution?
    * Build tool revisions (e.g. gcc, automake, autoconf, libtool)
    * If using cross-compiling + target information


Source Tree Structure
=====================

    +-opensaf/
      +-config/                     (`OpenSAF' configuration files)
      +-include/                    (`OpenSAF' and SAF header files)
      +-initd/                      (`OpenSAF' SysV init script)
      +-java/                       (Java AIS mapping implementation)
      | +-ais_api/
      | +-ais_api_impl/
      | +-ais_api_impl_native/
      | +-ais_api_test/
      +-lib/                        (`SAF AIS' implementation)
      | +-lib_ncs_core/
      | +-lib_SaAmf/
      | +-lib_SaCkpt/
      | +-lib_SaClm/
      | +-lib_SaEvt/
      | +-lib_SaImm/
      | +-lib_SaLck/
      | +-lib_SaLog/
      | +-lib_SaMsg/
      | +-lib_SaNtf/
      +-m4/                         (Extra M4 macros for the build process)
      +-mibs/                       (`OpenSAF' specific MIBS)
      +-samples/                    (Sample applications source)
      | +- ...
      +-scripts/                    (Various OpenSAF service scripts)
      +-services/                   (`OpenSAF' services implementation)
      | +-avsv/
      | +-cli/
      | +-common/
      | +-cpsv/
      | +-dtsv/
      | +-edsv/
      | +-glsv/
      | +-hisv/
      | +-ifsv/
      | +-immsv/
      | +-leap/
      | +-logsv/
      | +-mab/
      | +-mbcsv/
      | +-mds/
      | +-mqsv/
      | +-nid/
      | +-nms/
      | +-ntfsv/
      | +-rde/
      | +-srmsv/
      | +-vds/
      +-tests/                      (`OpenSAF' Test Cases)
      | +- ...
      +-tools/                      (Misc `OpenSAF' tools)
      | +-cluster_sim_uml/
      | +-pdrbd/


Prerequisites
=============

The following software is required to build `OpenSAF':

    * The Linux Kernel with TIPC development support
    * Net-SNMP 5.4+
    * An HPI implementation (e.g. OpenHPI)
    * Libxml2
    * automake
    * autoconf
    * libtool
    * pkg-config
    * flex and bison
    * gcc/g++
    * GNU make

    If enabling optional state replication:
        - DRBD 8.x.x

    If enabling optional Java support:
        - A Java 1.5+ capable JDK (e.g. OpenJDK 1.6)
        - Apache Ant

If a specific prerequisite package revision isn't working for you, please
let us know about it following the instructions from the Bug Reporting section.


Building `OpenSAF'
==================

If you are using a released archive (dist tarball) follow the simple common
steps:

    % ./configure && make

    OR

    % ./configure && make rpm

Note: `OpenSAF' can safely be built with parallel jobs using -jX, where X
      should be the number of CPU you want to dedicate + 1

If you are Building from `Mercurial', make sure you followed the required
steps before trying to `configure' (i.e. bootsrapping).

By default `OpenSAF' will assume to following `configure' options:

    --enable-hpi
    --with-openhpi
    --disable-java

If you wish to alter the default `configure' options see the list from:

    % ./configure --help

Use one of the following forms to enable an option:

    --with-<mumble>        OR --enable-<mumble>
    --with-<mumble>=yes		OR	--enable-<mumble>=yes
    --without-<mumble>=no	OR	--disable-<mumble>=no

Use one of the following forms to disable an option:

    --without-<mumble>		OR	--disable-<mumble>
    --without-<mumble>=yes	OR	--disable-<mumble>=yes
    --with-<mumble>=no		OR	--enable-<mumble>=no

The `--with-<mumble>' option forms are used for external package support and
the `--enable-<mumble>' option forms are used for internal component state.

By using `./configure --enable-hpi', you are enabling global HPI support and
one HPI implementation is enabled by default (for now OpenHPI). By default
this option is enabled. If using `./configure --disable-hpi', every other HPI
configure options are ignored. It's an explicit way to disabling global HPI
support.

Note: In this current release, `OpenSAF' still relies on <SaHpi.h>, even if
the user thinks he's not using a specific HPI implementation. This is a build
dependency for now and the user still needs to tell the build system to  be
using the proper CPPFLAGS/CFLAGS/CXXFLAGS where to find a dummy <SaHpi.h>

By using `./configure --with-openhpi', you are enabling support for `OpenHPI'
as the HPI implementation to be used. It sets the proper LIBS and CFLAGS
using the `pkg-config' utility for `opensaf.pc'. By default this option is
enabled.

By using `./configure --enable-java', you are enabling the Java AIS mapping
support in the build system. Specials M4 macros will be used to autodetect
what `JDK' and `Ant' program you are using. By default this options is
disabled.

By using `./configure --with-jdk=DIR', you can force the build system to use
a specific JDK installed in a non-standard location on your system.

By using `./configure --with-ant=DIR', you can force the build system to use
a specific Ant version installed in a non-standard location on your system.

By using `./configure --with-rpm-release', you can change the default release
token in the generated rpm filename. Predefined values exist like the build
timestamp, the Mercurial revision used to when building the code, and also you
can add your own free text token.

By using `./configure --with-hpi-interface', you can force the HPI interface
version used. By default it's autodetected by scanning the SaHpi.h header, but
when cross-compiling is used it's not possible to run the test code for that
autodetection.

By using `./configure --disable-rpm-target', you are masking out the `make rpm'
target, some build systems provide their own `make rpm` rule, it would then be
clashing with the OpenSAF rule.

By using `./configure --with-net-snmp-config-bin', you inform the build system
where to find the `net-snmp-config' script. It's not possible to autodetect the
proper one when cross-compiling, force its location in this case.

By using `./configure --with-xml2-config-bin', you inform the build system
where to find the `xml2-config' script. It's not possible to autodetect the
proper one when cross-compiling, force its location in this case.


Building `OpenSAF' RPMS
=======================

As shown in the Building `OpenSAF' section, generating RPMS can be done using:

    % ./configure && make rpm

The special `make rpm' target will generate a dist archive, create the
`OpenSAF' SRPM using the opensaf.spec file, and finally launch the rpmbuild
process. You can find the generated RPMS under `./rpms/RPMS/<build_arch>/':

Depending on what configure options you used, you should have the generated
the following RPMS:

    opensaf-<version>.<build_arch>.rpm
    opensaf-common-<version>.<build_arch>.rpm
    opensaf-controller-<version>.<build_arch>.rpm
    opensaf-debuginfo-<version>.<build_arch>.rpm
    opensaf-devel-<version>.<build_arch>.rpm
    opensaf-java-<version>.<build_arch>.rpm
    opensaf-libs-<version>.<build_arch>.rpm
    opensaf-payload-<version>.<build_arch>.rpm
    opensaf-samples-<version>.<build_arch>.rpm
    opensaf-tools-<version>.<build_arch>.rpm

opensaf: (Virtual)
Virtual package that prints out the basic installation usage.

    % rpm -ivh opensaf-3.0.0.B2-1.fc9.x86_64.rpm

      ---------------------------------------------------------------------
      * MESSAGE:                                                          *
      * Depending on the node type you are trying to configure:           *
      * Install the 'opensaf-controller' or the 'opensaf-payload' package *
      ---------------------------------------------------------------------

opensaf-common: (Required)
Contains the common configuration files, docs, and scripts.

opensaf-controller: (Mutually exclusive)
Contains the controller specific binaries and libs.

opensaf-debuginfo: (Optional)
Contains the debug information unstripped.

opensaf-devel: (Optional)
Contains the development headers and static & shared libs.

opensaf-java: (Optional)
Contains the Java AIS mapping jars and native libs.

opensaf-libs: (Required)
Contains the common shared libs.

opensaf-payload: (Mutually exclusive)
Contains the payload specific binaries and libs.

opensaf-samples: (Optional)
Contains the sample applications source code.

opensaf-tools: (Optional)
Contains samples and tools that relies on OpenSAF internals.


Installing `OpenSAF'
====================

`OpenSAF' can be installed in several ways. When doing development, a
user might prefer using the classic `make install' way to using RPMS.
It can be achieved like this:

    % make install

    OR

    % make install DESTDIR=<staged_install_directory>

After installing `OpenSAF' you will need to run `ldconfig' because `OpenSAF'
places internal libs in the `$pkglibdir' (e.g. /usr/lib/opensaf/). This ldconfig
operation is handled automatically by using rpm installation.

By default `make install' will install pre-configured for a controller node.
The only files that need to be changed for a payload is the `nodeinit.conf' and
the `slot_id' (a slot_id of 3 or higher will give you a payload node):

    % cd $pkgsysconfdir (e.g. /etc/opensaf/)
    % ln -sf nodeinit.conf.payload nodeinit.conf
    % echo 3 > $pkgsysconfdir/slot_id

In production systems, RPMS should be preferred to classic `make install`.

To setup a controller, install the following RPMS:

    % rpm -ivh opensaf-common-<version>.<build_arch>.rpm
    % rpm -ivh opensaf-libs-<version>.<build_arch>.rpm
    % rpm -ivh opensaf-controller-<version>.<build_arch>.rpm

To setup a payload, install the following RPMS:

    % rpm -ivh opensaf-common-<version>.<build_arch>.rpm
    % rpm -ivh opensaf-libs-<version>.<build_arch>.rpm
    % rpm -ivh opensaf-payload-<version>.<build_arch>.rpm

To setup a development environment, install following RPMS:

    % rpm -ivh opensaf-libs-<version>.<build_arch>.rpm
    % rpm -ivh opensaf-devel-<version>.<build_arch>.rpm
    % rpm -ivh opensaf-samples-<version>.<build_arch>.rpm
    % rpm -ivh opensaf-debuginfo-<version>.<build_arch>.rpm

If Java AIS mapping was enabled, install the following RPM:

    % rpm -ivh opensaf-java-<version>.<build_arch>.rpm

If you are installing `OpenSAF' from a supported `yum' server, it will resolve
the proper dependencies for you:

    % yum install opensaf-controller

    OR

    % yum install opensaf-payload

Here is where individual `OpenSAF' components get installed:

    * Documentation: `$docdir' (e.g /usr/share/doc/opensaf-<version>/)
    * Samples, Templates: `$pkgdatadir' (e.g. /usr/share/opensaf/)
    * OpenSAF headers: `$pkgincludedir' (e.g. /usr/include/opensaf/)
    * SAF headers: `$includedir' (e.g. /usr/include/)
    * Configuration: `$pkgsysconfdir' (e.g. /etc/opensaf/)
    * Runtime Misc.: `$pkglocalstate' (e.g. /var/lib/opensaf/)
    * OpenSAF internals: `$pkglibdir' (e.g. /usr/lib/opensaf/)
    * User programs: `$bindir' (e.g. /usr/bin/)
    * SAF libraries: `$libdir' (e.g. /usr/lib/)
    * ldconfig data: `$sysconfdir/ld.so.conf.d' (e.g. /etc/ld.so.conf.d/)
    * SysV scripts: `$sysconfdir/init.d/' (e.g. /etc/init.d/)
    * Logs: `$localstatedir/log/opensaf' (e.g. /var/log/opensaf/)
    * PID: `$localstatedir/run/opensaf/' (e.g. /var/run/opensaf/)


Notes on the Debuginfo package
==============================

You will notice that the RPM build process generated an `opensaf-debuginfo'
package. This package contains the debug information that GDB needs to perform
debugging on a specific binary.

There is a magic section in the regular ELF binary that says "look THERE for my
debug info", where "THERE" is the location the debuginfo RPM installs its files.
It's basically a "strip to file" operation, where a file with debuginfo is split
in 2, one is the normal stripped file (but with a "look THERE" record) and one
is only the debuginfo.

A debuginfo package contains stripped symbols from ELF binaries (*.debug in
/usr/lib/debug) as well as the source code related to them (in /usr/src/debug).

	% ll /usr/lib/debug/usr/bin
	total 9844
	-rwxr-xr-x 1 root root  185160 2008-12-11 15:21 ncs_cli_maa.debug
	-rwxr-xr-x 1 root root  310272 2008-12-11 15:21 ncs_cpd.debug
	-rwxr-xr-x 1 root root  235480 2008-12-11 15:21 ncs_cpnd.debug
	-rwxr-xr-x 1 root root  429752 2008-12-11 15:21 ncs_dts.debug
	-rwxr-xr-x 1 root root  290760 2008-12-11 15:21 ncs_eds.debug
	-rwxr-xr-x 1 root root  225216 2008-12-11 15:21 ncs_gld.debug
	-rwxr-xr-x 1 root root  246808 2008-12-11 15:21 ncs_glnd.debug
	-rwxr-xr-x 1 root root  207400 2008-12-11 15:21 ncs_hisv.debug
	-rwxr-xr-x 1 root root  545744 2008-12-11 15:21 ncs_ifd.debug
	-rwxr-xr-x 1 root root  297128 2008-12-11 15:21 ncs_ifnd.debug
	-rwxr-xr-x 1 root root   42968 2008-12-11 15:21 ncs_ifsv_ip_installer.debug
	-rwxr-xr-x 1 root root  258712 2008-12-11 15:21 ncs_mas.debug
	-rwxr-xr-x 1 root root  322536 2008-12-11 15:21 ncs_mqd.debug
	-rwxr-xr-x 1 root root  332192 2008-12-11 15:21 ncs_mqnd.debug
	-rwxr-xr-x 1 root root   45112 2008-12-11 15:21 ncs_nid.debug
	-rwxr-xr-x 1 root root  849464 2008-12-11 15:21 ncs_pcap.debug
	-rwxr-xr-x 1 root root   89832 2008-12-11 15:21 ncs_pdrbd.debug
	-rwxr-xr-x 1 root root  501616 2008-12-11 15:21 ncs_psr.debug
	-rwxr-xr-x 1 root root   81832 2008-12-11 15:21 ncs_rde.debug
	-rwxr-xr-x 1 root root 3840560 2008-12-11 15:21 ncs_scap.debug
	-rwxr-xr-x 1 root root   34976 2008-12-11 15:21 ncs_snmp_subagt.debug
	-rwxr-xr-x 1 root root  203056 2008-12-11 15:21 ncs_srmnd.debug
	-rwxr-xr-x 1 root root  104160 2008-12-11 15:21 ncs_vds.debug
	-rwxr-xr-x 1 root root  146792 2008-12-11 15:21 opensaf_saflogd.debug
	-rwxr-xr-x 1 root root   18040 2008-12-11 15:21 saflogger.debug

	% ll /usr/src/debug/opensaf-<version>/
	total 32
	drwxr-xr-x  2 root root 4096 2008-12-11 16:01 include
	drwxr-xr-x  3 root root 4096 2008-12-11 16:01 java
	drwxr-xr-x 23 root root 4096 2008-12-11 16:01 services
	drwxr-xr-x  3 root root 4096 2008-12-11 16:01 tools

The directory `/usr/lib/debug' contains a "mirror" of your main FHS, with all
the debuginfo packages installed. The directory `/usr/src/debug/<package>/'
contains a "mirror" of the original source tree, but with only the source files
used at compilation.

Recent GDB versions know how to automatically load the debuginfo from that
location and perform source code tracing (breakpoints at func() etc.).


Linking with `OpenSAF' AIS Services
===================================

Due to the historical origins of the internal of `OpenSAF', there is not a
direct one to one mapping of the `OpenSAF' shared libraries and the SA Forum AIS
shared libraries.

For instance, `libSaAmf.so' from `OpenSAF', while providing the API to AMF,
still relies on some "internal" `OpenSAF' libraries to provide full linking
support to the application developer who wish to use AMF.

To make the developer's life easier, `OpenSAF' provides `pkgconfig' files for
each AIS services. Those files are installed with the other system `*.pc' files
in `$(libdir)/pkgconfig':

   * opensaf-amf.pc
   * opensaf-ckpt.pc
   * opensaf-clm.pc
   * opensaf-evt.pc
   * opensaf-imm.pc
   * opensaf-lck.pc
   * opensaf-log.pc
   * opensaf-msg.pc
   * opensaf-ntf.pc

There is two easy ways to use `pkgconfig'. The first one is to call it directly
from a `Makefile' and assign its content to make variables e.g.:

   AMF_LIBS=`pkg-config opensaf-amf --libs`
   AMF_CFLAGS=`pkg-config opensaf-amf --cflags`

Or if your application is using the `GNU Build System' with `autotools', you can
use a special `autoconf' macro provided by `pkgconfig' e.g.:

   -- in configure.ac --
   PKG_CHECK_MODULES(AMF, opensaf-amf)

It will then provides two special variables available to `automake':

   -- in Makefile.am --
   mumble_CFLAGS = @AMF_CFLAGS@
   mumble_LDADD = @AMF_LIBS@


How to Configure `OpenSAF'
==========================

See file `00-README.conf' for the configuration file syntax/format.

More TODO


How to Run `OpenSAF'
====================

The `OpenSAF' SysV init script is installed in `$sysconfdir/rc.d/init.d'
(e.g. /etc/rc.d/init.d/) and can be executed like that:

    % /etc/init.d/opensafd [start|stop|status]

More TODO


Building `OpenSAF' Samples
==========================

The sources of the `OpenSAF' sample applications will get installed in
`$pkgdatadir/samples' (e.g. /usr/share/opensaf/samples/).

Each sample application has its own `Makefile', independent from the `OpenSAF'
build system. To build one, simply make sure you have write access or make a
copy of the source somewhere else:

    % make

The `Makefile' looks for installed `SAF AIS' headers and `OpenSAF' libs in
standard system wide locations (e.g. /usr/include/ & /usr/lib/):

    INCLUDES = -I.
    LDFLAGS = -lSaAmf -lsaf_common -lavsv_common

If you have `OpenSAF' development packages installed somewhere else, override
the default values e.g.:

    % make INCLUDES="-I. -I/tmp/usr/local/include" \
        LDFLAGS="-L/tmp/usr/local/lib -lSaAmf -lsaf_common -lavsv_common"

To run the an application you will need a configured `OpenSAF' node running.

See file `00-README.samples' for more information on each of the application.


Using the Simulation Environment
================================

See file `00-README.uml' for more information about User Mode Linux and `OpenSAF'


Linux Distro-Specific Notes
===========================

`OpenSAF' is known to run on the following Linux installations:

    * Red Hat Enterprise Linux 4 (RHEL4)
    * Red Hat Enterprise Linux 5.2 (RHEL5)
    * Wind River Platform for Network Equipment, Linux Edition (PNE-LE)
    * MontaVista Carrier Grade Edition 4.0
    * Fedora Core 9
    * SUSE Linux Enterprise 10 SP2
    * ...

More TODO

<RHEL firewall rules, SELinux, SUSE crashes with -02, missing decent tipc.h etc.>

-------------------------------------------------------------------------------

Copyright (C) 2008 The OpenSAF Foundation
