tex/manual/SCRAM.tex

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% SCRAM manual                                                %
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% modified : Fri Jun 21 14:50:47 2002 / SFA                   %
%          :                                                  %
%          :                                                  %
%          :                                                  %
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% Revision : $Id: SCRAM.tex,v 1.5 2002/07/11 13:01:34 sashby Exp $
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\documentclass[11pt]{report}
\usepackage{a4p}
\usepackage{html}
\usepackage{makeidx}

%% Begin the document:
\begin{document}
\pagestyle{plain}
\oddsidemargin=-0.25cm
\topmargin=-1.5cm
\setlength{\textwidth}{16.5cm}
\setlength{\textheight}{24.0cm}
\headheight 0.0cm
\footskip 35pt
\parskip 2ex plus 2pt minus 1pt

%% Define some variables here:
\newcommand{\authorname}{S.~Ashby, \textsc{CERN EP D}ivision.}
\newcommand{\revision}{1.0}
%% Need to automate the update of this variable:
\newcommand{\thisrelease}{CURRENTRELEASETAG} %% Something like this
                                             %% should work: V0\_19\_4

%% More variables:
\newcommand{\scramdevelopers}{scram-developers@cern.ch}
\newcommand{\scram}{\textsc{scram}}
\newenvironment{indentlist}[2]{\begin{description}}{\end{description}}
\newcommand{\indentitem}[1]{\item[#1]}
\newcommand{\url}[2]{\htmladdnormallink{#1}{#2}}
\newcommand{\email}[1]{\htmladdnormallink{#1}{mailto:#1}}

%% Heading for title page:
\begin{titlepage}
\begin{center}
 {\Large\bf SCRAM User Manual } \\ \vspace{1cm}
\end{center}

%% Author and contact info:
\vspace{1cm}    
\begin{center}
  \begin{indentlist}{2.5cm}{3.0cm}
    \indentitem{Documentation author:}\authorname
    \indentitem{Contributors:}
  \end{indentlist}
\end{center}
\vspace{3cm}

\indent \scram~ (\textsc{s}oftware \textsc{c}onfiguration 
\textsc{r}elease \textsc{a}nd  \textsc{m}anagement) is a tool
originally conceived for the software development environment of a
high-energy physics experiment at \textsc{cern}. It provides a set of
commands allowing users to develop and test code in different physical
locations while ensuring that a common configuration is available, but
also allowing developers the flexibility to easily choose
different tools as they wish (compilers or external packages, etc\.).


Further information can be obtained from the \url{SCRAM home page.}
{http://cmsdoc.cern.ch/Releases/SCRAM/current/cgi/scrampage.cgi}

\textbf{please report bugs to \email{\scramdevelopers}}

\end{titlepage}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

%% Start the sections


% Introduction and History


% An overview for the impatient user

% Setting up a project for the first time


% Creating a developer area


% using debug libraries

% Changing tools or adding/configuring new tools


\section{Introduction and History}


%Stuff here from Dr Dobbs....
%Description

%      What is SCRAM? 
%      Current SCRAM Functionality 


\subsection{Background}

SCRAM has been developed to enable large, geographically dispersed and
autonomous groups to work together on software development projects.
The groups, primarily based in universities and academic institutions,
independently manage their own resources. As such it can be extremely
difficult or even impossible to impose software process, adequate
documentation levels and heavy resource requirements - such as
dedicating entire machines to a single software development project.

The philosophy throughout the SCRAM project is, therefore, to
encourage good practice by making the intuitive, easy and convenient
way to do things the right way to do things. Take for example the fact
that SCRAM is controlled and configured through XML-like documents.
Thus documentation has to be kept up to date for SCRAM to even
function.

\subsection{What is SCRAM?}


\subsubsection{A Configuration Manager}

The main task of the SCRAM tool is to ensure that all developers are
working with the same consistent set of products, libraries,
environments and source codes.
\begin{itemize}
\item External Products Configuration A requirement of any SCRAM
  managed project is an explicit statement, in the form of an XML
  document, of all the underlying products and versions of external
  libraries and other software products used. Each product must have a
  description document to tell SCRAM how it is to be used, dependency
  information, environmental variables, default system locations etc.
  Such description documents can be maintained independently of SCRAM
  and imported into the project by SCRAMs built-in url down-loading
  mechanism.
\item Common Configurations It is often the case that many projects
  need to share the same configuration in order that they can be
  inter-operable (e.g. Two applications using the same database).
  SCRAM thus provides a mechanism for importing independently
  maintained configuration documents automatically.
\item Source Code Control SCRAM itself is not a code repository but
  any project must have access to one or more such repositories from
  which it can checkout the appropriate code into the appropriate
  place in the project structure. Currently it only supports CVS but
  this could be easily extended to other repositories if required.
\item Environment Control The build and runtime environments are built
  up from the descriptions of those required by external products and
  project specific environment description documents.
\end{itemize}


\subsubsection{A distribution system}

SCRAM projects can be 'bootstrapped' from a single document that
describes its structure and download information of other project
documents and components.

From this document SCRAM can construct a copy of a project release.
Connected to a web browser such as netscape makes "single click"
installation possible. At present SCRAM is unable to automatically
install external components although the user can be directed to the
correct documentation to do this themselves. Binary distribution is
not supported as building a distribution is seen as a check that all
the components of the system have been installed properly. This
feature could be added if user distributions were required.


\subsubsection{A System resource/application interface}

As no two people are the same, it is often the case that no two
machines are the same. Products can be installed in different places
depending on the whims of system administrators, policy makers and
system constraints. SCRAM matches up the request for a product from a
projects configuration to the system it is installing the project on.
SCRAM does this by a number of automated means and if it fails to find
a product will prompt the user for its location. SCRAM then maintains
a database of such system information for future reference.

\subsubsection{A Build System}

Abstraction of logical build elements from implementation details
Associate directory and file structure with build operations (e.g.
everything in a directory libsrc could be automatically compiled into
a library , e.g.2 every binary in a directory test could be
automatically linked with a test utilities library) Definition of
Classes of build objects. e.g A library class can have types such as
debug, archive, shared, shared debug, profiled etc. Default types can
be assigned to a class/directory structure but are easy to override on
the command line.  Strong environment control. Makes it easy to tweak
general rules for special cases - something not easy to do in make.
Abstract dependency specification. You can link in a product by
specifying its name, SCRAM does the rest taking care of system
specifics, dependencies etc.  Module Interfaces can be defined for
large software modules to define dependencies etc. Other modules can
then simply load the interface to use the module.  Compatible with
code repositories such as CVS which checkout the file with the same
date that it was checked in. This is a major flaw in make which simply
checks to see if a datea > dateb rather than datea != dateb.

\subsubsection{A development environment}

Once a copy of a release is installed and built on a system it can be
made available for developers by adding it to SCRAMs list of projects.
Upon selecting an item from this list, a new development area is
created in which the developer can work independently of everyone
else. The development area will have the same configuration,
environment etc. as the base release. It will also automatically use
libraries/headers etc. from the base release if not rebuilt in the
local development area. The user is free to use whatever editors and
other tools he/she prefers.

\subsubsection{Project Isolation}

SCRAM ensures that an installed release is independent of any other.
This allows developers to easily switch between projects/versions they
might be working on even though they may have very different and
conflicting environment requirements.

%%
%% Insert this pic here: <SCRAM-base>/doc/images/scram.jpg
%%
%%


Summary
\begin{itemize}
\item Project installation with a click on a web page
\item Control of Build Environment, including dependency tracking
\item Fully configurable build operations, including default
  operations.
\item Abstraction of logical build elements from the implementation
  details
\item Reuse of configuration management elements between projects.
\item Configuration specifications with XML documents.
\end{itemize}


\section{Installing SCRAM}

This section is for administrators who need to set things up. General
users can skip straight to the "Working within a SCRAM Environment"
section.


\subsection{Download and general installation instructions} 
            
o Requirements

\subsubsection{Downloading and installing from CVS}

1.Click on the boxes next to the numbers as you proceed to ensure you
don't miss any steps.  2. Create a directory SCRAM in which you wish
to keep your scram installation e.g.  mkdir SCRAM

3.Go into the directory you have just made e.g.  cd SCRAM

4. Set the Environment variable CVSROOT variable to

%:pserver:anonymous@cmscvs.cern.ch:/cvs_server/repositories/SCRAM

e.g.  csh, tcsh: setenv CVSROOT
%:pserver:anonymous@cmscvs.cern.ch:/cvs_server/repositories/SCRAM
Bourne Shell, zsh, ksh: set
%CVSROOT=:pserver:anonymous@cmscvs.cern.ch:/cvs_server/repositories/SCRAM;
export CVSROOT 5. Type cvs login At the prompt enter the password
98passwd

6. Check out the version of scram you require with the following
command.

cvs co -d version -r version SCRAM

Where version is one of versions listed as available on the main scram
page. Make sure you replace version twice in the above command!  Note:
Once a version of scram has been installed, installation of any other
version becomes trivial with the scram version Version command

7. Goto the Installation directory of the version you have just
checked out

e.g cd version/Installation

8. Run the installation script (install_scram).  e.g.  ./install_scram

This will create a file wrapper script called "scram" in your top
level directory.  The Perl executable will be found from your path,
automatically. If you want to specify the location, you can use the
-perl option (from V0\_15\_0): e.g ./install_scram -perl /usr/bin/perl

Experienced Users Note: The install script will make the current
version the default.


\subsubsection{Further information on SCRAM installation at remote sites}

At external sites, where the SITENAME is not CERN, the installation
script has been modified to allow the site managers to set default
values. The script asks for a value for SITENAME:

** Please enter the name of your SITE (e.g. CERN[default], FNAL) **
sitename?:

and for a default location for the CMS tools file:

** Please enter the default location of the cmstools file **
cmstools-CERN.conf location?: ~/siteconf

This location should be a directory name (e.g. ~/siteconf), under
which a valid file can be found (note that the filename of this file
must be cmstools-SITENAME.conf).

During a setup or project command, this directory will be used in
place of [project_name]/config/site/.

If the -f argument is given to either the project or setup commands,
the file specified will be used and any other file location (including
the default location described above) will be overridden.  For
example, if the filename was given as ~/myfile.conf, then a file with
this name in the home directory will be used if it exists.

Note that any filename given as a cmstools file name must end in
".conf".

The main part of the filename can be anything at all.

9. change back to your top level directory e.g.  cd ../..

You will see the new file in this directory. Ensure that your system
can see it too by either: Creating a link from an area in your path to
the scram file in this directory Extending your PATH environment
variable to include your top level directory Creating an alias to
point to the scram file in this directory

10. If you wish to use SCRAM directly through web page links you will
now need to configure your browser to use scram.
   
\subsubsection{Configuring Netscape to use SCRAM} 


Netscape navigator Checklist

From the EDIT menu Select "Preferences" to open the preferences window
Click on "Navigator" and select "Applications" Click on New and fill
in the form Fill in the MIME-type box with application/scram_bootstrap
Click on the Button Next to the Application box to select it.  Fill in
the Application Box with :

xterm -e scram -re project -d myinstall_area file:%s 

Remember to replace myinstall_area in the above line with the location
where you will want your downloaded projects to be.  i.e. Where the
src/binaries etc should go.

Make sure that the directory you specified actually exists!

Click on OK Click on OK in the preferences box

So now whenever you click on a project bootstrap file netscape will
start SCRAM for you automatically to install the project.

Note: If you have just installed SCRAM, it may be necessary to restart
netscape in an environment where it can find the scram executable


\subsubsection{Controlling SCRAM versions}

 
Checking the Current Version

The version of scram you are using can be found with the command

scram version

Downloading Alternative SCRAM versions

Once a version of SCRAM has been installed, downloading of other SCRAM
versions becomes trivial with the scram version command.  If you
supply a version number as an argument, SCRAM will check to see if the
specified version is already installed. If it isn't then it will
attempt to download the requested version automatically.

e.g.

scram version V0\_19\_2

Making a version the default

Running the install_scram script of a specific version (see
Installation Guide) will make that version the default.

Overriding the default Version

The default version can be overridden at any time by setting the
SCRAM_HOME environment variable to point to the top of the version
required. It must be the full path to the version.

Automated Version Selection

Scram can be made to automatically select the correct version to use
for a given project area, overridding any other version selection
mechanism. To turn this feature on, simply create the file
scram_version in your project development/release configuration
directory. The file should contain the required SCRAM version tag. Any
scram command issued in that area is then directed to the correct
version if it is installed. If the required version is not installed
then a warning is issued and execution of the command continues within
the current version.

\subsubsection{SCRAM database handling} 


The scram database maintains lookup tables for the various projects
that may be installed on your system as accessed through the scram
project project_name project_version command.

The scram database will usually live in the scramdb directory of your
scram installation area and is accessible to all scram versions. This
default can be overriden with the environment variable SCRAM_LOOKUPDB.

Linking databases

Scram databases can be linked together. You may want to do this, for
example, if you are maintaining a local copy of a project to speed up
development but would still like access to a remote copy of the
project (e.g over a shared file system such as AFS) which may be
slower but will have a guaranteed configuration, minor versions etc.

Maintaing Links

Database linking is maintained through

scram db [link dbfile|unlink dbfile|showlinks]

Please see the online help for more details.

Installing a Project in the Database

A project can be installed into the database with the

scram install [project_name project_version]

command. If no name/version tags are given the defaults are used from
the project configuration files.

Removing A project from the database

A project can be removed from the local database with the command

scram remove [project_name project_version]


\subsubsection{Installation of projects that use SCRAM}

It is envisaged that releases of a project need to be distributed to
remote sites. The release is contained in a special scram area which
is created by scram according to the projects requirements. Once a
carbon copy of the area has been built on the remote site, the
administrator can "scram install" it into the remote sites scram
database.  Once installed many other scram development areas can be
based upon it, for use by other users,developers etc.

SCRAM aims to make installion of projects as painless a possible. It
is designed to be used with a web browser so that you simply click on
a link to start the installation process.

You will be asked questions about your system as required by the
project during the installation. Some basic error and consistency
checking is provided. These values can be changed later (see below) so
dont worry too much if you get things wrong at this stage.

Once through the installation process you can make the project
available for general use by using the scram install command. This
command will update your local SCRAM database with the project details
so that other SCRAM user can simply refer to it by name and version.

Automated setup procedure for new project areas

The setup procedure is first invoked for a new project area to set up
the tools selected via the RequirementsDoc. This is now automated by
using site information read from a configuration file. The
configuration file is called cmstools-SITENAME.conf, where SITENAME is
the name of your site (the default is CERN). The sitename can be
changed by modifying the file sitename. These files reside in the
configuration area for each SCRAM-based project under a subdirectory
site. The configuration file contains default paths to tools,
especially ANAPHE tools, which scram will use first to find locations
of libraries and include files. At remote sites that have read access
to the CVS repository only, and want to install a project that was
previously released at CERN, the sitename can be chosen during scram
installation. An optional flag to the project command can allow the
administrator to specify the cmstools file to read. A default location
for this file can also be chosen during scram installation.

Here is an extract from the CERN configuration file:

\begin{verbatim}
###############################################################
# cmstools-CERN.conf                                          #
###############################################################
# modified : Tue Nov 20 11:39:49 2001 / SFA                   #
#          :                                                  #
#          :                                                  #
# function : Configuration file for setting up ANAPHE and CMS #
#          : tools at CERN.                                   #
#          :                                                  #
###############################################################
# Set some ANAPHE variables.
# Specify the OS. All subsequent defns will be for this OS:
ANAPHEOS:Linux
# ANAPHE version:
ANAPHEVER:3.6.2
# Specify the top of the ANAPHE tool dir, excluding the version:
ANAPHETOP:/afs/cern.ch/sw/lhcxx/specific/redhat61/gcc-2.95.2
#
CMSTOOL:g77:
   +FC:/usr/local/gcc-alt-2.95.2/bin/g77
CMSTOOL:gcc:
   +CXX:/usr/local/gcc-alt-2.95.2/bin/c++
   +CC:/usr/local/gcc-alt-2.95.2/bin/gcc
   +LD_LIBRARY_PATH:/usr/local/gcc-alt-2.95.2/lib
CMSTOOL:htl:
   +SCHEMA_HOME:/afs/cern.ch/sw/lhcxx/share/HTL/1.3.1.1/schema
CMSTOOL:zlib:
   +ZLIB_BASE:/afs/cern.ch/cms/Releases/sw/zlib/Linux__2.0/zlib-1.1.3
# Specify the OS. All subsequent defns will be for this OS:
ANAPHEOS:SunOS
# ANAPHE version:
ANAPHEVER:3.6.2
# Specify the top of the ANAPHE tool dir, excluding the version:
ANAPHETOP:/afs/cern.ch/sw/lhcxx/specific/sun4x_57/CC-5.2
#
CMSTOOL:f77:
   +FC:/afs/cern.ch/project/sun/solaris/opt/SUNWspro61/bin/f77
CMSTOOL:cc:
   +CXX:/afs/cern.ch/project/sun/solaris/opt/SUNWspro61/bin/CC
   +CC:/afs/cern.ch/project/sun/solaris/opt/SUNWspro61/bin/cc
   +CXX_PRIOR_LIBS:/afs/cern.ch/project/sun/solaris/opt/SUNWspro61/WS6U1/lib
CMSTOOL:htl:
   +SCHEMA_HOME:/afs/cern.ch/sw/lhcxx/share/HTL/1.3.1.1/schema
CMSTOOL:zlib:
   +ZLIB_BASE:/afs/cern.ch/cms/Releases/sw/zlib/SunOS__5.6/zlib-1.1.3
\end{verbatim}
   
   For ANAPHE tools under linux, the default path is ANAPHETOP and
   normally the libraries are found under a subdirectory matching the
   ANAPHE version (set using the variable ANAPHEVER in the above
   file). If a tool is not likely to be found under this default path,
   an entry is made in the file. This entry begins with CMSTOOL and
   the tool name (as known to scram), and any variables that scram
   must set for this tool are subsequently listed, each entry
   beginning with a plus sign.
   
   When setting up the tools, scram reads this file and creates a
   lookup-table. If the default search fails, scram looks for an entry
   in the lookup table for the tool. If an entry exists, subsequent
   values for the scram variables will be used. If no entry is found,
   scram will drop to an interactive environment and request user
   input.

\subsubsection{Changing Installation Defaults}

You can use the command

scram setup [-i] [-f cmstools.conf] tool_name [[tool_version]
[tool_file]]

to rerun the setup of a specific tool.

The -f flag causes setup to read the filename given. This filename
must end in ".conf".

You can override the automatic mechanisms that SCRAM uses to install a
tool with the -i option, or if this is not sufficient, you can modify
the values directly. Simply edit the appropriate .dat file in the
.SCRAM/SCRAM_ARCH directory from the top of the project installation.


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%


\section{Configuration Management with SCRAM}

\subsection{What is CM?}

\subsection{Configuring a new project} 

o Tools Available in a SCRAM environment 
o Installing/removing tools 
o Tool Description Documents 

\subsection{Working Within a SCRAM Environment}

o Getting Started 
o The Runtime Environment 
o Building Binaries 

o Controlling the Build - The BuildFile 

o            <Export> tags. Configuring the <Use> tag for your Software Package. 
o            Defining the Configuration Interface for a Software Unit 
o            BuildFile Tags 
o            Glossary of BuildFile Tags 
o            Some Example BuildFiles 
o            Asking for Externals 
o            Advanced Use of the build command 
o            Specific Tools in SCRAM 


%% Appendices:

%            Release Notes 
%            GNU Public Licence 

% Template CMSTOOLS file


\end{document}


Revision:	1.6
Committed:	Fri Jul 12 13:43:26 2002 UTC (22 years, 10 months ago) by sashby
Content type:	application/x-tex
Branch:	MAIN
Changes since 1.5:	+10 -10 lines
Log Message:	Improved CVS interface to allow local CVS repository to be used. Also started tests of new arch checking based on libc version.