Allow patterns like T1,T3

Fix for edmConfigHash with CMSSW 2.1

Fix on a typo

missed $...  $Njob is a variable

add IOError to exception list while creating the tar-ball

enable hierarchical crab.cfg

Correct print when output file is not in crab.cfg/TFileService

fix for correct tar ball creation with needd ProdCommon stuff (and not all)

fix for data

fixed tarbal preparation bugs

added support to run on dataset with new blocks , skipping the already analyzed. Plus some minor cleaning

improvement in dir walking to find data/ directory, thanks to Brian bbockelm@cse.unl.edu

fully added support for two tier inputs

syntax

fix for not processed files

Don't make broken links. Still needs full testing.

Switch to single quotes

Redo escapes

Escape python to more or less match original .sh

indentation

Printing CMSSW Pset in STDOUT is now optional. Specify USER.pset_debug=True to enable the print. Updated related documentation

must handle also CMSSW_X_Y_Z/src dir...

Finalize support for 2_1_x and supplying python config file in crab.cfg

Remove PsetManip150

chatched error with eval scramv1

 reimplemented dashboard communication sending again  both pre and post submssion infos, plus many Submitter code reorganization

Many changes to have LSF working with BossLite
Introduce Killer class to handle -kill which works again
Work_space::res() return the correct output directory also in case user has set a non default one, Likewise for logDir()
USER.outputdir is not to be used anywhere outside workspace class
Some cleanup in submit logic, to reduce call of Scheduler specific classes from Submitter.py
crab -clean works as well (well, almost, still need to remove twice the directory)
Fill startDirectory and outputDirectory to Task
GetOuput check status and not schedulerStatus (not stantard)
Some cleanup in the use of BlackWhiteListParser
No explicit check of scheduler concrete type in Submitter at listMatch level: move different behaviour in SchedulerXYZ implementation
Plus other things I'm forgetting...

Stefano

fixed problem with previous commit

cope with RELEASETOP found in different areas od .SCRAM

Fix problems with mv

the executable in jdl must be without absolute path. The wrapper must be included within the inifiles

changes to write the FJR from beginning of job wrapper

code number for OSG WORKING_DIR could not be deleted on WN is now 10017

removed jobDB and TaskDB usage. now works with sqlite

Python config file, remove reliance on sed

add requirement for CE in Production state for glitecoll scheduler

Consistent case

Redesign the inheritance three of Scheduler Classes, and remove the SchedulerBoss one
Introduce ScheuledGrid and ScheduleLocal as base classes for Grid and local scheduler, respectively.
All interaction with boss are done via Boss class, whose (unique) instance is owned by Scheduler
These changes are done to reduce code duplication.

Plus other minor modification and cosmetcs, as usual

removed ProdAgent API dependencies for DBS publication

changed a message about no produced output file

fixed problem with output copy and check output sandbox size

fixed a syntax mistake

Bug fix

Bug fix on output limit

Fixing problem on arguments check (WN)

fix bug in setting sourceSeed from cfg

added echos

removed cmssw and architecture check for version <1_5_X

Better error checking for temp directory

fixed printout for pythia job

fixed the output creation control

Removing not proper print out

Allowing submission for tasks with a missing location for a block - this is a first version that block the creation and then the submision of jobs assigned to no location

the job report will be modified only for data publication

Sending full datasetpath to the dashboard

bug fix for g4Seed and mixSeed in pset manipulator

bug fix in line 1192, removed additional '

re-aligned version

Stops job creation in case of arch-sw version mismatch

Problem setting up software for >= CMSSW_1_5_2 on OSG T2 sites solved.

Introduced general setting of

BUILD_ARCH

before setting up the CMS software made the scramv1 p command not see the correct
release area.

Changed so that only in the case of LCG, this is set, on OSG, it is not.

moved parsing of black/white se lists into

BlackWhiteListParser.py

which is intantiated in

SchedulerEdg.py: list parsing during xml creation
cms_cmssw.py:    list parsing for screen output

also SchedulerGlite.py has to use new member functions and had to be changed

cosmetics in checking output file existence

fix sorting in screen output

export SCRAM_ARCH and BUILD_ARCH before source cmsset_default.sh execution

fixed tipe error --> modifyReport_result

printout removed

changed ApplicationFamily name

changes to publish option

Changed job creation and output to be independent of black and white lists.

All jobs are created after data discovery and job spitting using black and
white lists independent from the entries in the destination list of jobs.
If a job does not have any valid destination sites, it is nontheless created.

After job splitting, a screen output listing all blocks and all jobs belonging to a block
is shown listing available sites per block.

The user can submit x number of jobs while CRAB will skip in addition to the normal job submission
all jobs which have an empty destination list.

re-introduce handling of various random seeds, lost in version 1.85

fix for reporting correct error code if output file is missing

merge version 1.84 with 1.83

move setting of SCRAM_ARCH before actual creatio of scram project

InputSandBox max size default to 9.5 Mb

DBS2 is default

input sandbox is set by default to 9.5 Mb: actual liit is 10, 1/2 Mb is for crab stuff

fix bug in checking existence of additional_input_files

fixed case when user defined executable is not within the CMSSW user project area

Provide check for default.tgz tarball size to limit input sandbox size
for all submission systems.

Condor_G has no sandbox size check as EGEE does, so a size check on the
default.tgz tarball which now also contains the data directories was introduced.

The default is 100 MB.

It can be changed by:

[EDG]

maxtarballsize = X

where X is the size in MB (float).

The switch is not documented in the template crab.cfg not to mislead users.

re-establish submission and execution of user script

cms_cmssw.py

Added card "use_dbs_2" to crab.cfg to steer usage of DBS-1 (default) or DBS-2

fix check existence of additional input files before doing anything with them

cleanup

add all data dir found in swarea to tar-ball

set relative path also for additional input files

ML files added to the tgz

fix to change name, adding _NUM, also to FJR

forgot printout

added FrameworkJobReport crab_fjr.xml per default to output sandbox, copy ProdAgentApi directory from  to swAera of local USER CMSSW project, include ProdAgentApi into PYTHONPATH on WN

commented out last call to analisys_common_info

correct variable replacement in CMSSW parameter-set

fixed splitting logic, added forgotten increase of counter, changed event caluclation for last job

bug fix to write correctly xml in case of None datasetPath jobs

Fixed cmsset.sh dir for OSG sites

introduced sub fileblock splitting and added discovery only for requested jobs (-create 100)

targetSite sent to dashboard is job specific

Burt Holzman fix for arguments and VO_CMS_SW_DIR/cmsset_default.sh

Burt Holzman fix for arguments and /uscmst1/prod/sw/cms/cmsset_default.sh

printout removed

cosmetics for VTx set

copy pset to pset.cfg even if untouched: bugfix

copy pset to pset.cfg even if untouched

better job splitting for cmssw

simplify and fix handling of list in JobDB

fixed bug related to total number of file and jobs

fix bug if asked all events but more files per job than total available

removed GridSyncJobId from the fields which are reported every time from the WN script to the DashBoard

changed common.logger.message for total_number_of_events

support for None input

added taskId to be reported at Submission time

moved SyncGridId creation in wrapper script to front that error exit codes during job setup are reported correctly to the DashBoard

add optional (secret) switches for DLS type (both in ORCA and CMSSW jobtype) and for DBS instances (in CMSSW jobtype).

bug fix in copying multiple files on SE. BTW same code in 4 different classescvscheckcvscheck BAD!