make sure PSETHASH is always defined: https://savannah.cern.ch/bugs/?95570

do not allow to exceed 100MB ISB, users with a real neeed now have Crab3: https://savannah.cern.ch/bugs/index.php?102210

100MB ISB for remoteGlidein, https://savannah.cern.ch/bugs/?100206

enforce umask permission when untarring sandbox: https://savannah.cern.ch/bugs/index.php?99592

propagate to watchdog and JDL the max_wall_clock from crab.cfg for remoteGlidein, see https://savannah.cern.ch/bugs/index.php?99008

max 5000 job on crab server https://savannah.cern.ch/bugs/?98597

allow 5K job/task for remoteGlidein

fixed savannah bug 94491

add diagnostic printouts when looking for /cvmfs

last fix was missing self. :-(

make sure proxy protection is not overridden, fix for https://savannah.cern.ch/bugs/?87233

rollback for savannah 85243

changed file permission in a+w for savannah bug #78585

remove "data" dir check

more info to dashboard

changed url for CMSSW available releases download, savannah bug 79277

Ignore release type, patch from Belforte

add ISB size check with server

Restrict characters in output files

Cleaner logic on retriveValue and throw exception for split by run with no runselection

Enforce split by lumi for data, enable split by lumi with runselection but no lumi mask

fix for bug 62338

Fix for bug #65439: updated URL for Server_available_for_users

add LumiMask support

Enable split by lumi for any dataset

Add check on CMSSW releases from https://cmstags.cern.ch/cgi-bin/CmsTC/ReleasesXML

\2\7\1\ \f\i\x\e\s\ \f\r\o\m\ \H\E\A\D

Code to allow lumi-filtering (pseudo-ADS)

Note on CAF's in Submitter, clarify

Branch for DBSMART, probably never will be used

Fix for unique output name when return_data=1 is used

Fix for bug #58043: now stageout uses unique names for the files sent to SE

changed for backup copy to close SE. adding temp in the path dir and wrapper exit code 60308

add support for WMBS

fill ouptputBaseName field

Change first_run: defaults to 1 for MC, integer increment, not string, and not used in cms_cmssw.py

added a check to validate the syntax of the input dataset name, savannah bug 55539

changes related CopyData, savannah task 8700

Changed link for server available to users

Drop support for CMSSW 1.x, 2.0 (.cfg language and old-style random #s)

Move from limiting # of created jobs to # of submitted jobs

Protect case where dataset=None

changes related publication name too long, savannah task 10664

Detect ADS, not user set, and fetch lumis for ADS

 type, self.CMSSW_major, self.CMSSW_minor, self.CMSSW_patch = tuple(self.version.split(_) cannot works for pre release & special tag

typo

fix for bug #52484. Add arguments.xml to the ISB files list. not more included in default.tgz

added the check of length of datasetname to publish

changes to allows use of CREAM ce by setting GRID.use_cream = 1

add support for script_arguments (to be used with script_exe). Fixed problem using CMSSW pre-release

add CMSSW.ignore_edm_output to allow production of EDM output w/o saving it anywhere

Introduce a CrabLogger class which hanlde the std python logging

improved logging

adapting code to EDG --> GRID migration

adapting code to logging usage. (crab_logger removed)

movig the call to edmConfigHash _after_ the untarring of user code bug #50798

re-enabled -extend functionality. Revisited default.tgz preparation

WMCore/__init__.py needed in the ISB

removed not needed code. Temporary commented check processed files. Cannot works given the arguments changes

argsList must be always 1... to be improved

add SEName, LFN and PFN to the FJR in the case of copy without publication for edm and not edm files

prepare correctly cms environment also for lsc/caf

iniatialize variables

ModifyJObReport call only for data publication, for the copy we have to consider also no edm file

remove debug print out

sge schedeuler specific support

More clear messages

diff -q --> diff -qbB as suggested by StefanoB

Remove support for CMSSW < 1.5, old method of adding FJR

fixed bug with exe_exit_status

set correctly maxEvent for mc prod jobs

preserver order of blocks

added warning if no output files are defined nor in the crab.cfg neither in the cmssw configuration

fix for no_block_boundary configurable

Raise exception if no location is found for any blocks

Add support for generator=comphep

Do it right for MadGraph

Add support for generators like madgraph

dereference link for tar-ball also for lib/ and module/

Don't pass cfg_params to SiteScreening

dereference link into ISB tarball

Transition to WMCore SiteScreening and SiteDB

added ProcessedDataset for publication

added abs path to $file_list

include ProdCommon/Storage within the ISB

fix for ui_working_dir absolute path

typo

*** empty log message ***

added protection if datasetpath=""

ISB limit automatically removed while using the server. Improved a bit the error message to limit (hopefully)  HN formum spamming

wrapper exit if cmsRun fails. bug #39675

changes related the new LFN and new datasetpath to publish (task 7471)

Allow patterns like T1,T3

added changes for PU_dataset and the LFN in the case of publication for CAF and LSF

Fix for edmConfigHash with CMSSW 2.1

Fix typo

some changes according to the head of CRAB (24/06/2008)

missed $...  $Njob is a variable

add IOError to exception list while creating the tar-ball

enable hierarchical crab.cfg

Correct print when output file is not in crab.cfg/TFileService

fix for correct tar ball creation with needd ProdCommon stuff (and not all)

fix for data

fixed tarbal preparation bugs

added support to run on dataset with new blocks , skipping the already analyzed. Plus some minor cleaning

improvement in dir walking to find data/ directory, thanks to Brian bbockelm@cse.unl.edu

fully added support for two tier inputs

syntax

fix for not processed files

Don't make broken links. Still needs full testing.

Switch to single quotes

Redo escapes

Escape python to more or less match original .sh

indentation

Printing CMSSW Pset in STDOUT is now optional. Specify USER.pset_debug=True to enable the print. Updated related documentation

must handle also CMSSW_X_Y_Z/src dir...

Finalize support for 2_1_x and supplying python config file in crab.cfg

Remove PsetManip150

chatched error with eval scramv1

 reimplemented dashboard communication sending again  both pre and post submssion infos, plus many Submitter code reorganization

Many changes to have LSF working with BossLite
Introduce Killer class to handle -kill which works again
Work_space::res() return the correct output directory also in case user has set a non default one, Likewise for logDir()
USER.outputdir is not to be used anywhere outside workspace class
Some cleanup in submit logic, to reduce call of Scheduler specific classes from Submitter.py
crab -clean works as well (well, almost, still need to remove twice the directory)
Fill startDirectory and outputDirectory to Task
GetOuput check status and not schedulerStatus (not stantard)
Some cleanup in the use of BlackWhiteListParser
No explicit check of scheduler concrete type in Submitter at listMatch level: move different behaviour in SchedulerXYZ implementation
Plus other things I'm forgetting...

Stefano

fixed problem with previous commit

cope with RELEASETOP found in different areas od .SCRAM

Include changes from 2_1_1 to 2_1_2 in head

Fix problems with mv

the executable in jdl must be without absolute path. The wrapper must be included within the inifiles

fix publication

changes to write the FJR from beginning of job wrapper

code number for OSG WORKING_DIR could not be deleted on WN is now 10017

removed jobDB and TaskDB usage. now works with sqlite

Python config file, remove reliance on sed

add requirement for CE in Production state for glitecoll scheduler

Consistent case

Redesign the inheritance three of Scheduler Classes, and remove the SchedulerBoss one
Introduce ScheuledGrid and ScheduleLocal as base classes for Grid and local scheduler, respectively.
All interaction with boss are done via Boss class, whose (unique) instance is owned by Scheduler
These changes are done to reduce code duplication.

Plus other minor modification and cosmetcs, as usual

removed ProdAgent API dependencies for DBS publication

changed a message about no produced output file

fixed problem with output copy and check output sandbox size

fixed a syntax mistake

Bug fix

Bug fix on output limit

Fixing problem on arguments check (WN)

fix bug in setting sourceSeed from cfg

added echos

removed cmssw and architecture check for version <1_5_X

Better error checking for temp directory

fixed printout for pythia job

fix bug in setting path for private executable on WN

fixed the output creation control

fixed print message when a block is not found

Removing not proper print out

Allowing submission for tasks with a missing location for a block - this is a first version that block the creation and then the submision of jobs assigned to no location

Allowing submission for tasks with a missing location for a block - this is a first version that block the creation and then the submision of jobs assigned to no location

added a control about output production, before copy

the job report will be modified only for data publication

Sending full datasetpath to the dashboard

bug fix for g4Seed and mixSeed in pset manipulator

bug fix in line 1192, removed additional '

re-aligned version

Stops job creation in case of arch-sw version mismatch

Problem setting up software for >= CMSSW_1_5_2 on OSG T2 sites solved.

Introduced general setting of

BUILD_ARCH

before setting up the CMS software made the scramv1 p command not see the correct
release area.

Changed so that only in the case of LCG, this is set, on OSG, it is not.

moved parsing of black/white se lists into

BlackWhiteListParser.py

which is intantiated in

SchedulerEdg.py: list parsing during xml creation
cms_cmssw.py:    list parsing for screen output

also SchedulerGlite.py has to use new member functions and had to be changed

cosmetics in checking output file existence

fix sorting in screen output

export SCRAM_ARCH and BUILD_ARCH before source cmsset_default.sh execution

fixed tipe error --> modifyReport_result

printout removed

changed ApplicationFamily name

changes to publish option

Changed job creation and output to be independent of black and white lists.

All jobs are created after data discovery and job spitting using black and
white lists independent from the entries in the destination list of jobs.
If a job does not have any valid destination sites, it is nontheless created.

After job splitting, a screen output listing all blocks and all jobs belonging to a block
is shown listing available sites per block.

The user can submit x number of jobs while CRAB will skip in addition to the normal job submission
all jobs which have an empty destination list.

re-introduce handling of various random seeds, lost in version 1.85

fix for reporting correct error code if output file is missing

merge version 1.84 with 1.83

move setting of SCRAM_ARCH before actual creatio of scram project

InputSandBox max size default to 9.5 Mb

DBS2 is default

input sandbox is set by default to 9.5 Mb: actual liit is 10, 1/2 Mb is for crab stuff

fix bug in checking existence of additional_input_files

fixed case when user defined executable is not within the CMSSW user project area

Provide check for default.tgz tarball size to limit input sandbox size
for all submission systems.

Condor_G has no sandbox size check as EGEE does, so a size check on the
default.tgz tarball which now also contains the data directories was introduced.

The default is 100 MB.

It can be changed by:

[EDG]

maxtarballsize = X

where X is the size in MB (float).

The switch is not documented in the template crab.cfg not to mislead users.

re-establish submission and execution of user script

cms_cmssw.py

Added card "use_dbs_2" to crab.cfg to steer usage of DBS-1 (default) or DBS-2

fix check existence of additional input files before doing anything with them

cleanup

add all data dir found in swarea to tar-ball

set relative path also for additional input files

add code version to DB

put sw version on TaskDB

The FJR produced by each jobs is always returned via output sandbox even if the ouput is copied to a SE, as already happens for stdout and err

strip whitespaces from file names

cosmetics

remove common.analisys_common_info

allow wildcards in additional input files

working dir creation on OSG moved to mktemp