minor fix for -cfg
refer to SWGuideCrabHowTo twiki instead of obsolete CrabHowTo
added check on used python version as suggested by Brian
if falg -c load only previusly cached cfg
enable hierarchical crab.cfg
missing -extend to the option list
remove unused os.environ
added support to run on dataset with new blocks , skipping the already analyzed. Plus some minor cleaning
Added [-renewProxy] command
fixed bug on already existing ui working dir check
add copyLocal functionality
move from printJdl to createJdl. More clear
Fix isCreating crash on undefined, add exit function for vanilla condor
print properties only if debug level > 6 or at the creation time
check if arguments are int, and report correctly if not
removed commented code
Added -printJdl functionality plus many bugs fixed
restored Resubmitter
implement -testJdl==-listMatch==-match functionality
minor fix for get output and cleaned range parsing
Many changes to have LSF working with BossLite Introduce Killer class to handle -kill which works again Work_space::res() return the correct output directory also in case user has set a non default one, Likewise for logDir() USER.outputdir is not to be used anywhere outside workspace class Some cleanup in submit logic, to reduce call of Scheduler specific classes from Submitter.py crab -clean works as well (well, almost, still need to remove twice the directory) Fill startDirectory and outputDirectory to Task GetOuput check status and not schedulerStatus (not stantard) Some cleanup in the use of BlackWhiteListParser No explicit check of scheduler concrete type in Submitter at listMatch level: move different behaviour in SchedulerXYZ implementation Plus other things I'm forgetting... Stefano
Now two different scheduler for GliteColl and Glite are not more needed
removed getOutput stuff and various fix...
added protection for server_name parameter
restored -printId functionality using bosslite
allignment to bosslite changes. many improvments on db interaction, minor changes on list usage to avoid error on job index
minor changes for ServerConfig usage... able to accept any capitalization
Chances in server classes in order to support server config from url
Many changes to integrate BossLite. Creation step fully implemented and optimized... Submission is now working again. Here the missing things are the support for jobs submission by range, the message sending to ML, and the listmatch_match check. Actually the requirements can be changed on the fly as was in the past. The status is fully working with BossLite. The exit code display is not there since the new boss does not implement the RealTime mon.. here the functionality is under development by Federica: to be integrated.
minor changes for sqlite interaction
removed jobDB and TaskDB usage. now works with sqlite
Fix problem in resubmission when passing a parameter the number of the job you want to resubmit eg -resubmit 4 now means (as was before) to resubmit job #4, not to resubmit 4 jobs. Create a new class Resubmitter (derived from Submitter) to deal with the logic of selecting the job to be resubmitetd from the user request and job status. This clean up a bit crab.py The actual submission is always done by Submitter::run()
Fix for the -resubmitt option with the server (submit_directive)
fix kill and resubmit
Redesign the inheritance three of Scheduler Classes, and remove the SchedulerBoss one Introduce ScheuledGrid and ScheduleLocal as base classes for Grid and local scheduler, respectively. All interaction with boss are done via Boss class, whose (unique) instance is owned by Scheduler These changes are done to reduce code duplication. Plus other minor modification and cosmetcs, as usual
move all Submitter inizialization from crab.py to Submitter.py constructor
remove yet one other try: except:
Add support for LSF/CAF direct submission Re-establish a correct inheritance pattern for Scheruled* classes Start to remove some unneeded try: catch: statement and replace them with appropriate if:then: Erase any use of cfg_param as a common block (expecially in conjuction with apmon) and replace it with the user of task DB Several minor cleanup
some inprovements for data publication
added -publish option
remove support for DBS1, and remove DBS2 string from files/class
import bossession must be uncommented
added comment about env management
Fixes for -create -submit use case plus some code cleaning
prioritarize the UI env and manage paths at creation level
removed old call to obsolete function
Fix to -create -submit tcsh env settings
Small fix on a putenv call
Changes for the environments interactions
Fix on printout (added typecasting)
Allowing submission for tasks with a missing location for a block - this is a first version that block the creation and then the submision of jobs assigned to no location
Checking that range or all is passed to the kill command
change C in X in testJdl
Fix to kill and submit ranges interactions
Fix for jobs with None as datasetpath
Fixed blocking python error about chosenJobsList var
Modified submitterServer for range submission and connected crab.py
Modified submission codes to support range specificatio, both for stand-alone and server
change the path before execute grid commands
add some protection and error handling in case of problem with working directory
fix for jobs with empty destination, prevent submission of jobs which have empty destination
Command Manager CW_watchdog CW updated for prekill and killrange.
Server import moved inside the code
Cleaner for tasks submitted through the server
speed up -printId by loading the DB and not querying the L&B
DBS2Pub is an actor, plus various protection against errors
added support for printId option when crab use server. In this case it return the task UUID.
if datasetpath is None, perform submission even if no destination is selected
move import DBS2Publisher fro beginning of file to inside the only method where is used
removed old username and dataname
changes for DBS2 output publication
Changed job creation and output to be independent of black and white lists. All jobs are created after data discovery and job spitting using black and white lists independent from the entries in the destination list of jobs. If a job does not have any valid destination sites, it is nontheless created. After job splitting, a screen output listing all blocks and all jobs belonging to a block is shown listing available sites per block. The user can submit x number of jobs while CRAB will skip in addition to the normal job submission all jobs which have an empty destination list.
add switch to be used in server mode for task kill purposes
add support for PostMortem fuctionality also while using server
add support for use_server, added Task_UUID function
write all cfg parameters to log in any case
fix double free problem in -clean: need boss >= BOSS_4_3_3-sl3-sl4-pre2. Cleaner is now an Actor
fix for Done (Abort) status mangement. now these job are killable
use always common.logger.message instead of print
fix bug to handle automatic -c with -submit
enable the choice of scheduler also from command line
remove multiple tries for registering plugins (not needed anymore with boss 4.3 and check the status of submission (failure or success) after actual submission and use this rsult to report to user
fix for "move output" in backup directory when non aborted jobs are re-submitted
switched to: from BossSession import *
Added try-except clause on boss api methods
removed some int and neadless lines
Fix for postMortem passing a range of jobs rather than all
add try:except around filter for RuntimeWarning, since it is not supported in python 2.2.3
Start using Boss API and avoid syncro problems with JobDB
add filtering for RuntimeWarning
optimization for status: instead of getting the status of the jobs one by one, load into memory that of all and use the cached result. Second optimization: use range of type <first:last> for consecutive jobs in all boss commnand, instead of passing the full list n,n+1,n+2,...m-1,m
must return job SID also if the status is R
fix problem for full path in working dir
add tracking of Z flags
Add check in case of jobs flagged as R in crabDB
fixed cancelAndResubmit functionality
print real scheduler name and not boss
remove use of unused flag "use_boss", now hardcoded to true
allow any value for -create but reset it to 'all' per default and print message
protect against arg to create
fix problem in choosing jobs to be submitted
deactivate pre-parsing of job status before handing over list of jobs to status check in scheduler
fixied bug to use a non default name for WorkDir
more mode for merging with boss4 branch
BOSS4 + sub-file splitting + taskDB
test_tag
added again w/o +x
temp remove to change +x
job starts from 1 and not from 0
minor fix in printout
bug fix for resubmit option
-get is an alias for longer -getoutput
cosmetics
minor problem in numbering for printId
added -printId option, to get the SID of a job
Integrate CMSSW: cfg names of job_list now from jobtype
Removed "2.2" in python2.2 pound-bang Changed "/usr/bin/python" to "/usr/bin/env python"
Substitute os.getlogin with os.environ['USER']
apmon instance moved right after main crab configurations
minor fox for -list
added abort state to if query, secured mv output files to selected output directory if directory is not available
Bug fix for resubmit
removed a print
removed self.creator.jobType() from resubmit option
removed jobtype object passed to Submitter
fixed bug with creator.jobType()
bug fix for multiple jobtype instances
Added a free() call for apmon
compulsory use of boss and monalisa
Fix Submitter parameters
Fix Submitter parameters
Modified to be compliant with the new Boss TaskId definition
control to getoutput dir
fixed problem with ui_working_dir
add screen message for -c option, and implemented cancelAndResubmit option
Fixed ML bug
fixed postmortem
bug fixed
added other status
Apmon is instanciated here now
crab submit bug fixed
removed some print
new function to move retrieved output
resubmit
resubmit
many modification for -clean: including small re-structuring of Status class
PostMortem with boss
now common block for proto-mon
cosmetics in error reporting
patch
kill job boss
back to 1.28
warning for submitting jobs not yet created
documentation
py2.2
documentation
useboss=0
fixing stuff for submit
Fix problem in submit param
mods to run with central bossDB or task-limited bossDB
crab-boss getoutput
boss_getoutput
moved EDG_WL_LOCATION ...
changed for copy and register data
fixed bugs
boss integration
crab-boss_1
added -use_boss option
crab-mon
too many changes to be listed here...
more check on DB plus other
more changes
too many changes to be listed...
too many changes to list them here...
more flexible syntax for job range
check -continue or -create for submit
'-resubmit' implemented
Commands '-list', '-status', '-kill', '-retrieve' implemented
The '-continue' option implemented
Bug in '-debug' fixed
initial set of files
This form allows you to request diffs between any two revisions of this file. For each of the two "sides" of the diff, select a symbolic revision name using the selection box, or choose 'Use Text Field' and enter a numeric revision.