ViewVC Help
View File | Revision Log | Show Annotations | Root Listing
root/cvsroot/COMP/CRAB/python/crab_util.py
Revision 1.51 - (view) (annotate) - [select for diffs]
Mon Jul 14 09:39:45 2008 UTC (16 years, 9 months ago) by spiga
Branch: MAIN
CVS Tags: CRAB_2_3_2_pre1
Branch point for: AnaDataSet
Changes since 1.50: +13 -0 lines
Diff to previous 1.50
added fix for lsf listmatch. now works correctly also for private production (datasetPath=None)

Revision 1.50 - (view) (annotate) - [select for diffs]
Tue Jul 8 09:20:08 2008 UTC (16 years, 9 months ago) by spiga
Branch: MAIN
CVS Tags: CRAB_2_4_0_test
Changes since 1.49: +0 -1 lines
Diff to previous 1.49
removed wrong print in the log file

Revision 1.49 - (view) (annotate) - [select for diffs]
Wed Jul 2 12:36:17 2008 UTC (16 years, 10 months ago) by spiga
Branch: MAIN
Changes since 1.48: +13 -15 lines
Diff to previous 1.48
print on screen also if xml has been requested

Revision 1.48 - (view) (annotate) - [select for diffs]
Wed Jul 2 10:39:33 2008 UTC (16 years, 10 months ago) by spiga
Branch: MAIN
Changes since 1.47: +2 -0 lines
Diff to previous 1.47
added xml print into the log file

Revision 1.47 - (view) (annotate) - [select for diffs]
Mon Jun 16 14:02:36 2008 UTC (16 years, 10 months ago) by slacapra
Branch: MAIN
Changes since 1.46: +2 -2 lines
Diff to previous 1.46
enable hierarchical crab.cfg

Revision 1.46 - (view) (annotate) - [select for diffs]
Mon Jun 9 14:40:31 2008 UTC (16 years, 10 months ago) by spiga
Branch: MAIN
CVS Tags: CRAB_2_2_2_pre5, CRAB_2_3_0, CRAB_2_3_0_pre1, CRAB_2_3_0_pre6, CRAB_2_3_1_pre1, CRAB_2_3_1_pre2, CRAB_2_3_1_pre3
Branch point for: CRAB_2_3_0_br
Changes since 1.45: +25 -0 lines
Diff to previous 1.45
added support to run on dataset with new blocks , skipping the already analyzed. Plus some minor cleaning

Revision 1.45 - (view) (annotate) - [select for diffs]
Fri Jun 6 11:45:22 2008 UTC (16 years, 10 months ago) by slacapra
Branch: MAIN
Changes since 1.44: +55 -37 lines
Diff to previous 1.44
move func fileNumber from cms_cmssw to crab_util

Revision 1.44 - (view) (annotate) - [select for diffs]
Wed Jun 4 18:59:04 2008 UTC (16 years, 10 months ago) by ewv
Branch: MAIN
CVS Tags: CRAB_2_2_2_pre3, CRAB_2_2_2_pre4
Changes since 1.43: +20 -40 lines
Diff to previous 1.43
New, faster checksumming routine

Revision 1.43 - (view) (annotate) - [select for diffs]
Sat Apr 26 17:27:09 2008 UTC (17 years ago) by spiga
Branch: MAIN
CVS Tags: CRAB_2_2_0, CRAB_2_2_0_pre13, CRAB_2_2_0_pre15, CRAB_2_2_0_pre16, CRAB_2_2_0_pre17, CRAB_2_2_0_pre18, CRAB_2_2_0_pre19, CRAB_2_2_0_pre21, CRAB_2_2_1, CRAB_2_2_1_pre1, CRAB_2_2_1_pre2, CRAB_2_2_1_pre3, CRAB_2_2_1_pre4, CRAB_2_2_1_pre5, CRAB_2_2_1_pre6, CRAB_2_2_2_pre1, CRAB_2_2_2_pre2, PRODCOMMON_0_10_7_testCS2
Changes since 1.42: +22 -10 lines
Diff to previous 1.42
Added -printJdl functionality plus many bugs fixed

Revision 1.42 - (view) (annotate) - [select for diffs]
Sat Apr 26 08:25:00 2008 UTC (17 years ago) by spiga
Branch: MAIN
Changes since 1.41: +21 -13 lines
Diff to previous 1.41
Correct proxy management....not safe to check it everywhere. Added support for caf (e.g. local ) server. Code reorganization..

Revision 1.41 - (view) (annotate) - [select for diffs]
Thu Apr 17 12:18:51 2008 UTC (17 years ago) by spiga
Branch: MAIN
CVS Tags: CRAB_2_2_0_pre10, CRAB_2_2_0_pre11, CRAB_2_2_0_pre12, CRAB_2_2_0_pre8, CRAB_2_2_0_pre9, bp_osg_bdii
Branch point for: osg_bdii
Changes since 1.40: +12 -0 lines
Diff to previous 1.40
SE API usage to support client communication with a generic SE are there both for submission and getoutput. Better exception catching and refactorized a bit the code

Revision 1.40 - (view) (annotate) - [select for diffs]
Tue Apr 15 16:34:58 2008 UTC (17 years ago) by spiga
Branch: MAIN
CVS Tags: CRAB_2_2_0_pre7
Changes since 1.39: +22 -12 lines
Diff to previous 1.39
restored the inheritance between Status and StatusServer. Enabled the possibility to write an xml file instead of print the screen report, during the status query. Started to restore to sand infos to ML

Revision 1.39 - (view) (annotate) - [select for diffs]
Mon Mar 3 20:25:15 2008 UTC (17 years, 2 months ago) by spiga
Branch: MAIN
CVS Tags: CRAB_2_2_0_pre1, CRAB_2_2_0_pre2, CRAB_2_2_0_pre4, CRAB_2_2_0_pre5
Changes since 1.38: +16 -0 lines
Diff to previous 1.38
added new method to be used to show db-query results

Revision 1.38 - (view) (annotate) - [select for diffs]
Wed Feb 13 18:17:54 2008 UTC (17 years, 2 months ago) by slacapra
Branch: MAIN
CVS Tags: CRAB_2_1_0, CRAB_2_1_0_pre5, CRAB_2_1_0_pre6, CRAB_2_1_1, CRAB_2_1_1_pre1, CRAB_2_1_1_pre3, CRAB_2_1_2, CRAB_2_1_2_pre1, CRAB_2_1_2_pre2
Branch point for: CRAB_2_1_1_pre2, CRAB_2_1_2_br
Changes since 1.37: +2 -0 lines
Diff to previous 1.37
Fix problem in resubmission when passing a parameter the number of the job you want to resubmit
eg -resubmit 4
now means (as was before) to resubmit job #4, not to resubmit 4 jobs.
Create a new class Resubmitter (derived from Submitter) to deal with the logic of selecting the job to be resubmitetd from the user request and job status. This clean up a bit crab.py
The actual submission is always done by Submitter::run()

Revision 1.37 - (view) (annotate) - [select for diffs]
Tue Feb 12 15:48:53 2008 UTC (17 years, 2 months ago) by slacapra
Branch: MAIN
Changes since 1.36: +0 -3 lines
Diff to previous 1.36
fix scheduler name in case of glitecoll plus cleanup

Revision 1.36 - (view) (annotate) - [select for diffs]
Tue Nov 6 10:14:32 2007 UTC (17 years, 5 months ago) by spiga
Branch: MAIN
CVS Tags: CRAB_2_0_2, CRAB_2_0_2_pre2, CRAB_2_0_2_pre3, CRAB_2_0_2_pre4, CRAB_2_0_2_pre5, CRAB_2_0_2_pre6, CRAB_2_0_3, CRAB_2_0_3_pre1, CRAB_2_0_4, CRAB_2_0_4_pre1, CRAB_2_0_4_pre2, CRAB_2_1_0_pre1, CRAB_2_1_0_pre2, CRAB_2_1_0_pre3, CRAB_2_1_0_pre4
Changes since 1.35: +0 -23 lines
Diff to previous 1.35
cleaned path management function

Revision 1.35 - (view) (annotate) - [select for diffs]
Wed Oct 10 12:55:58 2007 UTC (17 years, 6 months ago) by mcinquil
Branch: MAIN
CVS Tags: CRAB_2_0_1, CRAB_2_0_1_pre1, CRAB_2_0_2_pre1
Changes since 1.34: +3 -1 lines
Diff to previous 1.34
Allowing submission for tasks with a missing location for a block - this is a first version that block the creation and then the submision of jobs assigned to no location

Revision 1.34 - (view) (annotate) - [select for diffs]
Tue Sep 18 15:18:47 2007 UTC (17 years, 7 months ago) by spiga
Branch: MAIN
CVS Tags: CRAB_1_5_4, CRAB_1_5_4_pre2, CRAB_2_0_0, CRAB_2_0_0_pre10, CRAB_2_0_0_pre9
Changes since 1.33: +1 -1 lines
Diff to previous 1.33
remove *only* cmssw python dir

Revision 1.33 - (view) (annotate) - [select for diffs]
Fri Sep 14 16:05:19 2007 UTC (17 years, 7 months ago) by spiga
Branch: MAIN
CVS Tags: CRAB_1_5_4_pre1
Changes since 1.32: +22 -0 lines
Diff to previous 1.32
add function to cache and manipulate path. It allow to avoid CMSSW/LCG env compatibility problems

Revision 1.32 - (view) (annotate) - [select for diffs]
Tue Jun 19 16:21:49 2007 UTC (17 years, 10 months ago) by gutsche
Branch: MAIN
CVS Tags: CRAB_1_5_3, CRAB_1_5_3_pre2, CRAB_1_5_3_pre3, CRAB_1_5_3_pre4, CRAB_1_5_3_pre5, CRAB_2_0_0_pre4, CRAB_2_0_0_pre5, CRAB_2_0_0_pre6, CRAB_2_0_0_pre7, configure
Branch point for: CRAB_1_5_4_SLC3_start
Changes since 1.31: +25 -0 lines
Diff to previous 1.31
Changed job creation and output to be independent of black and white lists.

All jobs are created after data discovery and job spitting using black and
white lists independent from the entries in the destination list of jobs.
If a job does not have any valid destination sites, it is nontheless created.

After job splitting, a screen output listing all blocks and all jobs belonging to a block
is shown listing available sites per block.

The user can submit x number of jobs while CRAB will skip in addition to the normal job submission
all jobs which have an empty destination list.

Revision 1.31 - (view) (annotate) - [select for diffs]
Thu Jan 18 18:28:41 2007 UTC (18 years, 3 months ago) by slacapra
Branch: MAIN
CVS Tags: CRAB_1_5_0, CRAB_1_5_0_pre5, CRAB_1_5_0_pre6, CRAB_1_5_0_pre7, CRAB_1_5_0_pre8, CRAB_1_5_0_pre9, CRAB_1_5_1, CRAB_1_5_1_pre1, CRAB_1_5_1_pre2, CRAB_1_5_1_pre3, CRAB_1_5_1_pre4, CRAB_1_5_2, CRAB_1_5_3_pre1, CRAB_2_0_0_pre1, CRAB_2_0_0_pre2, CRAB_2_0_0_pre3
Changes since 1.30: +1 -0 lines
Diff to previous 1.30
undefined variable

Revision 1.30 - (view) (annotate) - [select for diffs]
Wed Jan 17 18:17:59 2007 UTC (18 years, 3 months ago) by slacapra
Branch: MAIN
CVS Tags: CRAB_1_5_0_pre4
Changes since 1.29: +9 -9 lines
Diff to previous 1.29
many minor fixes reported by pychecker, mostly unsude import and unused variables

Revision 1.29 - (view) (annotate) - [select for diffs]
Wed Jan 17 13:38:13 2007 UTC (18 years, 3 months ago) by slacapra
Branch: MAIN
Changes since 1.28: +20 -12 lines
Diff to previous 1.28
use python tempfile to create temporary files

Revision 1.28 - (view) (annotate) - [select for diffs]
Mon Dec 4 11:57:02 2006 UTC (18 years, 4 months ago) by spiga
Branch: MAIN
CVS Tags: CRAB_1_5_0_pre2, CRAB_1_5_0_pre3
Changes since 1.27: +2 -0 lines
Diff to previous 1.27
run boss command become obso

Revision 1.27 - (view) (annotate) - [select for diffs]
Mon Nov 20 18:17:35 2006 UTC (18 years, 5 months ago) by corvo
Branch: MAIN
CVS Tags: CRAB_1_5_0_pre1
Changes since 1.26: +41 -31 lines
Diff to previous 1.26
change crab status table

Revision 1.26 - (view) (annotate) - [select for diffs]
Tue Nov 7 10:48:46 2006 UTC (18 years, 5 months ago) by slacapra
Branch: MAIN
CVS Tags: CRAB_1_4_1, CRAB_1_4_1_pre2, CRAB_1_4_2
Branch point for: branch_1_4_1
Changes since 1.25: +20 -0 lines
Diff to previous 1.25
optimization for status: instead of getting the status of the jobs one by one, load into memory that of all and use the cached result. Second optimization: use range of type <first:last> for consecutive jobs in all boss commnand, instead of passing the full list n,n+1,n+2,...m-1,m

Revision 1.25 - (view) (annotate) - [select for diffs]
Thu Oct 19 15:23:27 2006 UTC (18 years, 6 months ago) by spiga
Branch: MAIN
CVS Tags: CRAB_1_4_0, CRAB_1_4_1_pre1
Changes since 1.24: +1 -0 lines
Diff to previous 1.24
Add check in case of jobs flagged as R in crabDB

Revision 1.24 - (view) (annotate) - [select for diffs]
Mon Oct 9 23:22:44 2006 UTC (18 years, 6 months ago) by gutsche
Branch: MAIN
CVS Tags: CRAB_1_4_0_pre2, CRAB_1_4_0_pre3, CRAB_1_4_0_pre4
Changes since 1.23: +1 -1 lines
Diff to previous 1.23
increase timeout for runBossCommand to 1 hour. Due to the group submission, a boss submit can take longer than 10 minutes

Revision 1.23 - (view) (annotate) - [select for diffs]
Wed Oct 4 16:58:15 2006 UTC (18 years, 6 months ago) by gutsche
Branch: MAIN
CVS Tags: CRAB_1_4_0_pre1
Changes since 1.22: +1 -1 lines
Diff to previous 1.22
fixed savanah bug #20328, use of mktemp with argument

Revision 1.22 - (view) (annotate) - [select for diffs]
Tue Sep 26 21:57:51 2006 UTC (18 years, 7 months ago) by gutsche
Branch: MAIN
CVS Tags: CRAB_1_3_0, CRAB_1_3_0_pre6
Changes since 1.21: +1 -2 lines
Diff to previous 1.21
use mktemp to create temporary file

Revision 1.21 - (view) (annotate) - [select for diffs]
Wed Jun 28 13:41:03 2006 UTC (18 years, 10 months ago) by slacapra
Branch: MAIN
CVS Tags: CRAB_1_2_0, CRAB_1_2_0_pre8, CRAB_1_2_0_pre9, CRAB_1_2_1, CRAB_1_3_0_pre3, CRAB_1_3_0_pre4, CRAB_1_3_0_pre5, HEAD_20092006
Branch point for: CRAB_BOSS4, CRAB_BOSS4_v1
Changes since 1.20: +1 -1 lines
Diff to previous 1.20
reduce time to fil buffer

Revision 1.20 - (view) (annotate) - [select for diffs]
Wed May 24 21:04:23 2006 UTC (18 years, 11 months ago) by gutsche
Branch: MAIN
CVS Tags: CRAB_1_2_0_pre7, post_cmssw_integration_20060527, pre_cmssw_integration_20060527
Changes since 1.19: +32 -0 lines
Diff to previous 1.19
added Cksum function to calculate the Cksum of given file with full path also adding full path of file as first line of tmp file the hash is generated from

Revision 1.19 - (view) (annotate) - [select for diffs]
Mon Mar 13 19:47:15 2006 UTC (19 years, 1 month ago) by corvo
Branch: MAIN
CVS Tags: CRAB_1_0_5, CRAB_1_0_6, CRAB_1_0_7, CRAB_1_0_7_pre1, CRAB_1_1_0, CRAB_1_1_0_pre1, CRAB_1_1_0_pre3, CRAB_1_1_0_pre4
Changes since 1.18: +4 -1 lines
Diff to previous 1.18
Prevent user from switvhing off ML

Revision 1.18 - (view) (annotate) - [select for diffs]
Mon Dec 12 14:06:57 2005 UTC (19 years, 4 months ago) by slacapra
Branch: MAIN
CVS Tags: CRAB_1_0_3, CRAB_1_0_4
Changes since 1.17: +1 -1 lines
Diff to previous 1.17
fix int printout

Revision 1.17 - (view) (annotate) - [select for diffs]
Fri Dec 9 15:18:45 2005 UTC (19 years, 4 months ago) by fanzago
Branch: MAIN
Changes since 1.16: +5 -2 lines
Diff to previous 1.16
fixed problem with cmd_out and cmd_err in runCommand

Revision 1.16 - (view) (annotate) - [select for diffs]
Tue Dec 6 16:47:25 2005 UTC (19 years, 4 months ago) by slacapra
Branch: MAIN
Changes since 1.15: +52 -19 lines
Diff to previous 1.15
improved runCommand plus cosmetics

Revision 1.15 - (view) (annotate) - [select for diffs]
Fri Nov 18 13:09:05 2005 UTC (19 years, 5 months ago) by fanzago
Branch: MAIN
Changes since 1.14: +3 -2 lines
Diff to previous 1.14
fixed list_match bug

Revision 1.14 - (view) (annotate) - [select for diffs]
Thu Nov 3 00:33:22 2005 UTC (19 years, 6 months ago) by slacapra
Branch: MAIN
CVS Tags: CRAB_0_2_2, CRAB_1_0_0_rc1, CRAB_1_0_1, CRAB_1_0_2
Changes since 1.13: +0 -18 lines
Diff to previous 1.13
move away setOutLogDir

Revision 1.13 - (view) (annotate) - [select for diffs]
Wed Oct 19 16:05:39 2005 UTC (19 years, 6 months ago) by slacapra
Branch: MAIN
Changes since 1.12: +2 -0 lines
Diff to previous 1.12
more debug

Revision 1.12 - (view) (annotate) - [select for diffs]
Wed Oct 19 10:01:56 2005 UTC (19 years, 6 months ago) by slacapra
Branch: MAIN
Changes since 1.11: +2 -2 lines
Diff to previous 1.11
add timeout

Revision 1.11 - (view) (annotate) - [select for diffs]
Thu Oct 13 17:10:03 2005 UTC (19 years, 6 months ago) by slacapra
Branch: MAIN
CVS Tags: CRAB_1_0_0_beta4
Changes since 1.10: +55 -19 lines
Diff to previous 1.10
mods to run with central bossDB or task-limited bossDB

Revision 1.10 - (view) (annotate) - [select for diffs]
Thu Oct 13 12:08:20 2005 UTC (19 years, 6 months ago) by spiga
Branch: MAIN
Changes since 1.9: +20 -18 lines
Diff to previous 1.9
crab-boss getoutput

Revision 1.9 - (view) (annotate) - [select for diffs]
Sun Oct 9 17:32:21 2005 UTC (19 years, 6 months ago) by spiga
Branch: MAIN
Changes since 1.8: +20 -0 lines
Diff to previous 1.8
boss_getoutput

Revision 1.8 - (view) (annotate) - [select for diffs]
Mon Oct 3 12:17:44 2005 UTC (19 years, 7 months ago) by fanzago
Branch: MAIN
CVS Tags: CRAB_1_0_0_pre1_boss_2
Changes since 1.7: +4 -12 lines
Diff to previous 1.7
fixed bugs

Revision 1.7 - (view) (annotate) - [select for diffs]
Tue Aug 30 14:26:10 2005 UTC (19 years, 8 months ago) by spiga
Branch: MAIN
CVS Tags: CRAB_1_0_0_pre1_boss
Changes since 1.6: +1 -0 lines
Diff to previous 1.6
crab-mon

Revision 1.6 - (view) (annotate) - [select for diffs]
Wed Aug 10 16:52:51 2005 UTC (19 years, 8 months ago) by slacapra
Branch: MAIN
CVS Tags: CRAB_1_0_0_pre2, CRAB_1_0_0_pre3
Changes since 1.5: +1 -0 lines
Diff to previous 1.5
more check on DB plus other

Revision 1.5 - (view) (annotate) - [select for diffs]
Mon Jul 25 14:31:24 2005 UTC (19 years, 9 months ago) by slacapra
Branch: MAIN
CVS Tags: CRAB_1_0_0_pre1
Changes since 1.4: +1 -0 lines
Diff to previous 1.4
too many changes to be listed...

Revision 1.4 - (view) (annotate) - [select for diffs]
Thu Jun 30 10:35:27 2005 UTC (19 years, 10 months ago) by nsmirnov
Branch: MAIN
Changes since 1.3: +36 -0 lines
Diff to previous 1.3
more flexible syntax for job range

Revision 1.3 - (view) (annotate) - [select for diffs]
Thu Jun 23 16:53:25 2005 UTC (19 years, 10 months ago) by nsmirnov
Branch: MAIN
Changes since 1.2: +38 -0 lines
Diff to previous 1.2
Commands '-list', '-status', '-kill', '-retrieve' implemented

Revision 1.2 - (view) (annotate) - [select for diffs]
Tue Jun 21 11:06:51 2005 UTC (19 years, 10 months ago) by nsmirnov
Branch: MAIN
Changes since 1.1: +5 -6 lines
Diff to previous 1.1
Job submission implemented

Revision 1.1 - (view) (annotate) - [select for diffs]
Thu Mar 10 16:20:30 2005 UTC (20 years, 1 month ago) by nsmirnov
Branch: MAIN
initial set of files

Convenience Links

Links to HEAD: (view) (annotate) Links to AnaDataSet: (view) (annotate)

Compare Revisions

This form allows you to request diffs between any two revisions of this file. For each of the two "sides" of the diff, select a symbolic revision name using the selection box, or choose 'Use Text Field' and enter a numeric revision.

  Diffs between and
  Type of Diff should be a