Dashboard changes from Sanjay for Glidein

change from Sanjay

Move definition of dontCheckMyProxy

typos

adapting code to logging usage. (crab_logger removed)

Add some ARC specific bits to the jobs scripts.

minor fix

Schedulers inherit from Schedulers.py.... plus minor changes on cmscp call

users can decided to use the backup function. Publication of data is not supported with backup. Work in progress

print of cmscpReport

Added protection for cmscp.py crash

fix casting problem in forced BL setting

fixed misplaced quotes

Move checkProxy to base class, use it in condor if copy_output selected

make the T1s CE Black List configurable. use EDG.remove_default_blacklist = 1 to remove it.

Don't pass cfg_params to SiteScreening

Run OSG startup script, fix stage-out

propagate to cmscp the srm version... default is always v2

report the full output of the underlying stageout command in case of stage out failure

pass right arguments to cmscp

fixed a typo

Implemented all the changes needed to use cmscp.py. Integrated the Phedex API usage to obtain the complete endpoint needed for stage out. Main changes are in wsCopyOutput().

Throw away STDERR from voms-proxy-info

Unique task names if ui_working_dir defined

Allow patterns like T1,T3

move rfcp function from SchedulerLocal to cmscp in crab_template and added some changes for publication using CAF and LSF

added voms group support. check the validity of existing proxy [tnx to Stijn]

Clean up broken links

Fix on timing for stageout also

bug fix for exit_func

syntax

At the submission level is now used the "fast" listmatch while the full one is performed while testing Jdl (-testJdl or -match)

Bug fix on proxy check

print on screen adjustements

fix typo in parentesis as reported by Bockjoo

removed wrong blank

Warn if storage_path and srm_version don't agree. Use SMR v2 correctly.

check proxy adjustments

 reimplemented dashboard communication sending again  both pre and post submssion infos, plus many Submitter code reorganization

Many changes to have LSF working with BossLite
Introduce Killer class to handle -kill which works again
Work_space::res() return the correct output directory also in case user has set a non default one, Likewise for logDir()
USER.outputdir is not to be used anywhere outside workspace class
Some cleanup in submit logic, to reduce call of Scheduler specific classes from Submitter.py
crab -clean works as well (well, almost, still need to remove twice the directory)
Fill startDirectory and outputDirectory to Task
GetOuput check status and not schedulerStatus (not stantard)
Some cleanup in the use of BlackWhiteListParser
No explicit check of scheduler concrete type in Submitter at listMatch level: move different behaviour in SchedulerXYZ implementation
Plus other things I'm forgetting...

Stefano

Split wsSetupEnvironment into Grid, Glite, and CondorG parts

fix for data publication

restored output functionality. Some fix on arguments format avoiding wrapper crash. Implemented tarball creation for output sanbox... still remaining to enable the check on output sandbox size...and the related backup solution

Improvements in sched_parameter

Move checkProxy() to SchedulerGlite

restored the possibility to change WMS (the simply endpoint and/or the full config files) at submission time via command line, plus some adjustments on submission conde

Many changes to integrate BossLite. Creation step fully implemented and optimized... Submission is now working again. Here the missing things are the support for jobs submission by range, the message sending to ML, and the listmatch_match check. Actually the requirements can be changed on the fly as was in the past. The status is fully working with BossLite. The exit code display is not there since the new boss does not implement the RealTime mon.. here the functionality is under development by Federica: to be integrated.

ported to bosslite. deprecated all the stuff related to xml creation needed for the old boss, reorganized requirements (scheduler specific) management. Any file is now writted.. scheduler parameters are filled into DB

Python config file, remove reliance on sed

added support for SRMV2 client. Now it is possible to specify srm_version from crab.cfg ( [USER].srm_version ) , the logic is always the same: try srmv* if it fails try lcg_cp ( at the moment if no version is specified and the srmv1 fails also srmv2 is tried... )

call Scheduler::configure from SchedulerCondor_g

change cmscp exit code, 60307 if the copy to SE fails, 60303 if the file already exists in the SE dir

Redesign the inheritance three of Scheduler Classes, and remove the SchedulerBoss one
Introduce ScheuledGrid and ScheduleLocal as base classes for Grid and local scheduler, respectively.
All interaction with boss are done via Boss class, whose (unique) instance is owned by Scheduler
These changes are done to reduce code duplication.

Plus other minor modification and cosmetcs, as usual

added changes about black and white ce list

added echo

changed exitstring with copy_exit_status