PLEASE TEST: Jobsub v1.3.4 to be released on Wednesday, Aug. 18

Dear Jobsub users,

WHAT ARE WE DOING?
Computing will be releasing a new Jobsub client version v1_3_4 on Wednesday, Aug. 18. We have made a release candidate (1_3_4_rc03) available as “test” in UPS (Unix Product Support). This version has been successfully tested on jobsubdevgpvm01. You can view the release notes at: https://cdcvs.fnal.gov/redmine/projects/jobsub/repository/revisions/master/entry/doc/release.notes

WHEN WILL THIS OCCUR?
Wednesday, Aug. 18; at noon Central Time

WHAT IS THE IMPACT TO YOU?
This upgrade will be transparent to users unless they upload tarballs via jobsub (see details below).

WHAT DO YOU NEED TO DO?
We highly encourage all Jobsub users to test their workflows against version 1_3_4_rc03. To do so, submit jobs using jobsub_client 1_3_4_rc03 using the following commands:

setup -t jobsub_client
jobsub_submit -G –jobsub-server=jobsubdevgpvm01.fnal.gov file:///path/to/executable

With the current production release (v1_3_3), the Rapid Code Distribution Service (RCDS) via CVMFS became the default method for distributing tarballs to jobs. This worked in most cases, with the notable exception of using the -f dropbox:// flags to upload a single file using RCDS, for example:

jobsub_submit -G -f dropbox:///path/to/myfile –jobsub-server=jobsubdevgpvm01.fnal.gov file:///path/to/executable

In the current production release, users were able to use the –use-pnfs-dropbox flag to work around this issue.

The current test release (v1_3_4_rc03) fixes this bug, and users can now also submit jobs with the -f dropbox:// flags to upload files to RCDS.

Keep in mind that files uploaded using RCDS will be accessed differently than with the previous resilient dCache dropbox. Please see https://cdcvs.fnal.gov/redmine/projects/jobsub/wiki/Rapid_Code_Distribution_Service_via_CVMFS_using_Jobsub for details on where the unwound tarball is available to your jobs, and submit test jobs to make sure the files are getting accessed from the correct location.

To test uploading tarballs via RCDS, use the following commands:

setup -t jobsub_client
jobsub_submit -G –jobsub-server=jobsubdevgpvm01.fnal.gov –tar_file_name [dropbox|tardir]:///path/to/[mytarfile|dir] file:///path/to/executable

Information on using resilient dCache dropbox, which will now be the legacy upload method, can be found in the help text of the jobsub_submit command (jobsub_submit -G –help), but we highly discourage uploading tarballs this way.