How to compute/Job management

From MetaCentrum
Jump to: navigation, search

(Česká verze)

Warning.gif WARNING: This page will be removed soon

Related topics
A job tracking video tutorial

In this page we discuss how to

  • track the status of a job
  • get the data in case of failure
  • clean the scratch manually

Tracking job status

You can track your jobs via online application PBSmon:


qstat

  • Completed job is in the stage "F" (finished).
  • Command qstat views only waiting and running jobs. Displaying finished jobs is done by calling qstat -x.
  • For smaller groups of jobs, PBS Pro can display expected start (Est Start Time). This can be done with command qstat -T.

It is also possible to track your job via its ID (jobID) and terminal. For example:

qstat -u <login> lists all user running or queueing jobs on actual PBS server
qstat -u <login> @arien-pro.ics.muni.cz @wagap-pro.cerit-sc.cz @pbs.elixir-czech.cz list all running or queueing jobs on all PBS servers
qstat -x -u <login> list finished user jobs 
qstat -f <jobID> list details of the running or queueing job
qstat -x -f <jobID> list details of the finished job

Tracking running jobs

Follow these steps if you would like to check outputs of a job, which has not finished yet:

1. Find what machine is your job running on -> use e.g. PBSmon (https://metavo.metacentrum.cz/pbsmon2/person)

2. Login to the machine from any frontend with ssh target_machine command. E.g.:

ssh zapat112.cerit-sc.cz

3. Navigate to the /var/spool/pbs/spool/ directory and examine the files:

  • $PBS_JOBID.OU for standard output (stdout – e.g., “1234.arien-pro.ics.muni.cz.OU”)
  • $PBS_JOBID.ER for standard error output (stderr – e.g., “1234.arien-pro.ics.muni.cz.ER”)