This QuickStart section is inspired by the online QuickStart option shown on the HTCondor web pages.
This section assumes that:¶
|Assumption||Check / Set-up on BCC|
|HTCondor is running||HTCondor is installed on BCC|
|You have access to a machine within the pool that may submit jobs, termed a submit machine.||IP Address:
|You are logged in to and working on the submit machine||Your username is your UW-Madison
|Your program executable, your submit description file, and any needed input files are all on the file system of the submit machine|
|You should work in the
|Your job (the program executable) is able to run without any interactive input. Standard input (from the keyboard), standard output (seen on the display), and standard|
|error (seen on the display) may still be used, but their contents will be redi-rected from/to files.||We will run a test file that complies with these requirements.|
It is also assumed that you know how to converse within the line command and edit simple text files e.g. with text editor
Connect and set-up a working space¶
We’ll follow the above table process.
First, we connect and create a directory in the shared data space.
NetID to connect – represented here as myname
You should open a text-based terminal from your local machine and then issue the connect command: (replace myname with your login name.)
Then move to
/scratch and create a directory with your name and another directory within to work with as your project e.g.
Create a simple executable test file¶
nano word processor on the cluster or a copy/paste method we now create a file to be executed by HTCondor.
Within the file enter the following:
If you are using
nano, use Ctrl-X to exit from edit mode to save the file.
Now we make sure that the file is executable:
Create a simple submit file¶
The submit file contains information for running a job and is passed to HTCondor. The following file is a minimal file, more information could be provided to HTCondor for more job control.
Then enter the following text within the file and save:
Blank lines are ignored.
Upper/Lower case is ignore on the left side of the equal sign.
Line 1 specifies the file to run
Line 2 requests necessary files to be transferred
Line 3 – 5 specify the name of the standard output files
Line 6 places the job in the queue so it can be run.
The following implicit assumptions are made:
- general-use bash commands are available, *e.g.
- the process will use standard input, standard output and standard error (
Submit the job¶
We now follow the example with the cascade steps of submitting a file (
hello.sub) containing information about an executable (
hello.sh) that is calling on a software (
echo) that will create some output files (
hello.out, etc.) that will be transferred to the local directory when the job is done.
The submit command is as follows:
The job will be queued and executed rather rapidly, transferring output files to the local directory when done:
This is a very simple example.
Clearly, if we were to run this example again the files that we just created would be overwritten (clobbered) by the files created by the new run.
This and many other aspects of job control can be overcome by specific HTCondor command syntax.
For example, unique numbers could be created for the output files with syntax such as:
See examples in the manual under the “Submitting a Job” section for details.
For version 8.6 the link is:
The current version of the HTCondor software running is obtained with the command condor_version:
(This is the installed version on BCC as of December 28, 2021.
The most recent version of the manual is always available at http://research.cs.wisc.edu/htcondor/manual/
The complete HTCondor manual for version 8.6 is 1128 pages long! Therefore, the virtue of patience needs to be called upon to tackle and master using a cluster running HTCondor!