UNICORE6 in PRACE

Overview

Introduction

One method of creating, submitting and monitoring your PRACEproduction jobs is via the UNICORE Rich Client (URC), without the needfor the user to know the details of the target platform’s batch system.Indeed, UNICORE provides a seamless, uniform and secure access to theheterogeneous PRACE infrastructure. This document briefly describes theinstallation, configuration, and usage of the URC. Links for extendeddocumentation are available in the subsection Further Documentation below

This document describes the UNICORE installation on a German 32-bitWindows 7 system, thus the screen shots may differ from what appears onother platforms and the window language will be accordant to thedefault settings of your system.

Prerequisites

The URC is available on every super computer (target system) withinPRACE. Once the URC client is installed on your computer, jobs can besubmitted to each of your Execution Sites. The URC is written in Javaand is therefore platform independent. The only installationprerequisite is an SUN Java Runtime environment version 1.6 or higheron your workstation. Please be aware that a 32-bit Java version must beinstalled on a 32-bit system, and a 64-bit Java version must beinstalled on a 64-bit system.

URC uses Certificates for both authentication and authorization.With this Certificate, the execution sites can be accessed withouthaving to type the password again and again. This is calledsingle-sign-on.

Your URC submits jobs by communicating to UNICORE servers, whichreside at each of the DEISA Execution Systems. Just as each user has apersonal Certificate, each UNICORE server also has an individualCertificate. If the user trusts the signing Certification Authority(CA) of the server Certificates in the URC, then the client-serverconnection will be established. DEISA trusts all Certificates that aresigned by the EUGridPMA.

For further information on the use of Certificates within PRACE ingeneral, please see the PRACE Certificates FAQ at Certificates FAQ.

Further Documentation

This manual can give only an introduction into the UNICORE6 Client,namely the UNICORE Rich Client (URC). The URC provides a lot ofadditional functionality for the user which is not described here indetail.

Detailed URC documentation is available at http://www.unicore.eu/documentation/manuals/unicore6/

If you prefer a video documentation for all the URC functionalitiesplease visit http://www.unicore.eu/documentation/tutorials/unicore6/

In case of any trouble or any suggestions to enhance the URCfunctionality please refer to the PRACE support team.

This manual can give only an introduction into the UNICORE6 Client,namely the URC. The URC provides a lot of additional functionality forthe user which is not described here in detail.

Detailed URC documentation is available at http://www.unicore.eu/documentation/manuals/unicore6/

If you prefer a video documentation for all the URC functionalitiesplease visit http://www.unicore.eu/documentation/tutorials/unicore6/

In case of any troubles or if you want to make suggestions toenhance the URC functionality please refer to the PRACE support team.

UNICORE Client: Installation and Configuration

Installation

This Section describes all steps that are necessary to install theURC.

  • First get the latest version of the URC from http://www.unicore.eu/download/unicore6/:

-> Clients -> Rich Client -> Download

There different clients are available for Windows, Mac OS andLinux.

  • Download the relevant package and store it locally on yourcomputer.
  • Unzip the package and execute the UNICORE_Rich_Clientfile
  • Configuration

    Initialisation

    At start-up, the client will create a new keystore for you. TheUNICORE keystore will later contain your personal Certificate and allthe Certificates required to access the remote sites. First choose apath and generate an empty keystore file in the file browser (e.g.user.jks – see Figure 1 and Figure 2) and confirm thecreation of the new keystore (Figure 3). Please ensure that yourkeystore filename has the ’.jks’ suffix.

    JPEG - 131.3 kb
    Figure 1
    JPEG - 143.7 kb
    Figure 2
    JPEG - 52.5 kb
    Figure 3

    Next you will be asked to create a password for the keystore(Figure 4). This password has to be typed every time the URC will bestarted.

    JPEG - 58.2 kb
    Figure 4

    After confirming the password the URC will be configured for thefirst session (loading Workbench). This step might take a fewseconds. When the configuration is ready you will see a”Welcome”-window that contains five icons (Figure 5):

    • Overview: Get an overview of the features
    • First steps: Take your first steps
    • Tutorials: Go through tutorials
    • What’s New: Find out what is new
    • Workbench: Go to the workbench

    Choose any chapter you are interested in. If you want to skip theintroduction then choose the Workbench icon or minimizethe welcome window. This window will be available in the URC to letyou check the different documentations. The first URC view will besimilar to what is shown in Figure 6.

    JPEG - 191.6 kb
    Figure 5
    JPEG - 145.1 kb
    Figure 6

    UNICORE Keystore and the Truststore

    At this point, both the

    JPEG - 1.8 kb

    and the

    JPEG - 1.8 kb

    have to be populated.

    The keystore contains the private part of your Certificate (whichis known as the private key) and the truststore contains the publicpart of the Certificates of each PRACE site (which is known as thepublic key). The private key has been created by you in your webbrowser already when you made the PRACE certificate request from yournational Grid-CA. If you do not have your Certificate in.p12 format, open this web browser again and export theprivate part of your Certificate from your browser to a local file in.p12 format. You will then be asked to provide thepassword you created for your private key. This is also needed in thefollowing step.

    Import the users’ private key into the URC keystore:

    • Select the “Keystore” panel.
    • Press
      JPEG - 1020 bytes

      to search for the private key withthe upcoming file browser on your local workstation. The filter has tobe set from *.jks to *.p12 (see Figure 7)

    • Click on “Open”
    • Provide the private keys’ password (Figure 8)
    • Now the private key has been imported to the URC keystore and youshould see something similar as in Figure 9.
    JPEG - 268.3 kb
    Figure 7
    JPEG - 61.5 kb
    Figure 8
    JPEG - 156.9 kb
    Figure 9

    Next, populate the URC Truststore. For this step you need all therelevant EUGridPMA certificates as provided by SURFsara via http://winnetou.surfsara.nl/prace/certs/

    => Package with individual certs from CAcerts in gz (zip)format

    Download the certificate package, store it on your local computerand extract it to a folder of your choice. Then the UNICORE truststorehas to be updated with these certificates:

    • Select the “Truststore” panel
    • Press
      JPEG - 1020 bytes

      and select all EUGridPMAcertificates from your local computer (you might mark all entries andimport them at once – Figure 10

    • Click on “Open”
    • Now the EUGridPMA certificates have been imported into the UNICOREtruststore and you should see something similar as shown in Figure11
    JPEG - 268.2 kb
    Figure 10
    JPEG - 321 kb
    Figure 11

    Next, all the PRACE resources have to be integrated into the URC.Therefore add a new Registry as shown in Figure 12 and Figure 13. Therespective Registry URL is https://prace-unic.fz-juelich.de:9111/PRACE/services/Registry?res=default_registry.

    In case there is a temporary problem with the central registry, a backupinstance is maintened by CINECA and available at https://grid.cineca.it:9111/PRACE/services/Registry?res=default_registry.

    JPEG - 243.4 kb
    Figure 12
    JPEG - 88.5 kb
    Figure 13

    Double click on

    JPEG - 2.7 kb

    torefresh the URC view. Afterwards you should see all your ExecutionSites (Figure 14).

    UNICORE Jobs

    Now everything is configured properly and you are ready for yourfirst PRACE UNICORE job. The example below shows how to create andsubmit a simple job script.

    Select the site where you want to submit your job to (e.g.FZJ_JUGENE) in the URC GridBrowser view with a right mouse click (upperleft window, see Figure 15). If no site is selected, then any of theuser’s Execution Sites will be taken that achieve your jobproperties.

    Select Script v.2.0 from the Job Creation window andclick on Finish (Figure 16).

    JPEG - 246.1 kb
    Figure 15
    JPEG - 98.1 kb
    Figure 16

    A new window appears where any Linux commands or program calls,etc., may be included. An example is shown in Figure 17. NB URCautomatically prepends all scripts with module loadprace.

    JPEG - 299.2 kb
    Figure 17

    After that, submit the job to the selected target system (FZJ_JUGENEin the example) by clicking on (see Figure 18). After a few secondsyou will see your submitted job (Figure 19). The example shows apending job that is in the batch queue of the Execution System.

    JPEG - 300.4 kb
    Figure 18

    To monitor your submitted job, simply double click on the selectedExecution Site (FZJ_JUGENE) in the upper left window (GridBrowserview). You will see all the available file systems on the ExecutionSystem. If needed you may browse through the file systems (see Figure19). The status will be refreshed automatically by the URC, where theupdate duration depends on the queue status of each Execution System.When the icon changes to green the job has been executed successfully,red means an error (Figure 20).

    JPEG - 87.6 kb
    Figure 19
    JPEG - 81.9 kb
    Figure 20

    Double click the submitted script to get the job output (e.g. stdoutand stderr in Figure 21).

    JPEG - 64.3 kb
    Figure 22

    If you want to download the stdout or stderr to your local computer,then double click on stdout or stderr, respectively (Figure 22). Figure23 shows the stdout.

    JPEG - 64.3 kb
    Figure 22
    JPEG - 284.4 kb
    Figure 23

    File Transfer with the UNICORE Client

    The URC provides the user with integrated file transfer mechanism.To transfer files, first select your script in the URC and then theFiles panel at the bottom. The upper part of the new window isfor file import into a temporary directory on the remote supercomputerand the panel below is to export from the temporary directory into apermanent one on the remote HPC system (Figure 24).

    Please note that, for each job, a private, temporary directory iscreated on the target system which is called USPACE. If no otherdirectory is referred to in the job script, then the job executeswithin this USPACE directory. It is important to know that the USPACEis a temporary directory and all the content will be deleted after thebatch job has finished.

    Thus, if you are interested to keep all data after the job has beenfinished, then move to a permanent directory (for instance byincluding the following command at the beginning of your script: cd $PRACE_HOME). Otherwise onlystdout and stderr will be available.

    JPEG - 329.6 kb
    Figure 24

    Add the file that has to be submitted from your local computer tothe Execution System (Figure 25).

    JPEG - 349.7 kb
    Figure 25

    With a click in the Source File(s) panel you can browse your files.You may transfer either single files or whole directories (Figure26).

    JPEG - 343.3 kb
    Figure 26

    Next, the files have to be transferred from the temporary URCUSPACE to a permanent directory as shown in Figure 27.

    JPEG - 355.5 kb
    Figure 27

    Select UNICORE_Storage as Destination Type (Figure28).

    JPEG - 343.4 kb
    Figure 28

    The storage may be selected with a click in the File(s) atDestination / File ID panel (Figure 29). In the upcoming filebrowser the remote target storage is set.

    JPEG - 333.5 kb
    Figure 29

    MPI Jobs with the UNICORE Client

    When you intend to submit a MPI job, for instance, or any kind ofparallel job in fact, it will be necessary to define job resources likeTotal Tasks, Wall Clock Time, etc.

    To edit the resources on the respective target system select the”Resources” panel from the bottom of your script jobpreparation window. Please note that the content of the resourceswindow is dependent on the selected Execution Systems’architecture.

    To submit a basic job, users need only specify two resourceparameters which describe the total number of cores and the length ofthe job, as all other resource parameters are provided with theirdefault values. This permits users to submit jobs without knowledge ofthe batch system of the Execution Platform.

    Therefore, only the total number of cores (represented asTotal number of CPUs in the URC resource panel) and WallClock Time (represented as Wall time in the URC resourcepanel are available on all PRACE Execution Systems. All otherresources depend on the respective architecture (e.g.Topology will be available on any BlueGene system).

    The respective valid keywords, given for each architecture, aredescribed in PRACE Batch Systems

    Figure 30 shows an example of available BlueGene resources.

    JPEG - 300.1 kb
    Figure 30

    By clicking into any resource Value panel the values can bechanged. For instance in Figure 31 the Total Tasks are changed fromthe default value 32 to 64.

    JPEG - 214.6 kb
    Figure 31

    Next, put the MPI command in your URC script editor. NB you shouldnot provide any resource definitions in the MPI call itself; however,in the current URC version, you must include the MPI launcher, suchas poe, aprun, mpirun, etc. (poe is employed in the example).The information can be taken from the batch user guide (see URLabove). In a future URC release, the MPI launcher will be providedautomatically.

    A correct call for instance could be:

    cd $PRACE_HOMEpoe hello.out