TotalView Debugger

Description

The TotalView Debugger is a graphical tool for debugging sequential and parallel (MPI, OpenMP, POSIX threads etc.) programs.

Note: this description has been tested with TotalView version 8.15.4 on Leo3e.

Using TotalView

The TotalView debugger is available on the zid cluster systems and Mach.

Prerequisites

You need an X-Window-System capable workstation. For details, see the HPC Getting Started Guide. If the command xterm & opens a new terminal window on your workstation, your X installation is functional.

Compiling Programs For TotalView

In this respect TotalView is not different from other debuggers like GDB.

To enable display of program source and variable names in your debug session, you need to recompile and link all routines with the -g flag, which adds symbolic information to your executable.

Due to the optimizer's ability to rearrange program code and eliminate variables, debugging optimized code will in general yield uninterpretable results, so it is advisable to suppress all optimization by using the -o 0 flag.

Starting an Interactive Batch Session

You may skip this section if your program fits comfortably in the Login Node/CPUset and runtimes are very short.

zid Leo Clusters

Start an interactive job on a set of CPUs satisfying your program's runtime requirements. On clusters with Grid Engine, do, e.g.:

qsh -l h_rt=8:00:00 -l h_vmem=2000m -pe openmpi-fillup nproc

with

-l h_rt=8:00:00: Estimated duration of debug session(s) e.g. 8 hours.
-l h_vmem=1000m: Memory required per process, e.g. 1000 Megabytes
-pe openmpi-fillup nproc: OpenMPI parallel environment for nproc processes, use fillup unless you need more control over process allocation.

Parameters printed in italic may be modified according to your needs.

When the xterm window started by qsh opens, proceed.

Mach

in preparation

Preparing the Execution Environment

In the new interactive batch session, do:

module load compiler/version	optional, skip if using the default GNU compiler
module load openmpi/version	optional, skip if you do not use OpenMPI
module load totalview/version	enable TotalView
module load further modules	as needed by your program

Starting TotalView

In the session prepared above, enter totalview (no ampersand - input and output will go to this terminal.

In the dialog box which opens, choose the session type (typically new program (sequential or multithreaded/OpenMP program) or new parallel program (MPI only)). If you selected new parallel program, select your parallel system (typically Open MPI on the zid clusters or MPT on Mach).

For MPI, enter the number of tasks (should be equal or less than the nproc parameter for you interactive batch session.

Hit Next. TotalView correctly discovers your execution environment.

In the Program Details tab, enter the file name of your executable program and arguments expected by your program (if any) in the respective fields.

Hit the Start Session button (which becomes green when all required data have been entered) to start the debug session. Your main program will be displayed in the debug window.

Alternative Startup Method for OpenMPI

In the Web, you will find alternative methods for starting TotalView using OpenMPI's builtin debugger support, using a command line similar to

DON'T
mpirun --debug -np nproc program [ args ...]

Currently, this method is not supported on zid machines and may lead to orphaned processes.