gtransfer

The gtransfer tool is a command line utility that extends the capabilities of globus-url-copy with some features such as autocompletion and content listing, even for folders at a remote site.

Prerequisites

• The tool is available on SuperMUC, so you have to log into the resource as explained here. Instructions on how to install gtransfer on your machine are not provided by this guide.
• The utility is working only with a bash shell. Instructions on how to change this setting can be found here.
• Proxy credentials are needed to access other sites. If you logged into SuperMUC via GSISSH or GSISSH-Term, no further actions are needed to accomplish this and you can go to the next point. Otherwise, you have to create a delegated proxy using the grid-proxy-init command or download it from the MyProxy server. How to upload a proxy to the MyProxy server is described here.
• gtransfer is part of the Globus installation of SuperMUC, so the proper module should be loaded. The command to issue on the command line of SuperMUC is module load globus.

Syntax and examples

The basic syntax is:

gtransfer --source <sourceURL> --destination <destinationURL>

where <sourceURL> and <destinationURL> can be one of the following:

• file://<absolute path to your file>, if you are referring to a file in the SuperMUC filesystem;
• gsiftp://<remote machine>:<port><absolute path to your destination>, if you are referring to a file in a remote workstation.

Assuming that you want to move the foo file from your home folder on SuperMUC to your home folder on lxgt2.lrz.de, simply typing gtransfer -- and hitting the TAB key, you receive a suggestion on the possible parameters you can use:

The --source and --destination options are there.

So, we start to specify the source keyword and a space, then we look for another hint using TAB:

gtransfer shows you the two possible protocols and also suggests a couple of GridFTP servers.

Since you know the location of the source file, you can specify it (as $HOME/foo) and then take care of the destination. The next entry on the command line is --destination gsiftp://lxgt2.lrz.de/~/: you can type it making use of the auto-completion with the TAB key at any point. At the end you should have:

Please note the /~/ at the end of the destination address. In other words, after the GridFTP server address you have to partially specify the remote path, starting from the root of the filesystem. The shortcut for the home folder is accepted.

Finally, if you hit TAB two times and your credentials are valid for the destination server, you can see the content of the remote folder:

The operation can take some seconds, since gtransfer is contacting the remote resource for you.

Finally, you may want to specify the name of the destination file, i. e. bar, even if not strictly necessary (the file would preserve the same name). Press return and start the transfer:

A log file in your home folder will be created for each transfer.

Advanced topics

gtransfer also allows you to move entire folders: it is necessary to specify the -r and -cd options of globus-url-copy at the end of the command line. For example, the complete command, to copy source_folder from your SuperMUC home directory to destination_folder on lxgt2.lrz.de would be:

gtransfer --source file://$HOME/source_folder/ --destination gsiftp://lxgt2.lrz.de:2811/~/destination/ -- -r -cd

Please note that the folders are specified with a trailing slash and the additional globus-url-copy parameters are prefixed by the -- flag.

Finally, the remote completion option works with every GridFTP server you can access, but you may also want to add your favorite servers to the list of suggested resources gtransfer can propose to you. To achieve this goal, the steps are the following:

• create the folder $HOME/.gtransfer;
• paste the following on your command line: cp -r /lrz/sys/grid/globusToolkit/etc/gtransfer/{dparamps,dpaths} $HOME/.gtransfer;
• refer to the dpath man pages (man dpath at the prompt) to add your preferred GridFTP sites.