Using globus-url-copy with LRZ resources

The tool globus-url-copy is a command line client provided by the Globus Toolkit in order to move data using the GridFTP transfer protocol.

In this subsection we describe how you can use globus-url-copy to

copy data between your local workstation and SuperMUC
copy data from a remote workstation to SuperMUC (also called third party transfer)

After that we give some concrete examples that show how to use the globus-url-copy command.

Before starting

There are some prerequisites that have to be satisfied before you can move files:

You must have a Grid certificate: the LRZ Grid portal shows the possible ways to obtain one.
You must create a proxy certificate: this is a short-lived credential generated from your Grid certificate (see previous bullet) by means of the command grid-proxy-init and the password of your private key. The command and the related output are:
```
	localhost:~ johndoe$ grid-proxy-init 
	
```
If you are confident enough with this tool, maybe you want to store your credentials on a remote server (MyProxy) and retrieve your proxy certificate from there. This is explained in a dedicated section.
You need to have a Globus installation available on your workstation. The easiest way to satisfy this requirement is to install precompiled packages for your Linux distribution provided by the IGE Project. Instructions on how to properly set up repositories can be found here. An alternative could be to install Globus from the source code. The Globus Toolkit website explains how to do that.

Copying data between your local workstation and SuperMUC

First of all, you have to remember that this operation can be successfully carried out only if you have network access to SuperMUC, that is the IP address of your workstation has been registered into the firewall of SuperMUC (please refer to this page for more details).

The basic syntax of the command is:

	globus-url-copy <sourceURL> <destinationURL>

where <sourceURL> and <destinationURL> can be one of the following:

file://<absolute path to your file>, if you are referring to a file local on the machine where you call this command;
gsiftp://<remote machine><absolute path to your file>, if you are referring to a file in a remote workstation.

After the command globus-url-copy you can specify some options, for example -vb to enable the verbose mode and monitor the transfer.

Assuming that you want to copy a file named foo located into the home folder of your workstation to the home folder you have on SuperMUC, what you have to do is:

	localhost:~ johndoe$ globus-url-copy -vb file:///home/johndoe/foo gsiftp://supzero.lrz.de/~/

The following picture shows the output of the command:

You can see that:

it is not necessary to specify the name of the destination file (even if you can do that, i.e., globus-url-copy -vb file:///home/johndoe/foo gsiftp://supzero.lrz.de/~/bar);
it is possible to specify the home folder in the remote location using the '~' shortcut (very convenient since the absolute path of the home folder on the remote system is not very easy to remember);
since the verbose option is specified, some statistics and the progress of the transfer are shown.

The transfer can also occur in the opposite direction, from SuperMUC to your local workstation. So, if you want to copy the file back to your local workstation, calling this second version bar, you have to type:

	localhost:~ johndoe$ globus-url-copy -vb gsiftp://supzero.lrz.de/~/foo file:///home/johndoe/bar

Finally, you can also transfer a folder. You have to specify two additional options: -r, for a recursive transfer and -cd, to create the destination directory, if necessary.

So, if you want to copy the foo_dir folder from your local workstation to SuperMUC, the complete command to issue is:

	localhost:~ johndoe$ globus-url-copy -vb -cd -r file:///home/johndoe/foo_dir gsiftp://supzero.lrz.de/~/foo_dir/

Please note that in the <sourceURL> you specified the folder name with the trailing '/' character and you also added the same folder name to the <destinationURL>, created for you on the fly by globus-url-copy.

Copy data from a remote workstation to SuperMUC

The globus-url-copy allows you to move data between a remote workstation (i.e., a machine different from your local one however running a GridFTP server) to SuperMUC, provided that:

your credentials are valid on the remote workstation and you have a valid account there;
the remote workstation also has network access to SuperMUC (i.e., its IP address is allowed in the firewall of SuperMUC).

Supposing that all these prerequisites are met and you want to move the a file named foo from your home folder on SuperMUC to your home folder on lxgt2.lrz.de, changing the name to bar, you will have to type:

	localhost:~ johndoe$ globus-url-copy -vb gsiftp://supzero.lrz.de/~/foo gsiftp://lxgt2.lrz.de/~/bar

Please note that if the GridFTP server on the remote workstation runs on a port different from the standard one (2811), then you have to specify the port on the URL, which would become gsiftp://<remote workstation>:<port>/<absolute path to your file>. For example, if the GridFTP port on lxgt2.lrz.de were 2812, then the destination URL of the previous use case would be gsiftp://lxgt2.lrz.de:2812/~/bar.