B2STAGE for Site Administrator

About

Technical documentation about setting up the B2STAGE data staging service.

Modified: 13 March 2017

Synopsis

As discussed in the B2STAGE end user documentation page,  this service is based on the Data Storage Interface (DSI). The EUDAT DSI component provides an interface between GridFTP and iRODS servers. In this page we provide the instructions for the installation of the DSI component and the related GridFTP server at EUDAT sites.

Implementation details

The GridFTP DSI module is implemented using the iRODS 3.X C API, starting from a DSI stub which can be generated directly through the Globus Toolkit. A GridFTP server is the front-end for data transfer request, directory creation, file browsing and all standard GridFTP functions. When a request is received, the GridFTP server forwards it to the DSI module which implements the request interacting with the iRODS instance via its API functions. In order to properly handle the connection with the iRODS instance, the DSI utilizes the variables written in the .irodsEnv file (see line 76 of the README file below).

GridFTP and iRODS servers integration also implies some changes at security level. When a connection is received by the standard GridFTP server, the operating system (UNIX System V) forks the process and replaces the process owner from root to a non privileged user for security reasons. From iRODS point of view this introduces a big restriction because - for instance - it forces any iRODS user who wants to make a transfer to have a Unix account on the machine where the GridFTP server is running although that account will never be used. To overcome this restriction the GridFTP server must be launched with the option "-auth-level 4" which delegates the user authentication directly to the underlying iRODS server. The authentication process is performed at two different levels:

1. server-to-server:
   to ensure the GridFTP and iRODS servers recognize each other so as to permit the former to delegate the authentication to the latter.
2. user-to-server:
   to perform the real authentication of the user and to authorize access to the storage resources.

At configuration level this is done assigning the iRODS username to the DN of the GridFTP server certificate, see point 2 and 3 of "Configure and run" in the README file below. 

Installation and configuration notes

There are two aspects involved in enabling B2STAGE at your site: deploying DSI and adding support for EPIC and B2SAFE. This is covered in the README file of the module on GitHub.

Support for B2HANDLE and B2SAFE

In order to work correctly the DSS needs to obtain the PIDs of the staged files from the remote server. Please follow the instructions about B2HANDLE.

The architecture of EUDAT data infrastructure is based on iRODS; it interfaces data and PIDs using the so called "rules" in iRODS. The rules required by DSS are already packaged in the B2SAFE module, and thus all you need is to follow the instructions on how to configure B2SAFE.

Document Data

Version: 1.4

Authors:

Giovanni Morelli, g.morelli@cineca.it

Giacomo Mariani, g.mariani@cineca.it

Editors:

Kostas Kavoussanakis, kavousan@epcc.ed.ac.uk

Carl Johan Håkansson, cjhak@kth.se