User documentation
- EUDAT Primer
- Configure B2SAFE
- B2FIND
- B2DROP
- B2SHARE
- B2STAGE
- B2STAGE for Site Administrator
- PIDs in EUDAT
- iRODS Deployment
- The dCache to iRODS connection at SURFSara
- B2ACCESS Management
- B2ACCESS Service Integration
- B2ACCESS Usage
- B2FIND Integration
- B2FIND Usage
- B2HANDLE
- B2HANDLE for communities
- B2HANDLE for end-users
- B2HOST
- B2SAFE
- B2SHARE Advanced search
- B2SHARE Deployment
- B2SHARE FAQ
- B2SHARE HTTP REST API
- B2SHARE Usage
- How to join B2HOST as a Provider
- Joining B2SAFE
- License Selector
- Publish from B2DROP to B2SHARE
- Using B2SAFE
Primary tabs
About
Technical documentation about setting up the B2STAGE data staging service.
Modified: 13 March 2017
Synopsis
As discussed in the B2STAGE end user documentation page, this service is based on the Data Storage Interface (DSI). The EUDAT DSI component provides an interface between GridFTP and iRODS servers. In this page we provide the instructions for the installation of the DSI component and the related GridFTP server at EUDAT sites.
Implementation details
The GridFTP DSI module is implemented using the iRODS 3.X C API, starting from a DSI stub which can be generated directly through the Globus Toolkit. A GridFTP server is the front-end for data transfer request, directory creation, file browsing and all standard GridFTP functions. When a request is received, the GridFTP server forwards it to the DSI module which implements the request interacting with the iRODS instance via its API functions. In order to properly handle the connection with the iRODS instance, the DSI utilizes the variables written in the .irodsEnv file (see line 76 of the README file below).
GridFTP and iRODS servers integration also implies some changes at security level. When a connection is received by the standard GridFTP server, the operating system (UNIX System V) forks the process and replaces the process owner from root to a non privileged user for security reasons. From iRODS point of view this introduces a big restriction because - for instance - it forces any iRODS user who wants to make a transfer to have a Unix account on the machine where the GridFTP server is running although that account will never be used. To overcome this restriction the GridFTP server must be launched with the option "-auth-level 4" which delegates the user authentication directly to the underlying iRODS server. The authentication process is performed at two different levels:
- server-to-server:
to ensure the GridFTP and iRODS servers recognize each other so as to permit the former to delegate the authentication to the latter. - user-to-server:
to perform the real authentication of the user and to authorize access to the storage resources.
At configuration level this is done assigning the iRODS username to the DN of the GridFTP server certificate, see point 2 and 3 of "Configure and run" in the README file below.
Installation and configuration notes
There are two aspects involved in enabling B2STAGE at your site: deploying DSI and adding support for EPIC and B2SAFE. This is covered in the README file of the module on GitHub.
Support for B2HANDLE and B2SAFE
In order to work correctly the DSS needs to obtain the PIDs of the staged files from the remote server. Please follow the instructions about B2HANDLE.
The architecture of EUDAT data infrastructure is based on iRODS; it interfaces data and PIDs using the so called "rules" in iRODS. The rules required by DSS are already packaged in the B2SAFE module, and thus all you need is to follow the instructions on how to configure B2SAFE.
Document Data
Version: 1.4
Authors:
Giovanni Morelli, g.morelli@cineca.it
Giacomo Mariani, g.mariani@cineca.it
Editors:
Kostas Kavoussanakis, kavousan@epcc.ed.ac.uk
Carl Johan Håkansson, cjhak@kth.se