Research Data Services, Expertise & Technology Solutions

You are here

The B2SHARE HTTP REST API

Primary tabs

About

User documentation about the EUDAT B2SHARE HTTP REST API.

Modified: 29 January 2018

Synopsis

B2SHARE is the EUDAT service for storing and publishing data sets. In addition to its web-based GUI, B2SHARE offers an HTTP REST API. External applications or workflows can use this API, for example, to integrate B2SHARE with other websites like research community portals, or to allow uploads and downloads of large data sets that are not easily handled via a web browser. The API functionality is also used for metadata harvesting by external metadata catalogue services, including B2FIND.

What is B2SHARE?

B2SHARE is the EUDAT service for storing and publishing scientific data. B2SHARE is designed to be easy to use and currently supports access restrictions, registration of a PID for any uploaded data object, add on of checksums and transition of all metadata information to the EUDAT metadata search. Importantly, B2SHARE enforces the inclusion of metadata accompanying the deposited data, so as to increase the value and facilitate sharing of your assets. B2SHARE fosters Open Access to data and includes a tool to help the user choose the correct licence for their data. B2SHARE should be used to publish data that are not meant to change. For transient data, please consider B2DROP.

B2SHARE provides a graphical, web-based interface, whose functionality is discussed in the relevant User Documentation page. B2SHARE also offers an HTTP REST API, for programmatic access to the service. This page discusses how to use this API.

Basic concepts

The B2SHARE service uses several concepts which are briefly explained below.

A scientific community has the roles of creating and maintaining metadata schemas and curating the datasets which are part of a scientific domain or research project. B2SHARE users can be part of one or more communities. Some selected members of a community are also given the role of community administrators, which grants them the special rights needed for the schema definitions and record curation tasks.

Any user can upload scientific datasets into B2SHARE and thus create data records. A record is comprised of data files and associated metadata. The record’s metadata consists of a set of common fixed metadata fields and a set of custom metadata blocks, each block containing related metadata fields. A record is always connected to one scientific community which has the role of curating and maintaining it.

A data record can exist in several states. Immediately after creation a record enters the 'draft' state. In this state the record is only accessible by its owner and can be freely modified: its metadata can be changed and files can be uploaded into or removed from it. A draft can be published at any time, and through this action it changes its state from 'draft' to 'published', is assigned Persistent Identifiers, and becomes publicly accessible. Please note that the list of files in a published record cannot be changed.

A record contains a set of common metadata fields and a set of custom metadata blocks. This metadata is not free form, however, but is governed by static schemas; the common metadata schema is set by B2SHARE and defines a superset of Dublin Core elements, while the schema for the custom metadata block is specific to each community and can be customized by the community administrators. The schemas are formally defined in the JSON Schema format. A special HTTP API call is available for retrieving the JSON Schema of a record in a specific community. In order to be accepted, the records submitted to a community must conform to the schema required by the community.

Using the API

The REST API is used by making API requests using the GET or POST methods of the standard HTTP protocol. Typically an application like curl is used to make requests, but it is also possible to do this from within your custom-built applications. To intergrate the API in your application using Python, please refer to the B2SHARE API training module to learn how to use the API.

Authentication

Although listing and accessing public data is not access-controlled, only authenticated users can use the API to its full extent. Authentication during requests is done by passing an access token along with the request. The access token is an encrypted string which can be created in the B2SHARE user profile when logged in to the web user interface. B2SHARE’s access tokens follow the OAuth 2.0 standard.

Creating an access token

After logging in to the B2SHARE service click your user name and select 'Profile'. Under 'API tokens', enter a token name (Figure 1).

B2SHARE Access Token Step 1

Figure 1. B2SHARE Access Token Step 1: Applications.

Click “New token” on the right of the text box. Your token is now created and immediately displayed (Figure 2). Please note that this is the only time the access token is visible, so copy it to a safe place.

B2SHARE Access Token Step 2

Figure 2. B2SHARE Access Token Step 2: Create new token.

The access token is now ready for use. Access tokens can be deleted by clicking on the remove button next to each access token name.

Making requests

The HTTP requests are made to a URL with parameters as described below. Each URL consists of a protocol part (always 'https://'), a hostname and a path. One of the following hostnames can be used to identify the B2SHARE instance:

Please make sure that you are not using production instances for creating test records or testing the API in general.

Request structure

Each allowed request is discussed below as follows:

  • Description - A description of the function of the request.

  • URL path - grammar for the allowed paths used together with one of the base URLs above.

  • HTTP method - whether the HTTP protocol's GET or POST method is used.

  • Example - an example of usage using the program curl from the command line.

Variables in the descriptions:

  • COMMUNITY_ID - identifier of a user community in B2SHARE

  • RECORD_ID - identifier for a specific record, which can be in draft or published state

  • FILE_BUCKET_ID - identifier for a set of files. Each record has its own file set, usually found in the links -> files section

Note: A record is unchangeable and has a PID assigned to it. A user can create a record by first creating a draft record, which is modifiable. Initial files and metadata can be placed into a draft record, but not into a record. The metadata of an existing records can be modified by altering the corresponding draft record.

A publication workflow

The REST API does not impose a specific workflow for creating a record. The following example workflow only defines the most basic steps:

  1. Identify a target community for your data by using the REST API List all communities function
  2. Using the community's identifier, retrieve the JSON Schema of the record's metadata. The submitted metadata will have to conform to this schema. Use the Get community schema function
  3. Create a draft record: use the Create draft record function
  4. Upload the files into the draft record. You will have to use one HTTP request per file. Use the Upload file function
  5. Set the complete metadata and publish the record. Use the Submit draft for publication function

Migrating to the B2SHARE v2 HTTP API

The following changes are needed for a B2SHARE version 1 client, using the old HTTP API, in order to make it work with B2SHARE version 2 for creating and publishing a record:

  1. Identify the unique ID of your target community or communities: see List all communities function
  2. Update the URL for creating a new record, from/api/deposition/ to /api/records/; see Create draft record function
  3. Update the JSON structure of the newly created records to match the required JSON schema structure, see the Get community schema function
  4. Update the file upload calls, making sure that the file bucket URL is used instead of the old record URL, see the Upload file function
  5. Update the old 'commit' action as described in the Submit draft for publication function

The REST API

The REST API of the B2SHARE service support several endpoints which all return different types of information. Depending on the needs, the following API requests are available.

List all communities

List all the communities, without any filtering.

  • HTTP method: GET

  • URL path: /api/communities

  • Required parameters: access_token

  • Returns: the list of communities (in JSON format) or an error message.

Example:

$ curl https://$B2SHARE_HOST/api/communities/?access_token=$ACCESS_TOKEN

Get community schema

Retrieves the JSON schema of records approved by a specific community.

  • HTTP method: GET

  • URL path: /api/communities/$COMMUNITY_ID/schemas/last

  • Required parameters: access_token

  • Returns: the JSON schema, embedded in a JSON object, or an error message.

Example:

$ curl https://$B2SHARE_HOST/api/communities/$COMMUNITY_ID/schemas/last?access_token=$ACCESS_TOKEN

List all the records

List all the records, without any filtering.

  • HTTP method: GET

  • URL path: /api/records

  • Required parameters: access_token

  • Returns: the list of records (in JSON format) or an error message.

$ curl https://$B2SHARE_HOST/api/records/?access_token=$ACCESS_TOKEN

List records per community

List all records of a specific community.

  • HTTP method: GET

  • URL path: /api/records/?q=community:COMMUNITY_ID

  • Required parameters: access_token

  • Returns: the list of records (in JSON format) or an error message.

Example:

$ curl https://$B2SHARE_HOST/api/records/?q=community:$COMMUNITY_ID?access_token=$ACCESS_TOKEN

Search records

Search all the published records for a query string.

  • HTTP method: GET

  • URL path: /api/records/?q=$QUERY_STRING

  • Required parameters: access_token

  • Returns: the list of matching records (in JSON format) or an error message.

Example:

$ curl https://$B2SHARE_HOST/api/records/?q=$QUERY_STRING?access_token=$ACCESS_TOKEN

Search drafts

Search for all drafts (unpublished records) that are accessible by the requestor. Usually this means own records only.

  • HTTP method: GET

  • URL path: /api/records/?drafts

  • Required parameters: access_token, drafts

  • Returns: the list of matching drafts (in JSON format) or an error message.

Example:

$ curl https://$B2SHARE_HOST/api/records/?drafts&access_token=$ACCESS_TOKEN

Get a specific record

List the metadata of the record specified by RECORD_ID

  • HTTP method: GET

  • URL path: /api/record/RECORD_ID

  • Required parameters: access_token

Example:

$ curl https://$B2SHARE_HOST/api/records/47077e3c4b9f4852a40709e338ad4620?access_token=$ACCESS_TOKEN

Create a draft record

Create a new record, in the draft state.

  • HTTP method: POST

  • URL path: /api/records

  • Required URL parameter: access_token

  • Required data payload: json object with basic information about the object

  • Returns: the new draft record contents and location. Please note that the returned json object contains also the URL of the file bucket used for the record. Also note that the URL of the draft record, needed for setting record metadata, will end in '/draft/'

Example:

$ curl -i -H "Content-Type:application/json" -d '{"titles":[{"title":"TestRest"}], "community":"e9b9792e-79fb-4b07-b6b4-b9c2bd06d095", "open_access":true}' -X POST https://$B2SHARE_HOST/api/records/?access_token=$ACCESS_TOKEN

Upload file into draft record

To upload a new file into a draft record object, first you need to identify the file bucket URL. This URL can be found in the information returned when querying a draft record, in the 'links/files' section of the returned data.

  • HTTP method: POST

  • URL path: /api/files/FILE_BUCKET_ID/FILE_NAME

  • Required input data: the file, sent as direct stream

  • Required parameters: access_token

  • Returns: informations about the newly uploaded file

Example:

$ curl -X PUT -H 'Accept:application/json' -H 'Content-Type:application/octet-stream' -d @TestFileToBeUploaded.txt https://$B2SHARE_HOST/api/files/$FILE_BUCKET_ID/TestFileToBeUploaded.txt?access_token=$ACCESS_TOKEN

Note that the maximum size for a file is 10GB; and the maximum size for a record is 20GB.

List the files uploaded into a record object

List the files uploaded into a record object

  • HTTP method: GET

  • URL path: /api/files/FILE_BUCKET_ID

  • Required parameters: access_token

  • Returns: information about all the files in the record object

Example:

$ curl https://$B2SHARE_HOST/api/files/$FILE_BUCKET_ID?access_token=$ACCESS_TOKEN

Update draft record's metadata

This action updates the draft record with new information.

  • HTTP method: PATCH

  • URL path: /api/records/RECORD_ID/

  • Required input data: the metadata for the record object to be created, in the JSON patch format

  • Notes: The patch format contains one or more JSONPath strings. The root of these paths are the metadata object, as this is the only mutable object. For instance, to update the title field of the record, use this JSONPath: /title

Example:

$ curl -X PATCH -H 'Content-Type:application/json-patch+json' -d '[{"op": "add", "path":"/keywords", "value": ["keyword1", "keyword2"}]' https://$B2SHARE_HOST/api/records/$RECORD_ID/draft?access_token=$ACCESS_TOKEN

Submit a draft record for publication

This action marks the draft record as complete and submits it for publication. Please be advised that publishing the draft will make its files immutable.

A draft record is submitted for publication if a special metadata field, called 'publication_state' is set to 'submitted'. This field can be set using the PATCH call described above.

Depending on the domain specification, other fields could be required in order to successfully publish a record. In case one of the required fields is missing the request fails and an error message is returned with further details. Also, if a community requires that any submitted record be reviewed before publication, a record will only become available if one of the community administrators approves the publication.

  • HTTP method: PATCH

  • URL path: /api/records/RECORD_ID/draft

  • Required input data: the metadata for the draft record object to change its publication state, in the JSON patch format

  • Notes: The patch format contains a single JSONPath string for the field publication_state metadata field.

Example:

$ curl -X PATCH -H 'Content-Type:application/json-patch+json' -d '[{"op": "add", "path":"/publication_state", "value": "submitted"}]' https://$B2SHARE_HOST/api/records/$RECORD_ID/draft?access_token=$ACCESS_TOKEN

Report a record as an abuse record

The B2SHARE service foresees the following four cases of an abusive record:

  1. Abuse or Inappropriate content

  2. Copyrighted material

  3. Not research data

  4. Illegal content

The API allows a user to report inappropriate records on account of any of the above causes. The reporter can include a message to explain more about the problem. It is possible for an anonymous user to send the report and therefore authentication is not required. Posting this request causes an email to sent to the corresponding record administrator and it will be followed up.

  • HTTP method: POST

  • URL path: /api/records/RECORD_ID/abuse

  • Required data payload: JSON object with information about reporter and the reason why the record is reported as abusive

  • Optional parameters: access_token

  • Returns: a message that an email was sent and the record is reported

Example:

$ curl -X POST -H 'Content-Type:application/json' -d '{"noresearch": true, "abusecontent": false, "copyright": false, "illegalcontent": false, "message": "It is s not research data...", "name": "John Smith", "affiliation":"example University", "email":"j.smith@example.com", "address":"example street", "city":"exampleCity", "country":"exampleCountry", "zipcode":"12345", "phone":"7364017452"}' https://$B2SHARE_HOST/api/records/$RECORD_ID/abuse

Send a request to get access to restricted data in a record

For records with restricted access to data, a user (either authenticated or anonymous) can send a request to the record owner and ask for it.

  • HTTP method: POST

  • URL path: /api/records/RECORD_ID/accessrequests

  • Required data payload: JSON object with information about who is sending the request

  • Optional parameters: access_token

  • Returns: a message that an email was sent

Example:

$ curl -X POST -H 'Content-Type:application/json' -d '{"message": "Explain the request...", "name": "John Smith", "affiliation": "Example University", "email": "j.smith@example.com", "address": "Example street", "city":"Example city", "country": "Example country", "zipcode": "12345", "phone": "7364017452"}' https://$B2SHARE_HOST/api/records/$RECORD_ID/accessrequests?access_token=$ACCESS_TOKEN

Delete a draft record

Draft records can be deleted by either the owner of that record or the site administrator.

  • HTTP method: DELETE

  • URL path: /api/records/RECORD_ID/draft

  • Required data payload: none

  • Required parameters: access_token

  • Returns: no message returned, only status code 204

Example:

$ curl -X DELETE -H 'Content-Type:application/json' https://$B2SHARE_HOST/api/records/$RECORD_ID/draft?access_token=$ACCESS_TOKEN

Delete a file from a draft record

Individual files can be deleted from a draft records by either the owner of that record or the site administrator.

  • HTTP method: DELETE

  • URL path: /api/files/FILE_BUCKET_ID/FILE_NAME

  • Required data payload: none

  • Required parameters: access_token

  • Returns: no message returned, only status code 204

Example:

$ curl -X DELETE -H 'Content-Type:application/json' https://$B2SHARE_HOST/api/files/$FILE_BUCKET_ID/FILE_NAME?access_token=$ACCESS_TOKEN

Delete a published record

Published records can only be deleted by the site administrator.

  • HTTP method: DELETE

  • URL path: /api/records/RECORD_ID

  • Required data payload: none

  • Required parameters: access_token

  • Returns: no message returned, only status code 204

Example:

$ curl -X DELETE -H 'Content-Type:application/json' https://$B2SHARE_HOST/api/records/$RECORD_ID?access_token=$ACCESS_TOKEN

Responses

All response bodies are JSON encoded (UTF-8 encoded).

A record is represented as a JSON object: { "field1": value, … }

A collection of records is represented as a JSON array of objects:

[{ "field1": value, ... }, … ]}

Timestamps are in UTC and formatted according to ISO 8601:

YYYY-MM-DDTHH:MM:SS+00:00

Response fields

Every successful API request returns a JSON encoded string with the corresponding metadata fields. The actual fields differ per request and depend largely on the type of data requested.

Support

Hands-on material is available from the EUDAT Training Repository.

Our B2SHARE presentations offer training material for the service.

Support for B2SHARE and its REST API is available via the EUDAT ticketing system through the webform.

If you have comments on this page, please submit them though the EUDAT ticketing system.

Document Data

Version: 1.3

Authors:

Emanuel Dima, emanuel.dima@uni-tuebingen.de

Editors:

Hans van Piggelen, hans.vanpiggelen@surfsara.nl

Kostas Kavoussanakis, k.kavoussanakis@epcc.ed.ac.uk