Rucio Storage Endpoint (RSE) Names and Attributes

Introduction and motivation

Establishing clear and consistent naming conventions for Rucio Storage Endpoints (RSEs) will facilitate efficient data management, improved operational clarity, and easier collaboration across multiple sites and Nodes. It should aim to provide: * Clarity and Consistency: with unique (but not overly constrained) and unambiguous ways to name a site, avoiding confusion. * Ease of operational use: clear and simple naming, with ability to use automated tooling, where appropriate. * Scalability: Should allow for ease of scalability and extensibility with minimal changes necessary to existing infrastructure / standards.

Attributes are metadata properties assigned to Rucio Storage Elements (RSEs) that define various characteristics and configurations of the storage elements. While some are internal to rucio, for its functionality, user-defined fields can be specified to facilitate Data Management activities. Establishing a basic set of enhanced attributes will provide: * Interoperability across sites: facilitating the use of heterogenous storage across many distinct regions. * Ease of operational use: improved use of automated tooling and use of built-in rucio tools for management data placement based on higher-level requirements and needs. * Scalability: to allow to increase scale with current needs and to be easily extensible as future requirements are refined.

RSE Naming

RSEs shall adopt the following naming scheme:

{src-name}-{site-name}-{storage-class}[-{storage-label}]

  • src-name: The National SRC name (or SKAO), as defined in National SRC Naming Standards

  • site-name: a 1–10 character (A-Z0-9) label that defines the physical location of the storage element (SE), e.g. RAL, INFN

  • storage-class: one of the set of allowed keywords, which defines the type/usage of storage:

    • allowed values: {T0, T1, T2, NAT, TEST, INGEST}

  • site-label: an optional 1-10 character (A-Z0-9) label to distinguish different RSEs of same storageclass at a given SRC and site

Examples of RSEs that satisfy this scheme could be:

SKAO-STFC-T1
SKAO-MID-INGEST
UKSRC-RAL-T1
UKSRC-RAL-T2-DEV
CANSRC-XYZ-NAT-TAPE

Site Name

SEs also typically allow the setting of a site name within their configuration. For monitoring, grouping by site name may alo be useful. These shall be identified as (using the defintions above):

{src-name}-{site-name}

Additional Attributes

In addition to the default set of attributes that are attached to each RSE on creation, the following set of keys should also be created and the appropriate values added.

  • SRC: the SRC name

  • site : name of the site (matching the RSE name site)

  • storage-class: the storage class

  • country_code : ISO 3166-1 alpha-2 (2-letter code) defining the country (e.g. AU, CA, GB, … )

  • region_name : Top-level region (defined under UN M49) :

regions = [
    "Africa",
    "Asia",
    "Europe",
    "Oceania",
    "Americas"
]

  • se_type: the SE software used (e.g. xrootd, storm, dcache)

  • storage_tier: T0, T1, T2, INGEST

  • storage_type: disk, tape

  • Compliance attributes

    • srcnet_version: Defines if an RSE is compliant with, or included for, a specific srcnet version (e.g. 0.1)

      • (due to some possible issues with filtering on decimals) propose to have also

      • is_v0p1 = True, False

  • Usage attributes

    • is_national: true, false in case of national only RSEs.

RSE Name schema

A very simple schema for the RSE name is given below

{
"$schema": "http://json-schema.org/draft-07/schema#",
"type": "object",
"properties": {
    "rse_name": {
    "type": "string",
    "description": "Composite RSE name consisting of SRC, Site, and StorageClass with optional site storage label",
    "pattern": "^(((AUS|CAN|CN|FR|DE|IN|IT|JP|NL|PT|ZA|KR|ESP|SWE|CH|UK)SRC|SKAO)-[A-Z0-9]{1,10}-(T0|T1|T2|TEST|INGEST|NAT))(-[A-Z0-9]{1,10})?$"
    }
},
"required": ["rse_name"],
"additionalProperties": false
}