Usage@Archival storage

Archival storage administration

Action	Description
Cleanup	Roll back/delete failed objects at all logical storages and changes their state to ROLLED_BACK/DELETED in DB. In order to succeed, all storages must be reachable. In the case of success object with states [ARCHIVAL_FAILURE, ROLLBACK_FAILURE] are set to ROLLED_BACK. Objects in state DELETION_FAILURE are set to DELETED. Method deletes failed files at logical storages but does not touch Archival Storage workspace or Producers' Transfer Area.
Cleanup - all	Does the same as Cleanup method but also sets objects with states [PROCESSING, PRE_PROCESSING] to ROLLED_BACK. Also deletes the content of the Archival Storage workspace (the folder used for AIP import and DIP export, configured in arcstorage.tmpFolder property of the Archival Storage service). Does not touch the Producers' Transfer Area.
Set minimal storage count	Set minimal storage count. This value is checked upon archival storage start and during removal of a storage. User is responded with error if the deletion of storage would cause breaking the minimal count limit.
Set read only state	If true, the storage can perform only read requests. Archival storage is set to read only state automatically during initiation of attaching of new storage.
Set reachability check interval	The interval (in minutes) used by periodic reachability check of all attached logical storages. The result of the reachability check (i.e. statement about logical storage reachability) is displayed in the table of logical storages in the Logical storage administration section.

Logical storage administration

This page allows administration of logical storages.

List of storages

Table description:

Attribute	Description
Name	Name of the storage used in the list of storages.
Host	Address of the host of the running instance.
Port	Port of the running instance.
Priority	Value of type integer that determines the priority of the storage among other storage. The higher the priority, the more preferred the storage is.
Storage type	Type of the storage, currently supported types: FS, ZFS, CEPH.
Configuration file	JSON configuration file of the storage.
Mode	Default is R/W. Storage is set to Write only mode during its initial synchronization, It may also be set to Write only mode manually by admin.
Reachable	Boolean attribute, tested and updated by the system periodically and also during every write operation.

Actions description:

Action	Description
Attach new storage	User clicks Attach new storage, fills the form and clicks Submit. After the attachment, the storage is being synchronized. The synchronization ends when the storage contains all objects stored on other storages. See wiki of the Archival Storage for example storage configurations.
Edit storage	User clicks on one the storages in the list, edits the desired attributes and clicks Save and close.
Continue with sync	If the storage sync fails admin is expected to investigate the reason and, probably, manually solve it. Then the Continue with sync button should be used: User clicks on one the storages in the list. Then, if the red dot and text states that the synchronization has failed, user navigates to the Synchronization information section and clicks Continue with sync button. The system tries to continue with the synchronization from the timestamp of the last sync failure.
Delete storage	User clicks on one the storages in the list and clicks Delete.
Check now	Checks the reachability of all attached storages.

Storage detail

Colored dot on the top shows the state of the storage. Green dot means storage is ready and reachable. Orange dot means that storage is being synchronized. Red dot means that storage synchronization has failed (is stucked) or the storage is unreachable.

Storage

Storage attributes. Only some of them can be updated.

Synchronization information

Contains information about the storage synchronization. The synchronization is a one-time process executed after the storage attachment. If the synchronization fails, admin may try to solve the cause and then continue with the synchronization by clicking on the Continue with sync button.

Attribute	Description
Synchronization started	Timestamp of the synchronization start.
Information updated	Timestamp of the last modification of the synchronization information. The information is updated after every copied object, during synchronization error etc.
Phase	INIT: System waits for the 'quiet' time (without new writes) and prepares new storage (creates ARCLib dataspace etc.). COPYING_ARCHIVED_OBJECTS: Copying those objects which were already archived on other storages when the synchronization started. PROPAGATING_OPERATIONS: Propagating new remove/renew/delete operations which has occurred during the synchronization. POST_SYNC_CHECK: System waits for the 'quiet' time (without new writes), finishes the synchronization and checks state of all objects synced to the storage. DONE: Synchronization is completed.
Remains	How many objects/operations remains in the current sync phase.
Done	How many objects were copied / operations were propagated in the current sync phase. If the sync failed at some point of time and then it was continued by admin, this number does not include objects/operations processed before the sync failure.
Exception class	The name of the exception class because of which the synchronization has failed. Shown only if the synchronization has failed.
Exception message	The message of the exception because of which the synchronization has failed. The message may be the same as exception class name. Shown only if the synchronization has failed.

Storage state

This section contains the information which is typically obtained by calling commands specific to the technology of the logical storages:

FS

Attribute	Description
available	Free space of the FS (physical)
used	Occupied space of the FS (physical)

ZFS

Contains data of zfs list and zpool list commands.

Attribute	Description
Dataset > name	Name of the ZFS dataset of the logical storage.
Dataset > used	Occupied space of the ZFS dataset (logical value.. the physical space occupied is higher because of RAID settings etc.)
Dataset > available	Free space of the ZFS dataset (logical, e.g. 500MB means that two 250MB objects may be stored)
Pool > name	Name of the ZFS pool of the logical storage.
Pool > health	Health of the pool
Pool > size	Total space of the ZFS pool (physical)
Pool > alloc	Occupied space of the ZFS pool (physical)
Pool > free	Free space of the ZFS pool (physical)
Other pool information	For example dedup, altroot, expandsz, frag, cap keys of zpool list command

Ceph S3

Contains data obtained through S3 API and the data of ceph -s and ceph df commands.

Attribute	Description
accountId	id of the ARCLib account
accountName	name of the ARCLib account
usedBytes	count of bytes used by the account
objectsCount	count of RGW objects stored by the account
buckets	list of buckets used by the account (possibly just one bucket)
ceph -s	output of the ceph -s command
ceph df	output of the ceph df command