Skip to content

Commit

Permalink
Fix all broken links
Browse files Browse the repository at this point in the history
  • Loading branch information
ocaisa committed Sep 26, 2024
1 parent 72bf0c2 commit f27983a
Show file tree
Hide file tree
Showing 6 changed files with 28 additions and 162 deletions.
117 changes: 3 additions & 114 deletions docs/appendix/terminology.md
Original file line number Diff line number Diff line change
@@ -1,93 +1,12 @@
# CernVM-FS Terminology
# Terminology

An overview of terms used in the context of CernVM-FS, in alphabetical order.

---

## Catalog { #catalog }

A **catalog** of a CernVM-FS [repository](#repository) is a table that lists files and directories along with the corresponding
metadata (sizes, timestamps, etc.).

Catalogs can be *nested*: subtrees of the [repository](#repository) may have their own catalog.

For more information on the *catalog* concept, see [the CernVM-FS
documentation](https://cvmfs.readthedocs.io/en/stable/cpt-details.html#file-catalog).

---

## CernVM { #cernvm }

**CernVM** is a virtual machine image based on CentOS combined with a custom, virtualization-friendly Linux kernel,
and which includes the [CernVM-FS](../cvmfs/what-is-cvmfs.md) client software.

It is used for the CERN [Large Hadron Collider (LHC)](https://home.cern/science/accelerators/large-hadron-collider)
experiment, and was created to remove a need for the installation of the experiment software and to minimize the
number of platforms (compiler-OS combinations) on which experiment software needs to be supported and tested.

While originally developed in conjunction, the [CernVM File System](#cvmfs) today is a product
that is completely independent from the CernVM virtual appliance.

For more information on CernVM, see the [website](https://cernvm.cern.ch/appliance/)
and [documentation](https://cernvm.readthedocs.io).
An overview of terms used in the context of EESSI, in alphabetical order.

---

## CernVM-FS { #cvmfs }

*(see [What is CernVM-FS?](../cvmfs/what-is-cvmfs.md))*

---

## Client { #client }

A **client** in the context of CernVM-FS is a computer system on which a CernVM-FS [repository](#repository)
is being accessed, on which it will be presented as a [POSIX](https://en.wikipedia.org/wiki/POSIX)
read-only file system in a subdirectory of `/cvmfs`.

---

## Proxy { #proxy }

A **proxy**, also referred to as *squid proxy*, is a forward caching
[proxy server](https://en.wikipedia.org/wiki/Proxy_server) which acts as an intermediary between a CernVM-FS
[client](#client) and the [Stratum-1 replica servers](#stratum1).

It is used to improve the latency observed when accessing the contents of a [repository](#repository),
and to reduce the load on the [Stratum-1 replica servers](#stratum1).

A commonly used proxy is [Squid](http://www.squid-cache.org).

For more information on proxies, see the
[CernVM-FS documentation](https://cvmfs.readthedocs.io/en/stable/cpt-squid.html).

---

## Publishing { #publishing }

**Publishing** is the process of adding more files to a CernVM-FS [repository](#repository),
which is done via a *transaction* mechanism, and is on possible on the [Stratum-0 server](#stratum0),
via a [publisher](https://cvmfs.readthedocs.io/en/stable/cpt-repository-gateway.html#publisher-configuration),
or via a [repository gateway](https://cvmfs.readthedocs.io/en/stable/cpt-repository-gateway.html#repository-gateway-configuration).

The workflow of publishing content is covered in detail in the
[CernVM-FS documentation](https://cvmfs.readthedocs.io/en/stable/cpt-repository-gateway.html#publication-workflow).

---

## Repository { #repository }

A CernVM-FS **repository** is where the files and directories that you want to distribute via CernVM-FS are
stored, which usually correspond to a collection of software installations.

It is a form of [*content-addressable storage (CAS)*](https://en.wikipedia.org/wiki/Content-addressable_storage),
and is the single source of (new) data for the file system being presented as a subdirectory of `/cvmfs`
on [client](#client) systems that [mount](https://en.wikipedia.org/wiki/Mount_(computing)) the repository.

!!! note

A CernVM-FS repository includes [**software installations**](#software-installations),
not [*software packages*](https://en.wikipedia.org/wiki/Package_format) like RPMs.
*(see [What is CernVM-FS?](https://multixscale.github.io/cvmfs-tutorial-hpc-best-practices/cvmfs/what-is-cvmfs/))*

---

Expand All @@ -107,33 +26,3 @@ in a single *package* to distribute as a whole.
files that are part of that software installation.

---

## Stratum 0 server { #stratum0 }

A **Stratum 0 server**, often simply referred to a *Stratum 0* (Stratum Zero), is the central server
for one or more CernVM-FS [repositories](#repository).

It is the single source of (new) data, since it hosts the *master copy* of the repository contents.

Adding or updating files in a CernVM-FS repository ([publishing](#publishing)) can only be done on the Stratum 0 server,
either directly via the `cvmfs_server publish` command, or indirectory via a publisher server.

For more information, see the [CernVM-FS documentation](https://cvmfs.readthedocs.io/en/stable/cpt-repo.html).

---

## Stratum 1 replica server { #stratum1 }

A **Stratum 1 replica server**, often simply referred to a *Stratum 1* (Stratum One), is a standard web server
that acts as a mirror server for one or more CernVM-FS [repositories](#repository).

It holds a complete copy of the data for each CernVM-FS repository it serves,
and automatically synchronises with the [Stratum 0](#stratum0).

There is typically a *network* of several Stratum 1 servers for a CernVM-FS repository,
which are geographically distributed.

[Clients](#client) can be configured to automatically connect to the closest Stratum 1 server by using
the CernVM-FS [GeoAPI](https://cvmfs.readthedocs.io/en/stable/cpt-configure.html#ordering-of-servers-according-to-geographic-proximity).

For more information, see the [CernVM-FS documentation](https://cvmfs.readthedocs.io/en/stable/cpt-replica.html).
8 changes: 4 additions & 4 deletions docs/eessi/high-level-design.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,7 @@ and is aligned with the [motivation and goals](motivation-goals.md) of the proje
In the remainder of this section of the tutorial, we will **explore the layered structure of the EESSI software stack**,
and [how to use it](using-eessi.md).

In the [next section](../access/index.md) will cover in detail how you can get access to EESSI
(and other publicly available CernVM-FS repositories).
In the [next section](../access/index.md) will cover in detail how you can get access to EESSI.

## Layered structure

Expand All @@ -26,8 +25,9 @@ the EESSI project consists of 3 layers, which are constructed by leveraging vari
### Filesystem layer {: #filesystem_layer }


The *filesystem layer* uses [**CernVM-FS**](../cvmfs/what-is-cvmfs.md) to distribute the EESSI software stack to
client systems.
The *filesystem layer* uses
CernVM-FS**](https://multixscale.github.io/cvmfs-tutorial-hpc-best-practices/cvmfs/what-is-cvmfs/) to distribute the
EESSI software stack to client systems.

As presented in the previous section, CernVM-FS is a mature open source software project
that was created exactly for this purpose:
Expand Down
4 changes: 2 additions & 2 deletions docs/eessi/inspiration.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,8 @@ The EESSI concept is heavily inspired by software stack provided by the
[Digital Research Alliance of Canada](https://alliancecan.ca/en/about/alliance) (a.k.a. The Alliance, formerly known as *Compute Canada*), which is a shared software stack used on all
[national host sites for Advanced Research Computing in Canada](
https://alliancecan.ca/en/services/advanced-research-computing/federation/national-host-sites)
that is distributed across Canada (and beyond) using [CernVM-FS](../cvmfs/what-is-cvmfs.md);
see also [here](../cvmfs/flagship-repositories.md#the-alliance).
that is distributed across Canada (and beyond) using
[CernVM-FS](https://multixscale.github.io/cvmfs-tutorial-hpc-best-practices/cvmfs/what-is-cvmfs/).

EESSI is significantly more ambitious in its goals however, in various ways.

Expand Down
10 changes: 5 additions & 5 deletions docs/eessi/using-eessi.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,9 @@ Let's break it down step by step.

## 0) Is EESSI available?

First, check whether the EESSI [CernVM-FS](../cvmfs/what-is-cvmfs.md) repository is available on your system.
First, check whether the EESSI
[CernVM-FS](https://multixscale.github.io/cvmfs-tutorial-hpc-best-practices/cvmfs/what-is-cvmfs/) repository is
available on your system.

Try checking the contents of the `/cvmfs/software.eessi.io` directory with the `ls` command:

Expand Down Expand Up @@ -126,12 +128,10 @@ $ command -v python
You may notice a bit of "lag" initially when starting to use software provided by the EESSI software layer.

This is expected, since CernVM-FS may need to first download the files that are required
to run the software you are using;
see also [On-demand downloading of files and metadata](../cvmfs/what-is-cvmfs.md#features-ondemand).
to run the software you are using.

You should not observe any significant start-up delays anymore when running the same software shortly after,
since then CernVM-FS will be able to serve the necessary files from the local client cache;
see also [Multi-level caching](../cvmfs/what-is-cvmfs.md#features-caching).
since then CernVM-FS will be able to serve the necessary files from the local client cache.

---

Expand Down
2 changes: 1 addition & 1 deletion docs/eessi/what-is-eessi.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ is a collaboration between different European partners in the HPC (High Performa

EESSI provides a common stack of *optimized* scientific software installations that work on any Linux distribution,
and currently supports both `x86_64` (AMD/Intel) and `aarch64` (Arm 64-bit) systems, which is distributed
via [CernVM-FS](../cvmfs/what-is-cvmfs.md).
via [CernVM-FS](https://multixscale.github.io/cvmfs-tutorial-hpc-best-practices/cvmfs/what-is-cvmfs/).


---
Expand Down
49 changes: 13 additions & 36 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,63 +5,49 @@
</p>


This is an introductory tutorial to [CernVM-FS](https://cernvm.cern.ch/fs/), the CernVM File System,
This is an introductory tutorial to [EESSI](https://www.eessi.io/docs/), the European Environment
for Scientific Software Installations,
with a focus on employing it in the context of High-Performance Computing (HPC).

In this tutorial you will learn what CernVM-FS is, how to get access to existing CernVM-FS repositories,
how to configure CernVM-FS, and how to use CernVM-FS [repositories](appendix/terminology.md#repository)
In this tutorial you will learn what EESSI is, how to get access to EESSI,
how to customise EESSI, and how to use EESSI [repositories](appendix/terminology.md#repository)
on HPC infrastructure.

<div markdown="1" style="text-align:center;">
**[Ready to go? Click here to start the tutorial!](cvmfs/what-is-cvmfs.md)**
**[Ready to go? Click here to start the tutorial!](eessi/what-is-eessi.md)**
</div>


## Recording

A first virtual edition of this tutorial was held on 4 December 2023,
the recording is available here:

<p align="center">
<iframe width="560" height="315" src="https://www.youtube.com/embed/L0Mmy7NBXDU?si=Ob0DtYN2FH3K169V" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen></iframe>

<br/><a href="https://raw.githubusercontent.com/multixscale/cvmfs-tutorial-hpc-best-practices/main/files/Best-Practices-for-CernVM-FS-in-HPC-20231204.pdf">
<em>slides (PDF) available for download here</em></a>
</p>
Once we have a recording of this tutorial available it will appear here.

### Slides

[Available for download here](https://raw.githubusercontent.com/multixscale/cvmfs-tutorial-hpc-best-practices/main/files/Best-Practices-for-CernVM-FS-in-HPC-20231204.pdf)
Once we have slides for this tutorial available they will appear here.

## Intended audience

This tutorial is intended for people with a background in HPC (system administrators, support team members,
end users, etc.) and who are new to CernVM-FS; no specific prior knowledge or experience with it is required.
This tutorial is intended for a general audience who are familiar with running sofware from the command line;

Check failure on line 31 in docs/index.md

View workflow job for this annotation

GitHub Actions / build

sofware ==> software
no specific prior knowledge or experience is required.

We expect it to be most valuable to people who are interested in using or providing access to one or more existing
CernVM-FS repositories on HPC infrastructure.
We expect it to be most valuable to people who are interested in running scientific software on variety of
compute infrastructures.


## Prerequisites

- Basic knowledge of Linux shell environment
- Basic knowledge of networking (IP address, port, latency)
- Basic knowledge of Linux file systems
- Familiarity with High-Performance Computing environments is a plus
- Hands-on experience with running scientific software workloads is a plus


## Practical information

### Registration

Attendance is free, but **registration is required**: <https://event.ugent.be/registration/cvmfshpc202312>.

Registration for online tutorial on Mon 4 Dec 2023 is **closed** *(since Sun 3 Dec 2023)*

### Slack channel

Dedicated channel in EESSI Slack: [`#cvmfs-best-practices-hpc`](https://eessi-hpc.slack.com/archives/C068DV7GY3V)
Dedicated channel in EESSI Slack: [`#eessi-tutorial`](https://eessi-hpc.slack.com/archives/C068ACS7XK8)

[Click here to join the EESSI Slack](https://join.slack.com/t/eessi-hpc/shared_invite/zt-1wqy0t8g6-PZJTg3Hjjm5Fm3XEOkzECg)

Expand All @@ -81,22 +67,13 @@ Undertaking (JU)](https://eurohpc-ju.europa.eu) and countries participating in t

## Contributors

* Jakob Blomer (CERN, Switzerland)
* Bob Dröge (University of Groningen, The Netherlands)
* Kenneth Hoste (Ghent University, Belgium)
* Alan O'Cais (University of Barcelona, Spain; CECAM)
* Lara Peeters (Ghent University, Belgium)
* Laura Promberger (CERN, Switzerland)
* Thomas Röblitz (University of Bergen, Norway)
* Caspar van Leeuwen (SURF, The Netherlands)
* Valentin Völkl (CERN, Switzerland)


## Additional resources

* [CernVM-FS website](https://cernvm.cern.ch/fs)
* [CernVM-FS documentation](https://cvmfs.readthedocs.io)
* [CernVM-FS @ GitHub](https://github.com/cvmfs)
* [CernVM-FS forum](https://cernvm-forum.cern.ch)
* [Introduction to CernVM-FS by *Jakob Blomer (CERN)* (2021)](https://easybuild.io/eum21/#cvmfs-talk)
* [Introductory tutorial on CernVM-FS (2021)](https://cvmfs-contrib.github.io/cvmfs-tutorial-2021)
* TBD

0 comments on commit f27983a

Please sign in to comment.