Skip to content

Commit

Permalink
Merge pull request #5659 from mirelaminkova/patch-1
Browse files Browse the repository at this point in the history
Update 2025-01-28-src.md
  • Loading branch information
hexylena authored Dec 19, 2024
2 parents c3ebd75 + 60497df commit 0c3879a
Show file tree
Hide file tree
Showing 9 changed files with 68 additions and 13 deletions.
17 changes: 11 additions & 6 deletions events/2025-01-28-src.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,15 @@
layout: event

# Status of this page, remove these lines when ready
draft: true
status: wip
#draft: true
#status: wip

# Description of your event
title: Galaxy at SURF Research Cloud workshop
description: |
Half-day workshop demonstrating the local Galaxy instance built for users of SURF Research Cloud.
Half-day workshop demonstrating the local Galaxy instance built for users of SURF Research Cloud.
**Registration**: To sign up for the event, please send an email to [Mirela Minkova](mailto:[email protected]?subject=Registering%20SRC%20Galaxy%20Workshop)
Expand Down Expand Up @@ -79,11 +81,11 @@ program:

- name: surf-research-cloud-pulsar
topic: admin
time: "14:10 - 14:40"
time: "14:10 - 14:30"

- type: custom
name: Coffee Break
time: "14:40"
time: "14:30"
- name: mapping
topic: sequence-analysis
time: "14:40 - 15:40"
Expand All @@ -99,7 +101,10 @@ program:
- type: custom
name: Borreltje
time: "16:30 - 17:00"

tags:
- admin
- surf
- netherlands
---

During the workshop, you will see a short demo and get a chance to test the catalog item yourself. This will be followed by a discussion about your needs as Galaxy researchers, helping us work toward a better future for the Dutch research community. We will conclude the day with drinks and networking with fellow Galaxy NL users.
18 changes: 15 additions & 3 deletions topics/admin/tutorials/cloudbursting/slides.html
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,14 @@
logo: assets/images/gat.png

title: "Galaxy on the Cloud"
contributors:
contributions:
authorship:
- slugger70
editing:
- hexylena
subtopic: cloud

priority: 0
---

# All these Clouds
Expand Down Expand Up @@ -84,7 +89,7 @@
* Education grants
* Academic:
* OpenStack: open source community project
* NeCTAR in Australia, Jetstream in USA, CLIMB in UK, lots of others
* NeCTAR in Australia, Jetstream in USA, CLIMB in UK, SURF in Netherlands, lots of others
* Some free for researchers (NeCTAR, CLIMB), some with project grants (Jetstream)
]

Expand Down Expand Up @@ -129,7 +134,7 @@
* with Galaxy pre-installed
* with different sets of tools installed
* with access to reference data
* for different clouds (AWS globally, Jetstream, NeCTAR, CLIMB etc.)
* for different clouds (AWS globally, Jetstream, NeCTAR, CLIMB, SURF etc.)
* You just need credentials for the cloud you want to "launch" on.
* Credentials are generally strings
* An access key and a secret key or username and password with project details
Expand Down Expand Up @@ -282,6 +287,13 @@
<tool id="toolshed.g2.bx.psu.edu/repos/devteam/bwa/bwa/0.3.1" destination="slurm_4slots" />
...
```

---

# CloudMan Galaxy

* Configured for Slurm out of the box

``` ini
# COMPUTE NODES
NodeName=master NodeAddr=45.113.232.91 CPUs=15 RealMemory=64431 Weight=10 State=UNKNOWN
Expand Down
2 changes: 1 addition & 1 deletion topics/admin/tutorials/k8s-managing-galaxy/tutorial.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ requirements:
topic_name: admin
tutorials:
- k8s-deploying-galaxy
priority: 1
priority: 3
---

# Managing Galaxy on Kubernetes
Expand Down
2 changes: 2 additions & 0 deletions topics/admin/tutorials/surf-research-cloud-galaxy/tutorial.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,8 @@ contributions:
funding:
- surf

priority: 10

edam_ontology:
- topic_0605 # Informatics
- topic_3071 # Data Management
Expand Down
33 changes: 30 additions & 3 deletions topics/admin/tutorials/surf-research-cloud-pulsar/tutorial.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,12 @@ questions:
- How do I start a Pulsar instance on SURF Research Cloud?
- How do I connect to Pulsar?
objectives:
- Be able to attach a Pulsar node to Galaxy and use interactive tools
- Be able to attach a Pulsar node to Galaxy
- Send jobs to Pulsar
key_points:
- With SRC you can start your own Pulsar on-demand instance in a secure environment.
- The Pulsar node is publicly accessible for a Galaxy with the credentials to use.
- You can send jobs to a Pulsar node from either a Galaxy instance running on SRC, or even inside your own network.
requirements:
- type: "none"
title: Access to the SURF Research Cloud
Expand All @@ -26,6 +31,8 @@ contributions:
funding:
- surf

priority: 11

edam_ontology:
- topic_0605 # Informatics
- topic_3071 # Data Management
Expand All @@ -49,9 +56,14 @@ tags:

Using Pulsar via the {SRC} allows researchers to start Pulsar instances on-demand to expand their computational resources and even access GPUs to help and analyze their data in a secure environment following the {GDPR}.

The instance provides secure authentication, where users must have a SURF Research account prior to this tutorial, have set the {SRAM} authentication method, and connect an SSH key to their accounts. In case you are not familiar with {SRC} and need help in setting up your accounts, please follow the instructions on the [SURF Knowledge Base](https://servicedesk.surf.nl/wiki/display/WIKI/SURF+Research+Cloud)
There are two main use cases we envision this role being useful for:

Saving costs on SRC
: Maybe you're already running Galaxy in SRC, but you don't want to run a GPU node because it is very expensive. By using the SRC Pulsar Catalog Item, you can launch a node to do computations and then shut it down when you're done, saving money. Pulsar instances can be started and stopped on demand, depending on personal cases and requirements, giving you a lot of freedom!

Accessing a GPU from a local (in UMC/hospital Galaxy)
: If you do not have a GPU easily available within your institute, it may be attractive to send jobs securely to SRC, by launching a Pulsar node in SRC and attaching it to your institute's Galaxy instance.

Pulsar instances can be started and stopped on demand, depending on personal cases and requirements. Inside the SRC members should have access to all publicly available catalog items. If you are not able to create a catalog item, please [contact SURF servicedesk](mailto:[email protected]).

> <agenda-title></agenda-title>
>
Expand All @@ -62,6 +74,10 @@ Pulsar instances can be started and stopped on demand, depending on personal cas

# Prerequisites

The instance provides secure authentication, where users must have a SURF Research account prior to this tutorial, have set the {SRAM} authentication method, and connect an SSH key to their accounts. In case you are not familiar with {SRC} and need help in setting up your accounts, please follow the instructions on the [SURF Knowledge Base](https://servicedesk.surf.nl/wiki/display/WIKI/SURF+Research+Cloud)

Inside the SRC members should have access to all publicly available catalog items. If you are not able to create a catalog item, please [contact SURF servicedesk](mailto:[email protected]).

This tutorial assumes you are member of a {CO} in {SRAM} that has access to {SRC} and a wallet with budget in SRC with enough sources to create Galaxy and Pulsar catalog items. (For more information please refer to the [SURF Knowledge Base](https://servicedesk.surf.nl/wiki/display/WIKI/Budgets%2C+wallets%2C+contracts).

You should have previous experience working with data inside Galaxy.
Expand Down Expand Up @@ -229,4 +245,15 @@ You can find the tool ID from the dropdown at the top right, just to the left of
With that, you're done, and for the length of time your node is running, your chosen tools (or everything) will be executed on that Pulsar node with more memory and CPU than the Galaxy host, and maybe a GPU as well!
> <hands-on-title>Launch a Job on Pulsar</hands-on-title>
> 1. Login to your Galaxy
> 2. Run one of the tools you have decided to send to Pulsar
> 3. On the pulsar machine, you can check that it runs by following the logs:
>
> ```bash
> sudo journalctl -fu pulsar
> ```
>
{: .hands_on}
Congratulations on launching Pulsar in SRC! 🌌
2 changes: 2 additions & 0 deletions topics/admin/tutorials/terraform/slides.html
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@
subtopic: cloud
contributors:
- hexylena

priority: 30
---

### Why Terraform
Expand Down
2 changes: 2 additions & 0 deletions topics/admin/tutorials/terraform/tutorial.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@ tags:
- deploying
- cloud
priority: 3

priority: 30
---

# Overview
Expand Down
4 changes: 4 additions & 0 deletions topics/admin/tutorials/wireguard-headscale/tutorial.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,10 @@ subtopic: cloud
tags:
- wireguard
- networking
edam_ontology:
- topic_3263 # data security
- topic_3372 # software engineering
priority: 20
---

[Tailscale](https://tailscale.com/) makes secure networking easy, it really is like magic. If you've used wireguard before, you know it takes a bit to setup and some configuration if you need to do anything fancy.
Expand Down
1 change: 1 addition & 0 deletions topics/admin/tutorials/wireguard/tutorial.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ subtopic: cloud
tags:
- wireguard
- networking
priority: 20
---

In this tutorial we will briefly cover what [Wireguard](https://www.wireguard.com/) is and how you can leverage it for your needs. This will not make you an expert on Wireguard but will give you the tools you need in order to setup a local Wireguard network.
Expand Down

0 comments on commit 0c3879a

Please sign in to comment.