league-of-robots 23.04.1

Major changes

• The Calculon cluster is EOL and was replaced by the new nibbler_cluster stack.
• New GD stacks were added: betabarrel_cluster, copperfist_cluster and wingedhelix_cluster.

What's Changed

• Removed boxy and umcg from calculon vars and added both gattaca machines by @marieke-bijlsma in #426
• new user and small bug fix by @kdelange in #427
• Fix/mkdocs by @pneerincx in #428
• Pulp version locking of pulp_installer, pulpcore and pulp-rpm. Working solution for now by @scimerman in #432
• Fix/new lobby by @pneerincx in #433
• Added certs for NEMI irods + README (WIP). by @pneerincx in #429
• Feature/cineca ega client by @pneerincx in #434
• Changes needed for deployment of a Hyperchicken by @scimerman in #435
• Copied ega fuse client credentials from talos. by @erijpkema in #437
• Created pull request: changes in Cinder storage for compute nodes to work for Embassy, edited configuration for new Fender, created Fender CA, minor bug fixes by @scimerman in #438
• Fix/cleanup by @pneerincx in #439
• Feature/isilonquota by @pneerincx in #436
• Update/slurm by @pneerincx in #440
• Switched LDAP for Hyperchicken and Fender from ldap to ldaps with TLS certificate. by @pneerincx in #441
• FD deployment, HC & FD root pulp certificates, FD slurm, corrected generate password script by @scimerman in #442
• Fixed setting TMOUT env variable in slurm.taskprolog. by @pneerincx in #443
• Fix/ldapdhparams by @pneerincx in #444
• Latest changes in DAI and SAI deployment playbook, and tested on Fender DAI by @scimerman in #445
• Updated configs for new EGA Fuse client version 2.1.0. by @pneerincx in #446
• The import from test.test_sax fails after following the instructions. by @FokkeDijkstra in #450
• Added the upgrade of pip, the installation of wheel and the installation of ansible by @FokkeDijkstra in #448
• Added comment to README with regard to installing Ansible with pip. by @pneerincx in #452
• Feature: new data transfer servers by @pneerincx in #451
• Adjust characters used in passphrases by @FokkeDijkstra in #453
• Add rsc01 mount for umcg-pub group on Gearshift. by @pneerincx in #455
• Added missing email addresses for several users. by @pneerincx in #456
• Updated documentation for the adding custom rpm to the pulp repository by @scimerman in #457
• Feature: email by @pneerincx in #458
• Minor updates. by @pneerincx in #460
• Added gecos attribute. by @pneerincx in #459
• Bugfix for chrooted rsync by @pneerincx in #462
• Make sure the ssh_host_signer role does not error due to undefined variable. by @pneerincx in #463
• Added group_vars/template/users.yml. by @pneerincx in #454
• Wingedhelix: created vars, secrets and ssh-CA by @scimerman in #464
• Updating documentation: irods and pulp by @scimerman in #466
• Added hopko as an admin again. by @erijpkema in #467
• Fixed for loop to loop over jumphosts in documentation. by @pneerincx in #468
• Added extra notes from hackathon. by @pneerincx in #470
• irods parameters and icat vm deployment playbook by @Gerbenvandervries in #461
• Updated irods documentation from 2nd hackaton by @scimerman in #471
• Feature/nfs by @pneerincx in #472
• Updated email address for user. by @pneerincx in #473
• Added missing if to make irods group optional in template for /etc/hosts. by @pneerincx in #474
• Fixed config files for LDAP client by @pneerincx in #475
• Updated one public key. by @pneerincx in #476
• Updated Fender playbook by @scimerman in #477
• Added script + cronjob to recycle chrooted home dirs by @pneerincx in #465
• Added explicit settings for mount point root folders. by @pneerincx in #478
• Storage update by @pneerincx in #479
• Fix/slurm logrotate by @pneerincx in #480
• Wingedhelix: update after finished deployment by @scimerman in #481
• new key by @kdelange in #482
• Added: Kim keys by @scimerman in #483
• Feature/dt docs by @pneerincx in #484
• Added network addresses for first client machines. by @pneerincx in #486
• Updated Gearshift config for prm02 mounts from umcgst01 on dh1 Lustre silo. by @pneerincx in #485
• removed user by @kdelange in #487
• Feature: case insensitive slurm accounts by @pneerincx in #488
• Added info for new tl-repo and gs-repo machines for Pulp. by @pneerincx in #489
• Feature/tl-repo by @pneerincx in #490
• Added new umcg-rehabiliation group on Gearshift. by @pneerincx in #491
• Temporarily relocated EGA Fuse client mounts prm03 -> prm02 by @pneerincx in #495
• Bugfixes for logins and ldap roles. by @pneerincx in #494
• Feature/extra groups on nibbler by @pneerincx in #497
• Added some additional tools. Fixes #492. by @pneerincx in #498
• Feature/sssd by @pneerincx in #493
• Updates for DNS entries for Nibbler. by @pneerincx in #499
• Small changes: the working FD variables by @scimerman in #500
• Feature/isilon quota by @pneerincx in #496
• Hostnames change: cluster playbook - when hostnames are defined, the … by @scimerman in #501
• Update MOTD on Gearshift: beta -> production. by @pneerincx in #503
• Updated SRAM LDAP config for Nibbler. by @pneerincx in #502
• new user by @Gerbenvandervries in #505
• new user: mchahdil by @BenjaminsM in #506
• iRODS: v0.9.8 by @scimerman in #504
• Feature: new docs_library group and introduced more generic stack_name for groups of machines. by @pneerincx in #507
• Fixed the comment at Hopko's name. by @erijpkema in #509
• Fix/subgroup_directories by @pneerincx in #510
• Added pip3 to list of packages installed on DAIs. Added host_vars/README.txt. by @pneerincx in #511
• Fix to allow NFS to lookup GIDs. by @pneerincx in #512
• fix SLURM lua: assigned job to correct group account by @scimerman in #513
• Updated Circle CI config. by @pneerincx in https:/...

league-of-robots 21.06.1

Lots of new features and fixes.

league-of-robots 20.08.1

• Updated documentation including:
  • complete rewrite for logins sections and
  • AppleScripts for macOS users to configure their ssh client and mount shares from the the clusters using sshfs.
• Various bug fixes including
  • Fix for log rotation
  • Override OS default swappiness for Slurm cgroups and set it to zero.
  • Added missing roles to main cluster.yml playbook.
• Dedockered mariadb role
• Improved syntax: Added ansible-linter for pull requests and reduced ansible-linter errors from lots to one.
• Updated subgroup_directories role, so it can be used without root permissions.
• Feature: functional admins group:
  • Added variable for functional admins group to group_vars for Gearshift and Talos.
  • Allow functional admins group to sudo to the functional accounts of all groups.
    (Required to deploy subgroup directories with the corresponding role without root privileges.)
  • Allow functional admins group to sudo to the slurm user and manage jobs.
• Feature: Update public key LDAP attributes when keys have changed in repo.
• Added role to disable core dumps.
• Added quota support for Lustre file systems. Uses group quota as default (for now), because project quota are not supported on Data Handling Lustre file systems (yet).
• Added TLS support for grafana proxy.
• Implemented tiny quota for regular (non-admin) users on jumphosts.
• Security hardening for documentation server, which is accessible directly from the internet just like the jumphost.
• Added various users and groups on various sites.

league-of-robots 19.11.1

• Lot's of bug fixes including #43, #100, #141, #144, #151, #160
• Various improved/new roles:
  • sudoers: cleanup when rights are revoked and use /etc/sudoers.d/
  • slurm-*
  • online_docs: updated, bug fixes and rewrote instructions for users in Windows clients to use MobaXterm as opposed to PuTTY.
  • build-environment
  • prom_proxy
• Updated to new spacewalk server
• Updated CentOS 7.6 -> 7.7
• Updated Slurm to 17.x -> 18.08.8
• Removed several separate playbooks + re-ordered stuff in repo resulting in a single cluster.yml playbook to deploy everything and additional single_role_playbooks/*.yml to deploy/test debug individual steps.
• Added various RPMs to list of software that should be installed as part of the default environment of machines. Includes also FUSE-based EGA client.
• Lowered memory amount available for vcompute VMs to prevent OOM killer destroying VMs when the host OS runs out of memory.
• Refactored the LDAP wrapper, which fetches the public keys from LDAP for SSH logins, in Python.
• Addecd circleci config.
• Added new users and groups

league-of-robots 19.06.1

• Lots of bug fixes and minor improvements.
• Updated vars/configs for all clusters.
• Added Fender cluster.
• De-dockerized slurm-management role.
• Added various users, groups and mount points for group dirs.
• Added role to generate user documentation for each cluster based on vars/config in this repo
• Added envsync role to configure the HPC environment and allow synchronization using an envsync account. Works together with logic from https://github.com/molgenis/ansible-pipelines
• Refactored and updated documentation for better separation of deployment of OpentStack clouds using https://github.com/rug-cit-hpc/hpc-cloud and deployment of virtual HPC clusters using this repo.
• Updated callback plugins and dynamic inventory for Python 3.x
• Added role to configure a custom LDAP for a cluster and add users + groups to that LDAP.

league-of-robots 19.03.1

135 commits resulting in successful slurm job submission and completion on Talos and Hyperchicken.

league-of-robots 19.01.2

• Many bugfixes.
• Cleaner code, less hardcoded stuff in roles and more vars to make them more generic.
• Moved group_vars subdirs to match groups in inventories.
• Extra machines for Talos to remove dependencies on a few Gearshift machines and better separation of development versus production.

league-of-robots 19.01.1

• First release working towards public beta, but some bits are still work in progress.
• Inventories contain hosts for 3 clusters:
  ** Gearshift
  ** Talos
  ** Hyperchicken