Skip to content

Commit

Permalink
Merge branch 'main' into galaxy_interactive_tools_high_availability
Browse files Browse the repository at this point in the history
  • Loading branch information
kysrpex authored Oct 29, 2024
2 parents 6b90206 + 9969bf8 commit 48985e2
Show file tree
Hide file tree
Showing 773 changed files with 651,402 additions and 7,913 deletions.
18 changes: 11 additions & 7 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,13 @@
--> Please remove all this text before submitting your PR, it's just a checklist to help you :) <--
TODO: Describe your changes here..

1. Give your pull request a **good title**
- e.g. "Fix typo in ansible-galaxy tutorial" or "Add new transcriptomics tutorial covering a new sequencing technology"
2. Describe your changes in detail, list anything you still need some help with or things that are still TODO
3. Check that your images are allowed to be re-hosted by the GTN!
4. Still working on it? Make it a **Draft** pull request
- Once it is ready, choose **Ready for Review**
<!-- Contributor Checklist
1. Give your pull request a descriptive title
2. Describe your changes in detail at the top of this text box
3. List anything you still need some help with or things that are still TODO
4. Check that your images are allowed to be re-hosted by the GTN!
5. Not ready for review yet? Make it a **Draft** pull request
- Once you are done making changes, choose **Ready for Review**
- Then the automated tests will run and we will know to review and merge it
-->
6 changes: 3 additions & 3 deletions .github/workflows/ci-main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ on:
- 'news/**'
- 'events/**'
- CONTRIBUTORS.yaml
- FUNDERS.yaml
- GRANTS.yaml
- ORGANISATIONS.yaml


Expand Down Expand Up @@ -66,8 +66,8 @@ jobs:
- name: Check Tutorial Formatting
run: |
find topics/ -name tutorial.md | xargs -n 1 python bin/check-broken-boxes.py | reviewdog -efm="%f:%l: %m" -filter-mode=file -reporter=github-pr-review
bundle exec ruby bin/lint.rb --format rdjson | reviewdog -filter-mode=file -reporter=github-pr-review -f=rdjsonl -fail-on-error
find topics/ -name tutorial.md | xargs -n 1 python bin/check-broken-boxes.py | reviewdog -efm="%f:%l: %m" -filter-mode=file -reporter=github-pr-review -fail-level=any
bundle exec ruby bin/lint.rb --format rdjson | reviewdog -filter-mode=file -reporter=github-pr-review -f=rdjsonl -fail-level=any
env:
REVIEWDOG_GITHUB_API_TOKEN: ${{ secrets.GITHUB_TOKEN }}

Expand Down
15 changes: 2 additions & 13 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: "[CI] Accessibility Linting"
name: "[CI] GTN Build Testing"

concurrency:
group: ci-${{ github.head_ref }}
Expand All @@ -8,17 +8,6 @@ concurrency:
on:
pull_request:
types: [opened, reopened, synchronize, ready_for_review]
paths:
- 'Gemfile.lock'
- '**/*.md'
- '**/*.html'
- '*.yml'
- '*.yaml'
- 'metadata/*'
- 'topics/*/images/*'
- '_plugins/*'
- '_plugins_dev/*'
- '**/*.bib'

jobs:
build-site:
Expand Down Expand Up @@ -58,7 +47,7 @@ jobs:
- name: Validate all links, enforce alt text
run: |
bundle exec htmlproofer \
--ignore-urls "/.*localhost.*/","/.*vimeo\.com.*/","/.*gitter\.im.*/","/.*drmaa\.org.*/","/.*slides.html#.*/,/#embedded_jbrowse/","/.*videos.*.mp4.png/","/krona_(all|multisample).html/" \
--ignore-urls "/.*localhost.*/","/.*vimeo\.com.*/","/.*gitter\.im.*/","/.*drmaa\.org.*/","/.*slides.html#.*/,/#embedded_jbrowse/","/.*videos.*.mp4.png/","/krona_(all|multisample).html/","/workflows\/trs_import/" \
--ignore-files "/.*krona.*\.html/","/.*\/files\/.*/","/.*\/node_modules\/.*/","/\/tutorials\/.*\/docker\//","/.*content.html/","/.*recentrifuge.*\.html/" \
--swap-urls "github.com/galaxyproject/training-material/tree/main:github.com/${GITHUB_REPOSITORY}/tree/${GITHUB_HEAD_REF}" \
--disable-external=true \
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/deploy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -51,11 +51,12 @@ jobs:
${{ runner.os }}-gems-
- name: Install dependencies
run: |
sudo apt-get install -y build-essential libxi-dev libglu1-mesa-dev libglew-dev pkg-config # for node
sudo apt-get install -y build-essential libxi-dev libglu1-mesa-dev libglew-dev pkg-config graphviz # for node
gem install bundler
bundle config path vendor/bundle
bundle install --jobs 4 --retry 3
npm install
rm -f package.json package-lock.json yarn.lock
npm install markdown-yaml-metadata-parser lunr
bundle pristine ffi
# END Dependencies

Expand All @@ -68,7 +69,6 @@ jobs:
make annotate ACTIVATE_ENV=pwd
curl -L https://docs.google.com/spreadsheets/d/1NfZhi5Jav7kl9zFCkeb7rIC2F8xW1isruv1TeO4WpNI/export\?format\=tsv | ruby bin/prepare_feedback.rb
curl -L https://hexylena.github.io/toolshed-version-database/tool-meta.json > metadata/tool-meta.json
npm install
make rebuild-search-index ACTIVATE_ENV=pwd
cat metadata/swagger.yaml | python bin/yaml2json.py > api/swagger.json
rdoc bin _plugins/ --output gtn_rdoc
Expand Down
28 changes: 15 additions & 13 deletions .github/workflows/google-form-recordings.yml
Original file line number Diff line number Diff line change
Expand Up @@ -57,18 +57,20 @@ jobs:
New recording submitted!
**Submission Process checklist**
- [x] **Instructor** Record video
- [x] **Instructor** Submit video via form
- [x] **GTN** Pull request made by bot
- [ ] **GTN** Check if adjustments to PR are necessary
- [ ] Check [Submissions Form](https://docs.google.com/spreadsheets/d/1iXjLlMEH5QMAMyUMHi1c_Lb7OiJhL_9hgJrtAsBoZ-Y/edit?usp=sharing)
- [ ] New contributors need to be added?
- [ ] **GTN** Upload video to [GalaxyProject YouTube](https://www.youtube.com/c/galaxyproject)
- [ ] **GTN** Add youtube video id to PR after upload
- [ ] **GTN** Put Autogenerated captions in Google Doc ([folder](https://drive.google.com/drive/folders/1liyqDQDbxMNXvpQIaqFmoE2fB2aJIH9N?usp=drive_link))
- [ ] **Instructor** Fix autogenerated captions, comment here when done
- [ ] **GTN** Upload fixed captions to YouTube
- [ ] **GTN** Add the people who did the captioning to this PR
- [ ] **GTN** Merge PR!
- [x] **Instructor:** Record video
- [x] **Instructor:** Submit video via form
- [x] **GTN:** Pull request made by bot
- [ ] **GTN:** Check [Submissions Spreadsheet](https://docs.google.com/spreadsheets/d/1iXjLlMEH5QMAMyUMHi1c_Lb7OiJhL_9hgJrtAsBoZ-Y/edit?usp=sharing)
- [ ] Mark the 'PR made?' column as 'yes' in the spreadsheet
- [ ] Ping speakers on this PR
- [ ] Add any new contributors to `CONTRIBUTORS.yaml` file
- [ ] **GTN:** Upload video to [GalaxyProject YouTube](https://www.youtube.com/c/galaxyproject)
- [ ] **GTN:** Add YouTube video id to PR after upload
- [ ] **GTN:** Put Autogenerated captions in Google Doc ([folder](https://drive.google.com/drive/folders/1liyqDQDbxMNXvpQIaqFmoE2fB2aJIH9N?usp=drive_link))
- [ ] **GTN:** Check that permissions on the Doc are set to "Everybody with the link can edit"
- [ ] **Instructor:** Fix autogenerated captions, comment here when done
- [ ] **GTN:** Upload fixed captions to YouTube
- [ ] **GTN:** Add the people who did the captioning to this PR
- [ ] **GTN:** Merge PR!
5 changes: 3 additions & 2 deletions .github/workflows/monthly-release-backfill.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ jobs:
with:
ref: "${{ github.event.inputs.selected_tag }}"
fetch-depth: 0
fetch-tags: true

# BEGIN Dependencies
- uses: actions/setup-python@v5
Expand Down Expand Up @@ -46,7 +47,8 @@ jobs:
pip install pyyaml
bundle config path vendor/bundle
bundle install --jobs 4 --retry 3
npm install
rm -f package.json package-lock.json yarn.lock
npm install markdown-yaml-metadata-parser lunr
bundle pristine ffi
# END Dependencies

Expand All @@ -67,7 +69,6 @@ jobs:
sed -i s"|^github_repository_branch: .*|github_repository_branch: '${SOURCE_TAG}'|g" _config.yml
sed -i s"|^title: .*|title: 'GTN Archive ${SOURCE_TAG}'|g" _config.yml
curl -L https://hexylena.github.io/toolshed-version-database/tool-meta.json > metadata/tool-meta.json
npm install
make rebuild-search-index ACTIVATE_ENV=pwd
cat metadata/swagger.yaml | python bin/yaml2json.py > api/swagger.json
JEKYLL_ENV=production bundle exec jekyll build --strict_front_matter -d _site/training-material
Expand Down
7 changes: 5 additions & 2 deletions .github/workflows/monthly-release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ jobs:
with:
# Based on logic in deploy.yml
fetch-depth: 1200
fetch-tags: true

# BEGIN Dependencies
- uses: actions/setup-python@v5
Expand Down Expand Up @@ -53,7 +54,8 @@ jobs:
pip install pyyaml
bundle config path vendor/bundle
bundle install --jobs 4 --retry 3
npm install
rm -f package.json package-lock.json yarn.lock
npm install markdown-yaml-metadata-parser lunr
bundle pristine ffi
# END Dependencies

Expand All @@ -69,7 +71,6 @@ jobs:
sed -i s"|^github_repository_branch: .*|github_repository_branch: '${SOURCE_TAG}'|g" _config.yml
sed -i s"|^title: .*|title: 'GTN Archive ${SOURCE_TAG}'|g" _config.yml
curl -L https://hexylena.github.io/toolshed-version-database/tool-meta.json > metadata/tool-meta.json
npm install
make rebuild-search-index ACTIVATE_ENV=pwd
cat metadata/swagger.yaml | ruby bin/yaml2json.rb > api/swagger.json
JEKYLL_ENV=production bundle exec jekyll build --strict_front_matter -d _site/training-material
Expand All @@ -93,5 +94,7 @@ jobs:

- name: Create Release
id: create_release
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
gh release create ${{ env.release_tag }} --title "Release ${{ env.release_title }}" --notes "Monthly release of the GTN materials is now [available on the archive](https://training.galaxyproject.org/archive/${{ env.release_tag }}/)." --target main --latest
73 changes: 73 additions & 0 deletions .github/workflows/social.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
name: "[Social] Autogenerate missing social media cards"

on:
schedule:
# * is a special character in YAML so you have to quote this string
# We'll run this weekly at noon.
- cron: '0 12 * * 0'
workflow_dispatch:
inputs:
all_old:
type: string
description: If you want to re-generate ALL images set to a non-empty value

jobs:
runner-job:
if: github.repository_owner == 'galaxyproject'
runs-on: ubuntu-24.04
steps:
# Shallow should be fine for video
- uses: actions/checkout@v4
with:
fetch-depth: 500

# BEGIN Dependencies
- uses: actions/setup-python@v5
with:
python-version: '3.11'
architecture: 'x64'
- name: Install dependencies
run: |
sudo apt-get install -y imagemagick optipng inkscape
pip install requests awscli
mkdir -p ~/.fonts/
wget https://galaxy-training.s3.amazonaws.com/social/recursive.ttf -O ~/.fonts/recursive.ttf
fc-cache -rv
# END Dependencies


# We get the previous build ID to know how many videos were changed since
# then.
#
# We *could* fetch the diff in the past N days but if a build fails for
# any reason, we'd lose those changes permanently and things would never
# get built.
#
# So instead we just track the last successfully built timestamp
- name: Previous build ID
id: build_id
run: |
echo "timestamp=$(curl https://galaxy-training.s3.amazonaws.com/social/timestamp.txt -f 2>/dev/null || echo none)" >> $GITHUB_ENV
# Support regenerating all
if [[ "${{ github.event.inputs.all_old }}" != "" ]]; then
echo "timestamp=0" >> $GITHUB_ENV
fi
# The actual compilation process deposits everything in the ./social/ folder.
# And requires fetching metadata from the live website (api/social-meta.json)
- name: Build Social Media Cards
run: |
mkdir -p social/
python bin/social-cards.py $PREVIOUS_TIMESTAMP
env:
PREVIOUS_TIMESTAMP: ${{ env.timestamp }}

# Deploy step, scary.
- name: Build Social Media Cards
run: |
aws s3 cp --recursive ../gtn-social/ s3://galaxy-training/social/
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
AWS_DEFAULT_REGION: us-east-1
2 changes: 1 addition & 1 deletion .github/workflows/video.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ on:
jobs:
runner-job:
if: github.repository_owner == 'galaxyproject'
runs-on: ubuntu-latest
runs-on: ubuntu-22.04
services:
# Label used to access the service container
mozillatts:
Expand Down
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -25,3 +25,4 @@ plnmotmp*.json
# Planemo output for workflow autoupdate
tools.yml
api/**
/.obsidian
Loading

0 comments on commit 48985e2

Please sign in to comment.