Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

TSPS-174 renaming non cosmetic things from tsps to teaspoons #93

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

TSPS-174 renaming non cosmetic things from tsps to teaspoons #93

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
e84b9bd
first pass at renaming things from tsps to teaspoons for non cosmetic…
Jun 17, 2024
a89ca40
consilidate changesets before releasing teaspoons
Nov 6, 2024
44fcd84
get consolidate changesets working with tests and against dev
Nov 7, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/actions/create-bee/action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ inputs:
required: false
type: string
custom_version_json:
description: 'json containing custom versions to push. e.g. {"tsps":{"appVersion":"0.0.81-66ceced"}}'
description: 'json containing custom versions to push. e.g. {"teaspoons":{"appVersion":"0.0.81-66ceced"}}'
required: false
type: string

Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/build-and-test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -136,7 +136,7 @@ jobs:
SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}
with:
status: failure
channel: "#terra-tsps-alerts"
channel: "#terra-teaspoons-alerts"
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how hard is it to change slack channel names?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not hard!

username: "Teaspoons push to main branch"
author_name: "build-and-test"
icon_emoji: ":triangular_ruler:"
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/run-e2e-tests.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ jobs:
if [ -z ${{ inputs.custom-app-version }} ]; then
echo "custom-app-version-formatted={}" >> $GITHUB_OUTPUT
else
echo "custom-app-version-formatted={\\\"tsps\\\": {\\\"appVersion\\\":\\\"${{ inputs.custom-app-version }}\\\"} }" >> $GITHUB_OUTPUT
echo "custom-app-version-formatted={\\\"teaspoons\\\": {\\\"appVersion\\\":\\\"${{ inputs.custom-app-version }}\\\"} }" >> $GITHUB_OUTPUT
fi
project_name=$(echo "tmp-billing-project-$(uuidgen)" | cut -c -30)
echo "project_name=${project_name}" >> $GITHUB_OUTPUT
Expand Down Expand Up @@ -89,7 +89,7 @@ jobs:
permissions:
contents: read
id-token: write
uses: broadinstitute/dsp-reusable-workflows/.github/workflows/run_tsps_e2e_tests.yaml@main
uses: broadinstitute/dsp-reusable-workflows/.github/workflows/run_teaspoons_e2e_tests.yaml@main
with:
billing-project-name: '${{ needs.init-github-context-and-params-gen.outputs.project-name }}'
bee-name: '${{ needs.init-github-context-and-params-gen.outputs.bee-name }}'
Expand All @@ -116,6 +116,6 @@ jobs:
uses: broadinstitute/sherlock/.github/workflows/client-report-workflow.yaml@main
if: github.ref == 'refs/heads/main'
with:
notify-slack-channels-upon-workflow-completion: "#terra-tsps-alerts"
notify-slack-channels-upon-workflow-completion: "#terra-teaspoons-alerts"
permissions:
id-token: write
6 changes: 3 additions & 3 deletions .github/workflows/tag-publish.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -216,7 +216,7 @@ jobs:
env:
SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}
with:
channel: '#terra-tsps-alerts'
channel: '#terra-teaspoons-alerts'
status: failure
author_name: Publish image
fields: job
Expand All @@ -230,7 +230,7 @@ jobs:
if: ${{ needs.bump-check.outputs.is-bump == 'no' }}
with:
new-version: ${{ needs.tag-job.outputs.tag }}
chart-name: 'tsps'
chart-name: 'teaspoons'
permissions:
contents: 'read'
id-token: 'write'
Expand All @@ -242,7 +242,7 @@ jobs:
if: ${{ needs.bump-check.outputs.is-bump == 'no' && github.event_name != 'pull_request' }}
with:
new-version: ${{ needs.tag-job.outputs.tag }}
chart-name: 'tsps'
chart-name: 'teaspoons'
environment-name: 'dev'
secrets:
sync-git-token: ${{ secrets.BROADBOT_TOKEN }}
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/workflow-tester.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,7 @@ jobs:
SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}
with:
status: ${{ job.status }}
channel: "#terra-tsps-alerts"
channel: "#terra-teaspoons-alerts"
username: "Teaspoons push to main branch"
author_name: "build-and-test"
icon_emoji: ":triangular_ruler:"
Expand Down
14 changes: 7 additions & 7 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

## Overview

Terra Scientific Pipelines Service, or teaspoons, facilitates running a number of defined scientific pipelines
Terra Scientific Pipelines Service, or Teaspoons, facilitates running a number of defined scientific pipelines
on behalf of users that users can't run themselves in Terra. The most common reason for this is that the pipeline
accesses proprietary data that users are not allowed to access directly, but that may be used as e.g. a reference panel
for imputation.
Expand Down Expand Up @@ -62,7 +62,7 @@ To run locally:
If using Intellij (only IDE we use on the team), you can run the server with a debugger. Follow
the steps above but instead of running `./gradlew bootRun` to spin up the server, you can run
(debug) the App.java class through intellij and set breakpoints in the code. Be sure to set the
GOOGLE_APPLICATION_CREDENTIALS=config/tsps-sa.json in the Run/Debug configuration Environment Variables.
GOOGLE_APPLICATION_CREDENTIALS=config/teaspoons-sa.json in the Run/Debug configuration Environment Variables.

### Running Tests/Linter Locally
- Testing
Expand All @@ -88,7 +88,7 @@ SonarQube and want to debug the problem locally, you need to get the sonar token
before running the gradle task.

```shell
export SONAR_TOKEN=$(vault read -field=sonar_token secret/secops/ci/sonarcloud/tsps)
export SONAR_TOKEN=$(vault read -field=sonar_token secret/secops/ci/sonarcloud/teaspoons)
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we want to change our vault path?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good question for @choover-broad

./gradlew sonarqube
```

Expand All @@ -106,7 +106,7 @@ does all the setup for you. Clone that repo and make sure you're either on Broad
to the VPN. Then run the following command:

```shell
./db/psql-connect.sh dev tsps
./db/psql-connect.sh dev teaspoons
```

### Deploying to dev
Expand All @@ -115,8 +115,8 @@ Upon merging to main, the dev environment will be automatically deployed via the
(that workflow is defined [here](https://github.com/DataBiosphere/terra-scientific-pipelines-service/blob/main/.github/workflows/tag-publish.yml)).

The two tasks `report-to-sherlock` and `set-version-in-dev` will prompt Sherlock to deploy the new version to dev.
You can check the status of the deployment in [Beehive](https://beehive.dsp-devops.broadinstitute.org/apps/tsps) and in
[ArgoCD](https://ap-argocd.dsp-devops.broadinstitute.org/applications/ap-argocd/tsps-dev).
You can check the status of the deployment in [Beehive](https://beehive.dsp-devops.broadinstitute.org/apps/teaspoons) and in
[ArgoCD](https://ap-argocd.dsp-devops.broadinstitute.org/applications/ap-argocd/teaspoons-dev).

For more information about deployment to dev, check out DevOps' [excellent documentation](https://docs.google.com/document/d/1lkUkN2KOpHKWufaqw_RIE7EN3vN4G2xMnYBU83gi8VA/).

Expand All @@ -129,7 +129,7 @@ See [this DSP blog post](https://broadworkbench.atlassian.net/wiki/x/AoGlrg) for
### Running the end-to-end tests

The end-to-end test is specified in `.github/workflows/run-e2e-tests.yaml`. It calls [the test script defined
in the dsp-reusable-workflows repo](https://github.com/broadinstitute/dsp-reusable-workflows/blob/main/e2e-test/tsps_e2e_test.py).
in the dsp-reusable-workflows repo](https://github.com/broadinstitute/dsp-reusable-workflows/blob/main/e2e-test/teaspoons_e2e_test.py).

The end-to-end test is automatically run nightly on the dev environment.

Expand Down
2 changes: 1 addition & 1 deletion scripts/write-config.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -181,7 +181,7 @@ function vaultgetdb {
}

# grab teaspoons service account json from vault
vaultget "secret/dsde/firecloud/${fcenv}/tsps/tsps-account.json" "${outputdir}/tsps-sa.json"
vaultget "secret/dsde/firecloud/${fcenv}/teaspoons/teaspoons-account.json" "${outputdir}/teaspoons-sa.json"

# We made it to the end, so record the target and avoid redos
echo "$target" > "${outputdir}/target.txt"
2 changes: 1 addition & 1 deletion service/build.gradle
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -119,7 +119,7 @@ dependencies {
// set GOOGLE_APPLICATION_CREDENTIALS if this file exists - should only exist when
// write-config.sh is run.
// GOOGLE_APPLICATION_CREDENTIALS is set for us when running in a deployed environment
def googleCredentialsFile = "${rootDir}/config/tsps-sa.json"
def googleCredentialsFile = "${rootDir}/config/teaspoons-sa.json"
bootRun {
if(project.file(googleCredentialsFile).exists()) {
environment.put("GOOGLE_APPLICATION_CREDENTIALS", "${googleCredentialsFile}")
Expand Down
6 changes: 2 additions & 4 deletions service/src/main/java/bio/terra/pipelines/dependencies/stairway/JobService.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -156,8 +156,7 @@ public <T> JobResultOrException<T> retrieveJobResult(

switch (flightState.getFlightStatus()) {
case FATAL:
logAlert(
"Teaspoons Stairway flight {} encountered dismal failure", flightState.getFlightId());
logAlert("Stairway flight {} encountered dismal failure", flightState.getFlightId());
return handleFailedFlight(flightState);
case ERROR:
return handleFailedFlight(flightState);
Expand Down Expand Up @@ -264,8 +263,7 @@ private <T> JobResultOrException<T> handleFailedFlight(FlightState flightState)
.exception(new InternalServerErrorException("wrap non-runtime exception", exception));
}
}
logAlert(
"Teaspoons Stairway flight {} failed with no exception given", flightState.getFlightId());
logAlert("Stairway flight {} failed with no exception given", flightState.getFlightId());
throw new InvalidResultStateException("Failed operation with no exception reported.");
}

Expand Down
2 changes: 1 addition & 1 deletion service/src/main/java/bio/terra/pipelines/service/PipelineRunsService.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -231,7 +231,7 @@ protected PipelineRun writePipelineRunToDbThrowsDuplicateException(PipelineRun p
logger.info("pipelineRun saved for jobId: {}", pipelineRun.getJobId());
} catch (DataIntegrityViolationException e) {
if (e.getCause() instanceof ConstraintViolationException c
&& c.getConstraintName().contains("jobId_unique")) {
&& c.getConstraintName().contains("pipeline_runs_jobId_unique")) {
throw new DuplicateObjectException(
String.format("Duplicate jobId %s found", pipelineRun.getJobId()));
}
Expand Down
4 changes: 2 additions & 2 deletions service/src/main/resources/application.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ env:
authorityEndpoint: ${OIDC_AUTHORITY_ENDPOINT:https://terradevb2c.b2clogin.com/terradevb2c.onmicrosoft.com/oauth2/v2.0/authorize?p=b2c_1a_signup_signin_dev}
tokenEndpoint: ${OIDC_TOKEN_ENDPOINT:https://terradevb2c.b2clogin.com/terradevb2c.onmicrosoft.com/oauth2/v2.0/token?p=b2c_1a_signup_signin_dev}
ingress:
domainName: ${TSPS_INGRESS_DOMAIN_NAME:localhost:8080}
domainName: ${TEAPSOONS_INGRESS_DOMAIN_NAME:localhost:8080}
kubernetes:
in-kubernetes: ${TERRA_COMMON_KUBERNETES_IN_KUBERNETES:false}
pod-name: ${TERRA_COMMON_KUBERNETES_POD_NAME:}
Expand Down Expand Up @@ -159,7 +159,7 @@ terra.common:

# these values are used by TCL
stairway:
cluster-name-suffix: tsps-stairway
cluster-name-suffix: teaspoons-stairway
force-clean-start: false # ${env.db.init}
max-parallel-flights: 50
migrate-upgrade: true
Expand Down
34 changes: 4 additions & 30 deletions service/src/main/resources/db/changelog.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,36 +3,10 @@
<property name="uuid_type" value="uuid" dbms="postgresql"/>
<property name="uuid_function" value="gen_random_uuid()" dbms="postgresql"/>

<include file="changesets/20221025.yaml" relativeToChangelogFile="true"/>
<include file="changesets/20221128.yaml" relativeToChangelogFile="true"/>
<include file="changesets/20230711.yaml" relativeToChangelogFile="true"/>
<include file="changesets/20240104.yaml" relativeToChangelogFile="true"/>
<include file="changesets/20240202.yaml" relativeToChangelogFile="true"/>
<include file="changesets/20240417_add_workspace_id_to_pipelines.yaml" relativeToChangelogFile="true"/>
<include file="changesets/20240412_add_pipeline_input_defs_and_beagle.yaml" relativeToChangelogFile="true"/>
<include file="changesets/20240429_add_vcf_type.yaml" relativeToChangelogFile="true"/>
<include file="changesets/20240502_add_imputation_inputs.yaml" relativeToChangelogFile="true"/>
<include file="changesets/20240520_add_outputbasename_input_beagle.yaml" relativeToChangelogFile="true"/>
<include file="changesets/20240529_convert_jobs_table_to_pipeline_runs.yaml" relativeToChangelogFile="true"/>
<include file="changesets/20240610_change_beagle_reference_path_input.yaml" relativeToChangelogFile="true"/>
<include file="changesets/20240611_add_control_workspace_id_to_pipeline_runs.yaml" relativeToChangelogFile="true"/>
<include file="changesets/20240621_add_wdl_variable_name_to_input_definitions.yaml" relativeToChangelogFile="true"/>
<include file="changesets/20240625_add_workspace_storage_url_to_pipeline_runs.yaml" relativeToChangelogFile="true"/>
<include file="changesets/20240711_add_pipeline_output_definitions.yaml" relativeToChangelogFile="true"/>
<include file="changesets/20240716_add_file_suffix_to_input_defs_and_update_FILE_types.yaml" relativeToChangelogFile="true"/>
<include file="changesets/20240806_add_workspace_project_and_workspace_name.yaml" relativeToChangelogFile="true"/>
<include file="changesets/20240820_add_workspace_storage_container_url.yaml" relativeToChangelogFile="true"/>
<include file="changesets/20240824_add_wdl_version_update_pipeline_version.yaml" relativeToChangelogFile="true"/>
<include file="changesets/20240830_add_workspace_google_project_rename_storage_container_field.yaml" relativeToChangelogFile="true"/>
<include file="changesets/20240903_rename_workspace_project.yaml" relativeToChangelogFile="true"/>
<include file="changesets/20240906_add_wdl_version_to_pipeline_run.yaml" relativeToChangelogFile="true"/>
<include file="changesets/20240911_remove_isSuccess.yaml" relativeToChangelogFile="true"/>
<include file="changesets/20240923_update_input_defs_bucket_paths.yaml" relativeToChangelogFile="true"/>
<include file="changesets/20240925_update_ref_panel_prefix_input.yaml" relativeToChangelogFile="true"/>
<include file="changesets/20241030_add_pipeline_quotas.yaml" relativeToChangelogFile="true"/>
<include file="changesets/20241103_add_user_quotas.yaml" relativeToChangelogFile="true"/>
<include file="changesets/20241106_array_imputation_rename.yaml" relativeToChangelogFile="true"/>
<include file="changesets/20241106-base-tables.yaml" relativeToChangelogFile="true"/>
<include file="changesets/20241106-base-data-insertion.yaml" relativeToChangelogFile="true"/>
<include file="changesets/20241106-add-indexes.yaml" relativeToChangelogFile="true"/>

<include file="changesets/20240412-testdata.yaml" relativeToChangelogFile="true"/>
<include file="changesets/testdata.yaml" relativeToChangelogFile="true"/>

</databaseChangeLog>
73 changes: 0 additions & 73 deletions service/src/main/resources/db/changesets/20221025.yaml
View file
Open in desktop

This file was deleted.

52 changes: 0 additions & 52 deletions service/src/main/resources/db/changesets/20221128.yaml
View file
Open in desktop

This file was deleted.

25 changes: 0 additions & 25 deletions service/src/main/resources/db/changesets/20230711.yaml
View file
Open in desktop

This file was deleted.

Loading
Loading