Skip to content

Commit

Permalink
reorganized dockerfiles
Browse files Browse the repository at this point in the history
  • Loading branch information
taddyb committed Oct 21, 2024
1 parent 4fc70fa commit 8ca681a
Show file tree
Hide file tree
Showing 7 changed files with 117 additions and 40 deletions.
42 changes: 37 additions & 5 deletions doc/api/api_docs.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ The following doc is meant to explain the T-Route FastAPI implementation using d

T-Route is used in many contexts for hydrological river routing:
- NGEN
- Scientific Python
- Scientific Python Projects
- Replace and Route (RnR)

In the latest PR for RnR (https://github.com/NOAA-OWP/hydrovis/pull/865), there is an requirement to run T-Route as a service. This service requires an easy way to dynamically create config files, restart flow from Initial Conditions, and run T-Route. To satisfy this requirement, a FastAPI endpoint was created in `/src/app` along with code to dynamically create t-route endpoints.
Expand Down Expand Up @@ -76,7 +76,7 @@ docker login --username ${GH_USERNAME} --password ${GH_TOKEN} ghcr.io
- This command will log the user into the GitHub container registry using their credentials

```shell
docker build -t ghcr.io/NOAA-OWP/t-route/t-route-api:${TAG} -f Dockerfile.troute_api
docker build -t ghcr.io/NOAA-OWP/t-route/t-route-api:${TAG} -f docker/Dockerfile.troute_api
```
- This command builds the T-Route API container using a defined version `${TAG}`

Expand All @@ -99,7 +99,39 @@ If you want to build this off a forked version, change the container registry (`

## Testing:

### Testing the RnR Extension:
### Set up Using a Dockerfile:

To build the T-Route api from a dockerfile, you can use the following commands from the t-route root repo:

```shell
docker build -t troute_api -f docker/Dockerfile.troute_api .
```

```shell
docker run -p 8000:8000 \
--env-file docker/test_troute_api.env \
-v ${OUTPUT_VOLUME_SOURCE}:${OUTPUT_VOLUME_TARGET} \
-v ${DATA_VOLUME_SOURCE}:${DATA_VOLUME_TARGET} \
-v ${CORE_VOLUME_SOURCE}:${CORE_VOLUME_TARGET} \
-v ${TEST_SOURCE}:${TEST_TARGET} troute_api
```

### Set up Docker Compose:

Docker Compose uses a YAML file to configure docker containers and execution. To install compose, you can follow the examples on docker's docs: https://docs.docker.com/compose/install/linux/

To run compose, you can use the following command from the root directory:

```shell
docker compose --env-file docker/test_troute_api.env -f docker/compose.yaml up --build
```

if you want to build a compose container to mimic what is used in RnR, you can run the following steps
```shell
docker compose --env-file docker/rnr_compose.env -f docker/compose.yaml up --build
```

#### Testing the RnR endpoint in the API:
The following folder contains data files that are to be used to test the T-Route FastAPI code within src/app

To use these files, follow the steps below:
Expand All @@ -117,8 +149,8 @@ To use these files, follow the steps below:
5. Click execute
6. A Status 201 code means the run ran, and test/api/data/troute_output will be populated in the `{lid}/` folder

### Testing the LowerColorado test cases:
#### Testing the LowerColorado test cases through docker compose:
1. Run the compose.yaml file from the base dir using: `docker compose --env-file ./compose.env up --build`
2. visit `localhost:8000/docs` in your browser
3. Execute the `/api/v1/flow_routing/v4/tests/LowerColorado` endpoint using the default parameter file path
3. Execute the `/api/v1/flow_routing/v4/tests/LowerColorado` endpoint using the default parameter file path for LowerColorado_TX_v4
4. A Status 201 code means the run ran, and the defined yaml output will be populated
8 changes: 4 additions & 4 deletions Dockerfile.troute_api → docker/Dockerfile.compose
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,8 @@ RUN yum install -y netcdf netcdf-fortran netcdf-fortran-devel netcdf-openmpi

RUN yum install -y git cmake python python-devel pip

WORKDIR "/t-route/"
COPY . "/t-route/"
WORKDIR /t-route/
COPY . /t-route/

RUN ln -s /usr/lib64/gfortran/modules/netcdf.mod /usr/include/openmpi-x86_64/netcdf.mod

Expand All @@ -16,11 +16,11 @@ RUN uv venv
ENV VIRTUAL_ENV=/t-route/.venv
ENV PATH="$VIRTUAL_ENV/bin:$PATH"

RUN uv pip install --no-cache-dir -r requirements.txt
RUN uv pip install --no-cache-dir -r /t-route/requirements.txt

RUN ./compiler.sh no-e

RUN uv pip install --no-cache-dir -r requirements-app.txt
RUN uv pip install --no-cache-dir -r /t-route/requirements-app.txt

WORKDIR "/t-route/src/"
RUN mkdir -p /t-route/data/troute_restart/
39 changes: 39 additions & 0 deletions docker/Dockerfile.troute_api
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
FROM rockylinux:9.2 as rocky-base

RUN yum install -y epel-release
RUN yum install -y netcdf netcdf-fortran netcdf-fortran-devel netcdf-openmpi

RUN yum install -y git cmake python python-devel pip

WORKDIR "/t-route/"

# Copy the contents of the parent directory (repository root) into the container
COPY . /t-route/

RUN ln -s /usr/lib64/gfortran/modules/netcdf.mod /usr/include/openmpi-x86_64/netcdf.mod

ENV PYTHONPATH=/t-route:$PYTHONPATH
RUN pip install uv==0.2.5
RUN uv venv

ENV VIRTUAL_ENV=/t-route/.venv
ENV PATH="$VIRTUAL_ENV/bin:$PATH"

RUN uv pip install --no-cache-dir -r /t-route/requirements.txt

RUN ./compiler.sh no-e

RUN uv pip install --no-cache-dir -r /t-route/requirements-app.txt

WORKDIR "/t-route/src/"
RUN mkdir -p /t-route/data/troute_restart/

# Create volume mount points
RUN mkdir -p ${OUTPUT_VOLUME_TARGET} ${DATA_VOLUME_TARGET} ${CORE_VOLUME_TARGET} /t-route/test

# Set the command to run the application
CMD sh -c ". /t-route/.venv/bin/activate && uvicorn app.main:app --host 0.0.0.0 --port ${PORT}"

# Add healthcheck
HEALTHCHECK --interval=30s --timeout=5s --start-period=5s --retries=3 \
CMD curl --fail -I http://localhost:${PORT}/health || exit 1
8 changes: 4 additions & 4 deletions compose.yaml → docker/compose.yaml
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
services:
troute:
build:
context: .
dockerfile: Dockerfile.troute_api
context: ..
dockerfile: docker/Dockerfile.compose
ports:
- "${PORT}:${PORT}"
volumes:
Expand All @@ -16,8 +16,8 @@ services:
source: ${CORE_VOLUME_SOURCE}
target: ${CORE_VOLUME_TARGET}
- type: bind
source: ./test
target: /t-route/test
source: ${TEST_SOURCE}
target: ${TEST_TARGET}
command: sh -c ". /t-route/.venv/bin/activate && uvicorn app.main:app --host 0.0.0.0 --port ${PORT}"
healthcheck:
test: curl --fail -I http://localhost:${PORT}/health || exit 1
Expand Down
11 changes: 8 additions & 3 deletions compose.env → docker/rnr_compose.env
Original file line number Diff line number Diff line change
Expand Up @@ -9,13 +9,18 @@ PORT=8000
# The following variables are used in the compose.yaml file to define the shared volume mount with T-Route

# For saving output from the container
OUTPUT_VOLUME_SOURCE=./data/troute_output
OUTPUT_VOLUME_SOURCE=../data/troute_output
OUTPUT_VOLUME_TARGET=/t-route/output

# For mounting the data directory
DATA_VOLUME_SOURCE=./data
DATA_VOLUME_SOURCE=../data
DATA_VOLUME_TARGET=/t-route/data

# For mounting all core files within T-Route (Used for sharing template config files)
CORE_VOLUME_SOURCE=./src/app/core
CORE_VOLUME_SOURCE=../src/app/core
CORE_VOLUME_TARGET=/t-route/src/app/core

# For uploading test data scripts
TEST_SOURCE=../test
TEST_TARGET=/t-route/test

25 changes: 25 additions & 0 deletions docker/test_troute_api.env
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
# Port mapping
#-------------
# The following port will be used for spinning up the API server

PORT=8000

# Volume bindings
# ---------------
# The following variables are used in the compose.yaml file to define the shared volume mount with T-Route

# For saving output from the container
OUTPUT_VOLUME_SOURCE=../test/api/data/troute_output
OUTPUT_VOLUME_TARGET=/t-route/output

# For mounting the data directory
DATA_VOLUME_SOURCE=../test/api/data
DATA_VOLUME_TARGET=/t-route/data

# For mounting all core files within T-Route (Used for sharing template config files)
CORE_VOLUME_SOURCE=../src/app/core
CORE_VOLUME_TARGET=/t-route/src/app/core

# For uploading test data scripts
TEST_SOURCE=../test
TEST_TARGET=/t-route/test
24 changes: 0 additions & 24 deletions test/api/test_compose.yaml

This file was deleted.

0 comments on commit 8ca681a

Please sign in to comment.