Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

if the folder doesn't exist #835

Merged
merged 2 commits into from
Sep 5, 2024
Merged

if the folder doesn't exist #835

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
2168267
if the folder doesn't exist
AminTorabi-NOAA Aug 30, 2024
639a160
handling edge cases
AminTorabi-NOAA Sep 5, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 10 additions & 2 deletions src/troute-config/troute/config/output_parameters.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
from pathlib import Path
from pydantic import BaseModel, Field, validator

import os
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can be dropped.

from typing import Optional, List
from typing_extensions import Annotated, Literal
from .types import FilePath, DirectoryPath
Expand Down Expand Up @@ -93,11 +93,19 @@ class ParityCheckCompareFileSet(BaseModel):

class StreamOutput(BaseModel):
# NOTE: required if writing StreamOutput files
stream_output_directory: Optional[DirectoryPath] = None
stream_output_directory: Optional[str] = None
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's still require if this exists, that it is a pathlib.Path object.

Suggested change
stream_output_directory: Optional[str] = None
stream_output_directory: Optional[Path] = None

mask_output: Optional[FilePath] = None
stream_output_time: int = 1
stream_output_type:streamOutput_allowedTypes = ".nc"
stream_output_internal_frequency: Annotated[int, Field(strict=True, ge=5)] = 5

@validator('stream_output_directory', pre=True, always=True)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few minor improvements:

  • We only need to validate after pydantic has coerced the input into a pathlib.Path (drop pre=True and always=True)
  • Handle if we got an unexpanded path (e.g. ~/output/dir)
  • Raise if we got a file
  • Create dir if it doesn't exist, and all intermediate directories
    @validator('stream_output_directory')
    def validate_stream_output_directory(cls, value):
        if value is None:
            return None
            
        # expand ~/output/dir -> /home/user/output/dir
        value = value.expanduser()
        
        if value.exists() and not value.is_dir():
            raise ValueError(f"'stream_output_directory'={value!s} is a file, expected directory.")

        # make directory (and intermediates) if they don't exist
        value.mkdir(parents=True, exist_ok=True)
        return value

def validate_stream_output_directory(cls, value):
if value:
if not os.path.exists(value):
os.makedirs(value)
return value

@validator('stream_output_internal_frequency')
def validate_stream_output_internal_frequency(cls, value, values):
if value is not None:
Expand Down
Loading