diff --git a/README.md b/README.md index 2efb09f4..a2455b4f 100644 --- a/README.md +++ b/README.md @@ -39,7 +39,7 @@
tyro.cli() is a tool for generating CLI -interfaces. +interfaces in Python. We can define configurable scripts using functions: @@ -211,9 +211,10 @@ particularly like: [jsonargparse](https://github.com/omni-us/jsonargparse), which provide deeper integration with configuration file formats like YAML and JSON. - [clipstick](https://github.com/sander76/clipstick), which focuses on - generating CLIs from Pydantic models. + simplicity + generating CLIs from Pydantic models. - [datargs](https://github.com/roee30/datargs), which provides a minimal API for dataclasses. +- [defopt](https://defopt.readthedocs.io/), which has similarly comprehensive type annotation support. - [fire](https://github.com/google/python-fire) and [clize](https://github.com/epsy/clize), which support arguments without type annotations. diff --git a/docs/requirements.txt b/docs/requirements.txt index 5e86e9dc..597a81bc 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -3,6 +3,7 @@ furo==2024.8.6 docutils==0.20.1 sphinx-autoapi==3.0.0 m2r2==0.3.3.post2 +ansi2html==1.9.2 git+https://github.com/brentyi/sphinxcontrib-programoutput.git git+https://github.com/brentyi/ansi.git sphinxcontrib-googleanalytics==0.4 diff --git a/docs/source/conf.py b/docs/source/conf.py index 1142e677..02c1a517 100644 --- a/docs/source/conf.py +++ b/docs/source/conf.py @@ -112,7 +112,7 @@ exclude_patterns: List[str] = [] # The name of the Pygments (syntax highlighting) style to use. -pygments_style = "monokai" +pygments_style = "default" # -- Options for HTML output ------------------------------------------------- diff --git a/docs/source/examples/01_basics/01_functions.rst b/docs/source/examples/01_basics/01_functions.rst deleted file mode 100644 index e5729e93..00000000 --- a/docs/source/examples/01_basics/01_functions.rst +++ /dev/null @@ -1,56 +0,0 @@ -.. Comment: this file is automatically generated by `update_example_docs.py`. - It should not be modified manually. - -Functions -========================================== - -In the simplest case, :func:`tyro.cli()` can be used to run a function with -arguments populated from the CLI. - - -.. code-block:: python - :linenos: - - - import tyro - - - def main( - field1: str, - field2: int = 3, - ) -> None: - """Function, whose arguments will be populated from a CLI interface. - - Args: - field1: A string field. - field2: A numeric field, with a default value. - """ - print(field1, field2) - - - if __name__ == "__main__": - tyro.cli(main) - ------------- - -.. raw:: html - - python 01_basics/01_functions.py --help - -.. program-output:: python ../../examples/01_basics/01_functions.py --help - ------------- - -.. raw:: html - - python 01_basics/01_functions.py --field1 hello - -.. program-output:: python ../../examples/01_basics/01_functions.py --field1 hello - ------------- - -.. raw:: html - - python 01_basics/01_functions.py --field1 hello --field2 10 - -.. program-output:: python ../../examples/01_basics/01_functions.py --field1 hello --field2 10 diff --git a/docs/source/examples/01_basics/02_dataclasses.rst b/docs/source/examples/01_basics/02_dataclasses.rst deleted file mode 100644 index 01f7acb7..00000000 --- a/docs/source/examples/01_basics/02_dataclasses.rst +++ /dev/null @@ -1,57 +0,0 @@ -.. Comment: this file is automatically generated by `update_example_docs.py`. - It should not be modified manually. - -Dataclasses -========================================== - -Common pattern: use :func:`tyro.cli()` to instantiate a dataclass. - - -.. code-block:: python - :linenos: - - - import dataclasses - - import tyro - - - @dataclasses.dataclass - class Args: - """Description. - This should show up in the helptext!""" - - field1: str - """A string field.""" - - field2: int = 3 - """A numeric field, with a default value.""" - - - if __name__ == "__main__": - args = tyro.cli(Args) - print(args) - ------------- - -.. raw:: html - - python 01_basics/02_dataclasses.py --help - -.. program-output:: python ../../examples/01_basics/02_dataclasses.py --help - ------------- - -.. raw:: html - - python 01_basics/02_dataclasses.py --field1 hello - -.. program-output:: python ../../examples/01_basics/02_dataclasses.py --field1 hello - ------------- - -.. raw:: html - - python 01_basics/02_dataclasses.py --field1 hello --field2 5 - -.. program-output:: python ../../examples/01_basics/02_dataclasses.py --field1 hello --field2 5 diff --git a/docs/source/examples/01_basics/03_dataclasses_defaults.rst b/docs/source/examples/01_basics/03_dataclasses_defaults.rst deleted file mode 100644 index 6e857f18..00000000 --- a/docs/source/examples/01_basics/03_dataclasses_defaults.rst +++ /dev/null @@ -1,81 +0,0 @@ -.. Comment: this file is automatically generated by `update_example_docs.py`. - It should not be modified manually. - -Dataclasses + Defaults -========================================== - -The :code:`default=` argument can be used to override default values in dataclass -types. - - -.. warning:: - - We advise against mutation of configuration objects from a dataclass's - :code:`__post_init__` method [#f1]_. In the example below, - :code:`__post_init__` would be called twice: once for the :code:`Args()` - object provided as a default value and another time for the :code:`Args()` - objected instantiated by :func:`tyro.cli()`. This can cause confusing - behavior! Instead, we show below one example of how derived fields can be - defined immutably. - - .. [#f1] Official Python docs for ``__post_init__`` can be found `here `_. - - -.. code-block:: python - :linenos: - - - import dataclasses - - import tyro - - - @dataclasses.dataclass - class Args: - """Description. - This should show up in the helptext!""" - - field1: str - """A string field.""" - - field2: int = 3 - """A numeric field, with a default value.""" - - @property - def derived_field(self) -> str: - return ", ".join([self.field1] * self.field2) - - - if __name__ == "__main__": - args = tyro.cli( - Args, - default=Args( - field1="default string", - field2=tyro.MISSING, - ), - ) - print(args.derived_field) - ------------- - -.. raw:: html - - python 01_basics/03_dataclasses_defaults.py --help - -.. program-output:: python ../../examples/01_basics/03_dataclasses_defaults.py --help - ------------- - -.. raw:: html - - python 01_basics/03_dataclasses_defaults.py --field2 3 - -.. program-output:: python ../../examples/01_basics/03_dataclasses_defaults.py --field2 3 - ------------- - -.. raw:: html - - python 01_basics/03_dataclasses_defaults.py --field1 hello --field2 5 - -.. program-output:: python ../../examples/01_basics/03_dataclasses_defaults.py --field1 hello --field2 5 diff --git a/docs/source/examples/01_basics/04_collections.rst b/docs/source/examples/01_basics/04_collections.rst deleted file mode 100644 index bd9cf55c..00000000 --- a/docs/source/examples/01_basics/04_collections.rst +++ /dev/null @@ -1,61 +0,0 @@ -.. Comment: this file is automatically generated by `update_example_docs.py`. - It should not be modified manually. - -Multi-value Arguments -========================================== - -Arguments of both fixed and variable lengths can be annotated with standard -Python collection types. For Python 3.7 and 3.8, we can use either :code:`from -__future__ import annotations` to support :code:`list[T]` and :code:`tuple[T]`, -or the older API :code:`typing.List[T]` and :code:`typing.Tuple[T1, T2]`. - - -.. code-block:: python - :linenos: - - - import dataclasses - import pathlib - - import tyro - - - @dataclasses.dataclass(frozen=True) - class TrainConfig: - # Example of a variable-length tuple. `list[T]`, `set[T]`, - # `dict[K, V]`, etc are supported as well. - dataset_sources: tuple[pathlib.Path, ...] - """Paths to load training data from. This can be multiple!""" - - # Fixed-length tuples are also okay. - image_dimensions: tuple[int, int] = (32, 32) - """Height and width of some image data.""" - - - if __name__ == "__main__": - config = tyro.cli(TrainConfig) - print(config) - ------------- - -.. raw:: html - - python 01_basics/04_collections.py --help - -.. program-output:: python ../../examples/01_basics/04_collections.py --help - ------------- - -.. raw:: html - - python 01_basics/04_collections.py --dataset-sources ./data --image-dimensions 16 16 - -.. program-output:: python ../../examples/01_basics/04_collections.py --dataset-sources ./data --image-dimensions 16 16 - ------------- - -.. raw:: html - - python 01_basics/04_collections.py --dataset-sources ./data - -.. program-output:: python ../../examples/01_basics/04_collections.py --dataset-sources ./data diff --git a/docs/source/examples/01_basics/05_flags.rst b/docs/source/examples/01_basics/05_flags.rst deleted file mode 100644 index 433e6af7..00000000 --- a/docs/source/examples/01_basics/05_flags.rst +++ /dev/null @@ -1,71 +0,0 @@ -.. Comment: this file is automatically generated by `update_example_docs.py`. - It should not be modified manually. - -Booleans and Flags -========================================== - -Booleans can either be expected to be explicitly passed in, or, if given a default -value, automatically converted to flags. - -To turn off conversion, see :class:`tyro.conf.FlagConversionOff`. - - -.. code-block:: python - :linenos: - - - import dataclasses - - import tyro - - - @dataclasses.dataclass - class Args: - # Boolean. This expects an explicit "True" or "False". - boolean: bool - - # Optional boolean. Same as above, but can be omitted. - optional_boolean: bool | None = None - - # Pass --flag-a in to set this value to True. - flag_a: bool = False - - # Pass --no-flag-b in to set this value to False. - flag_b: bool = True - - - if __name__ == "__main__": - args = tyro.cli(Args) - print(args) - ------------- - -.. raw:: html - - python 01_basics/05_flags.py --help - -.. program-output:: python ../../examples/01_basics/05_flags.py --help - ------------- - -.. raw:: html - - python 01_basics/05_flags.py --boolean True - -.. program-output:: python ../../examples/01_basics/05_flags.py --boolean True - ------------- - -.. raw:: html - - python 01_basics/05_flags.py --boolean False --flag-a - -.. program-output:: python ../../examples/01_basics/05_flags.py --boolean False --flag-a - ------------- - -.. raw:: html - - python 01_basics/05_flags.py --boolean False --no-flag-b - -.. program-output:: python ../../examples/01_basics/05_flags.py --boolean False --no-flag-b diff --git a/docs/source/examples/01_basics/06_literals.rst b/docs/source/examples/01_basics/06_literals.rst deleted file mode 100644 index 746e7ec6..00000000 --- a/docs/source/examples/01_basics/06_literals.rst +++ /dev/null @@ -1,40 +0,0 @@ -.. Comment: this file is automatically generated by `update_example_docs.py`. - It should not be modified manually. - -Choices -========================================== - -:code:`typing.Literal[]` can be used to restrict inputs to a fixed set of literal choices. - - -.. code-block:: python - :linenos: - - - import dataclasses - from typing import Literal - - import tyro - - - @dataclasses.dataclass(frozen=True) - class Args: - # We can use Literal[] to restrict the set of allowable inputs, for example, over - # a set of strings. - strings: Literal["red", "green"] = "red" - - # Integers also work. (as well as booleans, enums, etc) - numbers: Literal[0, 1, 2] = 0 - - - if __name__ == "__main__": - args = tyro.cli(Args) - print(args) - ------------- - -.. raw:: html - - python 01_basics/06_literals.py --help - -.. program-output:: python ../../examples/01_basics/06_literals.py --help diff --git a/docs/source/examples/01_basics/07_unions.rst b/docs/source/examples/01_basics/07_unions.rst deleted file mode 100644 index 8ce43f51..00000000 --- a/docs/source/examples/01_basics/07_unions.rst +++ /dev/null @@ -1,57 +0,0 @@ -.. Comment: this file is automatically generated by `update_example_docs.py`. - It should not be modified manually. - -Unions -========================================== - -:code:`X | Y` or :code:`typing.Union[X, Y]` can be used to expand inputs to -multiple types. - - -.. code-block:: python - :linenos: - - - import dataclasses - import enum - from typing import Literal, Optional - - import tyro - - - class Color(enum.Enum): - RED = enum.auto() - GREEN = enum.auto() - BLUE = enum.auto() - - - @dataclasses.dataclass(frozen=True) - class Args: - # Unions can be used to specify multiple allowable types. - union_over_types: int | str = 0 - string_or_enum: Literal["red", "green"] | Color = "red" - - # Unions also work over more complex nested types. - union_over_tuples: tuple[int, int] | tuple[str] = ("1",) - - # And can be nested in other types. - tuple_of_string_or_enum: tuple[Literal["red", "green"] | Color, ...] = ( - "red", - Color.RED, - ) - - # Optional[T] is equivalent to `T | None`. - integer: Optional[Literal[0, 1, 2, 3]] = None - - - if __name__ == "__main__": - args = tyro.cli(Args) - print(args) - ------------- - -.. raw:: html - - python 01_basics/07_unions.py --help - -.. program-output:: python ../../examples/01_basics/07_unions.py --help diff --git a/docs/source/examples/01_basics/08_enums.rst b/docs/source/examples/01_basics/08_enums.rst deleted file mode 100644 index 08f14296..00000000 --- a/docs/source/examples/01_basics/08_enums.rst +++ /dev/null @@ -1,62 +0,0 @@ -.. Comment: this file is automatically generated by `update_example_docs.py`. - It should not be modified manually. - -Enums -========================================== - -In addition to literals, enums can also be used to provide a fixed set of -choices. - - -.. code-block:: python - :linenos: - - - import dataclasses - import enum - - import tyro - - - class OptimizerType(enum.Enum): - ADAM = enum.auto() - SGD = enum.auto() - - - @dataclasses.dataclass(frozen=True) - class TrainConfig: - # Enums are handled seamlessly. - optimizer_type: OptimizerType = OptimizerType.ADAM - """Gradient-based optimizer to use.""" - - learning_rate: float = 1e-4 - """Learning rate for optimizer.""" - - - if __name__ == "__main__": - config = tyro.cli(TrainConfig) - print(config) - ------------- - -.. raw:: html - - python 01_basics/08_enums.py --help - -.. program-output:: python ../../examples/01_basics/08_enums.py --help - ------------- - -.. raw:: html - - python 01_basics/08_enums.py --optimizer-type SGD - -.. program-output:: python ../../examples/01_basics/08_enums.py --optimizer-type SGD - ------------- - -.. raw:: html - - python 01_basics/08_enums.py --optimizer-type ADAM --learning-rate 3e-4 - -.. program-output:: python ../../examples/01_basics/08_enums.py --optimizer-type ADAM --learning-rate 3e-4 diff --git a/docs/source/examples/02_nesting/01_nesting.rst b/docs/source/examples/02_nesting/01_nesting.rst deleted file mode 100644 index d56858ee..00000000 --- a/docs/source/examples/02_nesting/01_nesting.rst +++ /dev/null @@ -1,99 +0,0 @@ -.. Comment: this file is automatically generated by `update_example_docs.py`. - It should not be modified manually. - -Hierarchical Configs -========================================== - -Structures (typically dataclasses) can be nested to build hierarchical configuration -objects. This helps with modularity and grouping in larger projects. - - -.. code-block:: python - :linenos: - - - import dataclasses - import enum - import pathlib - - import tyro - - - class OptimizerType(enum.Enum): - ADAM = enum.auto() - SGD = enum.auto() - - - @dataclasses.dataclass - class OptimizerConfig: - # Gradient-based optimizer to use. - algorithm: OptimizerType = OptimizerType.ADAM - - # Learning rate to use. - learning_rate: float = 3e-4 - - # Coefficient for L2 regularization. - weight_decay: float = 1e-2 - - - @dataclasses.dataclass - class ExperimentConfig: - # Various configurable options for our optimizer. - optimizer: OptimizerConfig - - # Batch size. - batch_size: int = 32 - - # Total number of training steps. - train_steps: int = 100_000 - - # Random seed. This is helpful for making sure that our experiments are all - # reproducible! - seed: int = 0 - - - def train( - out_dir: pathlib.Path, - config: ExperimentConfig, - restore_checkpoint: bool = False, - checkpoint_interval: int = 1000, - ) -> None: - """Train a model. - - Args: - out_dir: Where to save logs and checkpoints. - config: Experiment configuration. - restore_checkpoint: Set to restore an existing checkpoint. - checkpoint_interval: Training steps between each checkpoint save. - """ - print(f"{out_dir=}, {restore_checkpoint=}, {checkpoint_interval=}") - print() - print(f"{config=}") - - - if __name__ == "__main__": - tyro.cli(train) - ------------- - -.. raw:: html - - python 02_nesting/01_nesting.py --help - -.. program-output:: python ../../examples/02_nesting/01_nesting.py --help - ------------- - -.. raw:: html - - python 02_nesting/01_nesting.py --out-dir . --config.optimizer.algorithm SGD - -.. program-output:: python ../../examples/02_nesting/01_nesting.py --out-dir . --config.optimizer.algorithm SGD - ------------- - -.. raw:: html - - python 02_nesting/01_nesting.py --out-dir . --restore-checkpoint - -.. program-output:: python ../../examples/02_nesting/01_nesting.py --out-dir . --restore-checkpoint diff --git a/docs/source/examples/02_nesting/02_subcommands.rst b/docs/source/examples/02_nesting/02_subcommands.rst deleted file mode 100644 index 1992a3bb..00000000 --- a/docs/source/examples/02_nesting/02_subcommands.rst +++ /dev/null @@ -1,87 +0,0 @@ -.. Comment: this file is automatically generated by `update_example_docs.py`. - It should not be modified manually. - -Subcommands -========================================== - -Unions over nested types (classes or dataclasses) are populated using subcommands. - -For configuring subcommands beyond what can be expressed with type annotations, see -:func:`tyro.conf.subcommand()`. - - -.. code-block:: python - :linenos: - - - from __future__ import annotations - - import dataclasses - - import tyro - - - @dataclasses.dataclass(frozen=True) - class Checkout: - """Checkout a branch.""" - - branch: str - - - @dataclasses.dataclass(frozen=True) - class Commit: - """Commit changes.""" - - message: str - all: bool = False - - - def main(cmd: Checkout | Commit) -> None: - print(cmd) - - - if __name__ == "__main__": - # Note that we can also pass `Checkout | Command` directly into - # `tyro.cli()`; this is understood by tyro and pyright, but unfortunately not by - # mypy. - tyro.cli(main) - ------------- - -.. raw:: html - - python 02_nesting/02_subcommands.py --help - -.. program-output:: python ../../examples/02_nesting/02_subcommands.py --help - ------------- - -.. raw:: html - - python 02_nesting/02_subcommands.py cmd:commit --help - -.. program-output:: python ../../examples/02_nesting/02_subcommands.py cmd:commit --help - ------------- - -.. raw:: html - - python 02_nesting/02_subcommands.py cmd:commit --cmd.message hello --cmd.all - -.. program-output:: python ../../examples/02_nesting/02_subcommands.py cmd:commit --cmd.message hello --cmd.all - ------------- - -.. raw:: html - - python 02_nesting/02_subcommands.py cmd:checkout --help - -.. program-output:: python ../../examples/02_nesting/02_subcommands.py cmd:checkout --help - ------------- - -.. raw:: html - - python 02_nesting/02_subcommands.py cmd:checkout --cmd.branch main - -.. program-output:: python ../../examples/02_nesting/02_subcommands.py cmd:checkout --cmd.branch main diff --git a/docs/source/examples/02_nesting/03_multiple_subcommands.rst b/docs/source/examples/02_nesting/03_multiple_subcommands.rst deleted file mode 100644 index 5064916a..00000000 --- a/docs/source/examples/02_nesting/03_multiple_subcommands.rst +++ /dev/null @@ -1,103 +0,0 @@ -.. Comment: this file is automatically generated by `update_example_docs.py`. - It should not be modified manually. - -Sequenced Subcommands -========================================== - -Multiple unions over nested types are populated using a series of subcommands. - - -.. code-block:: python - :linenos: - - - from __future__ import annotations - - import dataclasses - from typing import Literal - - import tyro - - # Possible dataset configurations. - - - @dataclasses.dataclass - class Mnist: - binary: bool = False - """Set to load binary version of MNIST dataset.""" - - - @dataclasses.dataclass - class ImageNet: - subset: Literal[50, 100, 1000] - """Choose between ImageNet-50, ImageNet-100, ImageNet-1000, etc.""" - - - # Possible optimizer configurations. - - - @dataclasses.dataclass - class Adam: - learning_rate: float = 1e-3 - betas: tuple[float, float] = (0.9, 0.999) - - - @dataclasses.dataclass - class Sgd: - learning_rate: float = 3e-4 - - - # Train script. - - - def train( - dataset: Mnist | ImageNet = Mnist(), - optimizer: Adam | Sgd = Adam(), - ) -> None: - """Example training script. - - Args: - dataset: Dataset to train on. - optimizer: Optimizer to train with. - - Returns: - None: - """ - print(dataset) - print(optimizer) - - - if __name__ == "__main__": - tyro.cli(train, config=(tyro.conf.ConsolidateSubcommandArgs,)) - ------------- - -.. raw:: html - - python 02_nesting/03_multiple_subcommands.py --help - -.. program-output:: python ../../examples/02_nesting/03_multiple_subcommands.py --help - ------------- - -.. raw:: html - - python 02_nesting/03_multiple_subcommands.py dataset:mnist --help - -.. program-output:: python ../../examples/02_nesting/03_multiple_subcommands.py dataset:mnist --help - ------------- - -.. raw:: html - - python 02_nesting/03_multiple_subcommands.py dataset:mnist optimizer:adam --help - -.. program-output:: python ../../examples/02_nesting/03_multiple_subcommands.py dataset:mnist optimizer:adam --help - ------------- - -.. raw:: html - - python 02_nesting/03_multiple_subcommands.py dataset:mnist optimizer:adam --optimizer.learning-rate 3e-4 --dataset.binary - -.. program-output:: python ../../examples/02_nesting/03_multiple_subcommands.py dataset:mnist optimizer:adam --optimizer.learning-rate 3e-4 --dataset.binary diff --git a/docs/source/examples/02_nesting/04_nesting_in_containers.rst b/docs/source/examples/02_nesting/04_nesting_in_containers.rst deleted file mode 100644 index d9d5a812..00000000 --- a/docs/source/examples/02_nesting/04_nesting_in_containers.rst +++ /dev/null @@ -1,72 +0,0 @@ -.. Comment: this file is automatically generated by `update_example_docs.py`. - It should not be modified manually. - -Nesting in Containers -========================================== - -Structures can be nested inside of standard containers. - -Note that lengths must be inferable, either via a fixed-length tuple annotation or by -parsing default values. - - -.. code-block:: python - :linenos: - - - import dataclasses - - import tyro - - - class Color: - pass - - - @dataclasses.dataclass - class RGB(Color): - r: int - g: int - b: int - - - @dataclasses.dataclass - class HSL(Color): - h: int - s: int - l: int - - - @dataclasses.dataclass - class Args: - # Example of specifying nested structures via a fixed-length tuple. - color_tuple: tuple[RGB, HSL] - - # Examples of nested structures in variable-length containers. These need a default - # provided for length inference; we don't currently support specifying dynamic - # container lengths directly from the commandline. - color_tuple_alt: tuple[Color, ...] = ( - RGB(255, 0, 0), - HSL(0, 255, 0), - ) - color_map: dict[str, RGB] = dataclasses.field( - # We can't use mutable values as defaults directly. - default_factory={ - "red": RGB(255, 0, 0), - "green": RGB(0, 255, 0), - "blue": RGB(0, 0, 255), - }.copy - ) - - - if __name__ == "__main__": - args = tyro.cli(Args) - print(args) - ------------- - -.. raw:: html - - python 02_nesting/04_nesting_in_containers.py --help - -.. program-output:: python ../../examples/02_nesting/04_nesting_in_containers.py --help diff --git a/docs/source/examples/02_nesting/05_subcommands_func.rst b/docs/source/examples/02_nesting/05_subcommands_func.rst deleted file mode 100644 index 141c9a53..00000000 --- a/docs/source/examples/02_nesting/05_subcommands_func.rst +++ /dev/null @@ -1,76 +0,0 @@ -.. Comment: this file is automatically generated by `update_example_docs.py`. - It should not be modified manually. - -Subcommands from Functions -========================================== - -We provide a shorthand for generating a subcommand CLI from a dictionary. This -is a thin wrapper around :func:`tyro.cli()`'s more verbose, type-based API. If -more generality is needed, the internal working are explained in the docs for -:func:`tyro.extras.subcommand_cli_from_dict()`. - - -.. code-block:: python - :linenos: - - - import tyro - - - def checkout(branch: str) -> None: - """Check out a branch.""" - print(f"{branch=}") - - - def commit(message: str, all: bool = False) -> None: - """Make a commit.""" - print(f"{message=} {all=}") - - - if __name__ == "__main__": - tyro.extras.subcommand_cli_from_dict( - { - "checkout": checkout, - "commit": commit, - } - ) - ------------- - -.. raw:: html - - python 02_nesting/05_subcommands_func.py --help - -.. program-output:: python ../../examples/02_nesting/05_subcommands_func.py --help - ------------- - -.. raw:: html - - python 02_nesting/05_subcommands_func.py commit --help - -.. program-output:: python ../../examples/02_nesting/05_subcommands_func.py commit --help - ------------- - -.. raw:: html - - python 02_nesting/05_subcommands_func.py commit --message hello --all - -.. program-output:: python ../../examples/02_nesting/05_subcommands_func.py commit --message hello --all - ------------- - -.. raw:: html - - python 02_nesting/05_subcommands_func.py checkout --help - -.. program-output:: python ../../examples/02_nesting/05_subcommands_func.py checkout --help - ------------- - -.. raw:: html - - python 02_nesting/05_subcommands_func.py checkout --branch main - -.. program-output:: python ../../examples/02_nesting/05_subcommands_func.py checkout --branch main diff --git a/docs/source/examples/03_config_systems/01_base_configs.rst b/docs/source/examples/03_config_systems/01_base_configs.rst deleted file mode 100644 index 0ac47f25..00000000 --- a/docs/source/examples/03_config_systems/01_base_configs.rst +++ /dev/null @@ -1,132 +0,0 @@ -.. Comment: this file is automatically generated by `update_example_docs.py`. - It should not be modified manually. - -Base Configurations -========================================== - -We can integrate `tyro` into common configuration patterns: here, we select -one of multiple possible base configurations, create a subcommand for each one, and then -use the CLI to either override (existing) or fill in (missing) values. - -The helper function used here, :func:`tyro.extras.overridable_config_cli()`, is a -lightweight wrapper over :func:`tyro.cli()`. - - -.. code-block:: python - :linenos: - - - from dataclasses import dataclass - from typing import Callable, Literal - - from torch import nn - - import tyro - - - @dataclass(frozen=True) - class AdamOptimizer: - learning_rate: float = 1e-3 - betas: tuple[float, float] = (0.9, 0.999) - - - @dataclass(frozen=True) - class ExperimentConfig: - # Dataset to run experiment on. - dataset: Literal["mnist", "imagenet-50"] - - # Optimizer parameters. - optimizer: AdamOptimizer - - # Model size. - num_layers: int - units: int - - # Batch size. - batch_size: int - - # Total number of training steps. - train_steps: int - - # Random seed. This is helpful for making sure that our experiments are all - # reproducible! - seed: int - - # Activation to use. Not specifiable via the commandline. - activation: Callable[[], nn.Module] - - - # Note that we could also define this library using separate YAML files (similar to - # `config_path`/`config_name` in Hydra), but staying in Python enables seamless type - # checking + IDE support. - default_configs = { - "small": ( - "Small experiment.", - ExperimentConfig( - dataset="mnist", - optimizer=AdamOptimizer(), - batch_size=2048, - num_layers=4, - units=64, - train_steps=30_000, - seed=0, - activation=nn.ReLU, - ), - ), - "big": ( - "Big experiment.", - ExperimentConfig( - dataset="imagenet-50", - optimizer=AdamOptimizer(), - batch_size=32, - num_layers=8, - units=256, - train_steps=100_000, - seed=0, - activation=nn.GELU, - ), - ), - } - if __name__ == "__main__": - config = tyro.extras.overridable_config_cli(default_configs) - print(config) - ------------- - -.. raw:: html - - python 03_config_systems/01_base_configs.py --help - -.. program-output:: python ../../examples/03_config_systems/01_base_configs.py --help - ------------- - -.. raw:: html - - python 03_config_systems/01_base_configs.py small --help - -.. program-output:: python ../../examples/03_config_systems/01_base_configs.py small --help - ------------- - -.. raw:: html - - python 03_config_systems/01_base_configs.py small --seed 94720 - -.. program-output:: python ../../examples/03_config_systems/01_base_configs.py small --seed 94720 - ------------- - -.. raw:: html - - python 03_config_systems/01_base_configs.py big --help - -.. program-output:: python ../../examples/03_config_systems/01_base_configs.py big --help - ------------- - -.. raw:: html - - python 03_config_systems/01_base_configs.py big --seed 94720 - -.. program-output:: python ../../examples/03_config_systems/01_base_configs.py big --seed 94720 diff --git a/docs/source/examples/03_config_systems/02_overriding_yaml.rst b/docs/source/examples/03_config_systems/02_overriding_yaml.rst deleted file mode 100644 index e926429c..00000000 --- a/docs/source/examples/03_config_systems/02_overriding_yaml.rst +++ /dev/null @@ -1,61 +0,0 @@ -.. Comment: this file is automatically generated by `update_example_docs.py`. - It should not be modified manually. - -Overriding YAML Configs -========================================== - -If you have a library of existing YAML files that you want to use, `tyro` can -be used to override values in them. We generally recommend dataclass configs -for new projects. - - -.. code-block:: python - :linenos: - - - import yaml - - import tyro - - # YAML configuration. Note that this could also be loaded from a file! Environment - # variables are an easy way to select between different YAML files. - default_yaml = r""" - exp_name: test - optimizer: - learning_rate: 0.0001 - type: adam - training: - batch_size: 32 - num_steps: 10000 - checkpoint_steps: - - 500 - - 1000 - - 1500 - """.strip() - - if __name__ == "__main__": - # Convert our YAML config into a nested dictionary. - default_config = yaml.safe_load(default_yaml) - - # Override fields in the dictionary. - overridden_config = tyro.cli(dict, default=default_config) - - # Print the overridden config. - overridden_yaml = yaml.safe_dump(overridden_config) - print(overridden_yaml) - ------------- - -.. raw:: html - - python 03_config_systems/02_overriding_yaml.py --help - -.. program-output:: python ../../examples/03_config_systems/02_overriding_yaml.py --help - ------------- - -.. raw:: html - - python 03_config_systems/02_overriding_yaml.py --training.checkpoint-steps 300 1000 9000 - -.. program-output:: python ../../examples/03_config_systems/02_overriding_yaml.py --training.checkpoint-steps 300 1000 9000 diff --git a/docs/source/examples/04_additional/01_positional_args.rst b/docs/source/examples/04_additional/01_positional_args.rst deleted file mode 100644 index b547f021..00000000 --- a/docs/source/examples/04_additional/01_positional_args.rst +++ /dev/null @@ -1,83 +0,0 @@ -.. Comment: this file is automatically generated by `update_example_docs.py`. - It should not be modified manually. - -Positional Arguments -========================================== - -Positional-only arguments in functions are converted to positional CLI arguments. - -For more general positional arguments, see :class:`tyro.conf.Positional`. - - -.. code-block:: python - :linenos: - - - from __future__ import annotations - - import dataclasses - import enum - import pathlib - from typing import Tuple - - import tyro - - - def main( - source: pathlib.Path, - dest: pathlib.Path, - /, # Mark the end of positional arguments. - optimizer: OptimizerConfig, - force: bool = False, - verbose: bool = False, - background_rgb: Tuple[float, float, float] = (1.0, 0.0, 0.0), - ) -> None: - """Command-line interface defined using a function signature. Note that this - docstring is parsed to generate helptext. - - Args: - source: Source path. - dest: Destination path. - optimizer: Configuration for our optimizer object. - force: Do not prompt before overwriting. - verbose: Explain what is being done. - background_rgb: Background color. Red by default. - """ - print(f"{source=}\n{dest=}\n{optimizer=}\n{force=}\n{verbose=}\n{background_rgb=}") - - - class OptimizerType(enum.Enum): - ADAM = enum.auto() - SGD = enum.auto() - - - @dataclasses.dataclass(frozen=True) - class OptimizerConfig: - algorithm: OptimizerType = OptimizerType.ADAM - """Gradient-based optimizer to use.""" - - learning_rate: float = 3e-4 - """Learning rate to use.""" - - weight_decay: float = 1e-2 - """Coefficient for L2 regularization.""" - - - if __name__ == "__main__": - tyro.cli(main) - ------------- - -.. raw:: html - - python 04_additional/01_positional_args.py --help - -.. program-output:: python ../../examples/04_additional/01_positional_args.py --help - ------------- - -.. raw:: html - - python 04_additional/01_positional_args.py ./a ./b --optimizer.learning-rate 1e-5 - -.. program-output:: python ../../examples/04_additional/01_positional_args.py ./a ./b --optimizer.learning-rate 1e-5 diff --git a/docs/source/examples/04_additional/02_dictionaries.rst b/docs/source/examples/04_additional/02_dictionaries.rst deleted file mode 100644 index 7d28077e..00000000 --- a/docs/source/examples/04_additional/02_dictionaries.rst +++ /dev/null @@ -1,82 +0,0 @@ -.. Comment: this file is automatically generated by `update_example_docs.py`. - It should not be modified manually. - -Dictionaries and TypedDict -========================================== - -Dictionary inputs can be specified using either a standard `Dict[K, V]` -annotation, or a :code:`TypedDict` subclass. - -For configuring :code:`TypedDict`, we also support :code:`total={True/False}`, -:code:`typing.Required`, and :code:`typing.NotRequired`. See the `Python docs `_ for all :code:`TypedDict` features. - - -.. code-block:: python - :linenos: - - - from typing import TypedDict - - from typing_extensions import NotRequired - - import tyro - - - class DictionarySchemaA( - TypedDict, - # Setting `total=False` specifies that not all keys need to exist. - total=False, - ): - learning_rate: float - betas: tuple[float, float] - - - class DictionarySchemaB(TypedDict): - learning_rate: NotRequired[float] - """NotRequired[] specifies that a particular key doesn't need to exist.""" - betas: tuple[float, float] - - - def main( - typed_dict_a: DictionarySchemaA, - typed_dict_b: DictionarySchemaB, - standard_dict: dict[str, float] = { - "learning_rate": 3e-4, - "beta1": 0.9, - "beta2": 0.999, - }, - ) -> None: - assert isinstance(typed_dict_a, dict) - assert isinstance(typed_dict_b, dict) - assert isinstance(standard_dict, dict) - print("Typed dict A:", typed_dict_a) - print("Typed dict B:", typed_dict_b) - print("Standard dict:", standard_dict) - - - if __name__ == "__main__": - tyro.cli(main) - ------------- - -.. raw:: html - - python 04_additional/02_dictionaries.py --help - -.. program-output:: python ../../examples/04_additional/02_dictionaries.py --help - ------------- - -.. raw:: html - - python 04_additional/02_dictionaries.py --typed-dict-a.learning-rate 3e-4 --typed-dict-b.betas 0.9 0.999 - -.. program-output:: python ../../examples/04_additional/02_dictionaries.py --typed-dict-a.learning-rate 3e-4 --typed-dict-b.betas 0.9 0.999 - ------------- - -.. raw:: html - - python 04_additional/02_dictionaries.py --typed-dict-b.betas 0.9 0.999 - -.. program-output:: python ../../examples/04_additional/02_dictionaries.py --typed-dict-b.betas 0.9 0.999 diff --git a/docs/source/examples/04_additional/03_tuples.rst b/docs/source/examples/04_additional/03_tuples.rst deleted file mode 100644 index cb979eda..00000000 --- a/docs/source/examples/04_additional/03_tuples.rst +++ /dev/null @@ -1,65 +0,0 @@ -.. Comment: this file is automatically generated by `update_example_docs.py`. - It should not be modified manually. - -Tuples -========================================== - -Example using :func:`tyro.cli()` to instantiate tuple types. :code:`tuple`, -:code:`typing.Tuple`, and :code:`NamedTuple` are all supported. - - -.. code-block:: python - :linenos: - - - from typing import NamedTuple - - import tyro - - - # Named tuples are interpreted as nested structures. - class Color(NamedTuple): - r: int - g: int - b: int - - - class TupleType(NamedTuple): - """Description. - This should show up in the helptext!""" - - # Tuple types can contain raw values. - color: tuple[int, int, int] = (255, 0, 0) - - # Tuple types can contain nested structures. - two_colors: tuple[Color, Color] = (Color(255, 0, 0), Color(0, 255, 0)) - - - if __name__ == "__main__": - x = tyro.cli(TupleType) - assert isinstance(x, tuple) - print(x) - ------------- - -.. raw:: html - - python 04_additional/03_tuples.py --help - -.. program-output:: python ../../examples/04_additional/03_tuples.py --help - ------------- - -.. raw:: html - - python 04_additional/03_tuples.py --color 127 127 127 - -.. program-output:: python ../../examples/04_additional/03_tuples.py --color 127 127 127 - ------------- - -.. raw:: html - - python 04_additional/03_tuples.py --two-colors.1.r 127 --two-colors.1.g 0 --two-colors.1.b 0 - -.. program-output:: python ../../examples/04_additional/03_tuples.py --two-colors.1.r 127 --two-colors.1.g 0 --two-colors.1.b 0 diff --git a/docs/source/examples/04_additional/04_classes.rst b/docs/source/examples/04_additional/04_classes.rst deleted file mode 100644 index bf6f6452..00000000 --- a/docs/source/examples/04_additional/04_classes.rst +++ /dev/null @@ -1,53 +0,0 @@ -.. Comment: this file is automatically generated by `update_example_docs.py`. - It should not be modified manually. - -Instantiating Classes -========================================== - -In addition to functions and dataclasses, we can also generate CLIs from the -constructors of standard Python classes. - - -.. code-block:: python - :linenos: - - - import tyro - - - class Args: - def __init__( - self, - field1: str, - field2: int, - flag: bool = False, - ): - """Arguments. - - Args: - field1: A string field. - field2: A numeric field. - flag: A boolean flag. - """ - self.data = [field1, field2, flag] - - - if __name__ == "__main__": - args = tyro.cli(Args) - print(args.data) - ------------- - -.. raw:: html - - python 04_additional/04_classes.py --help - -.. program-output:: python ../../examples/04_additional/04_classes.py --help - ------------- - -.. raw:: html - - python 04_additional/04_classes.py --field1 hello --field2 7 - -.. program-output:: python ../../examples/04_additional/04_classes.py --field1 hello --field2 7 diff --git a/docs/source/examples/04_additional/05_generics.rst b/docs/source/examples/04_additional/05_generics.rst deleted file mode 100644 index 6578dcd9..00000000 --- a/docs/source/examples/04_additional/05_generics.rst +++ /dev/null @@ -1,53 +0,0 @@ -.. Comment: this file is automatically generated by `update_example_docs.py`. - It should not be modified manually. - -Generic Types -========================================== - -Example of parsing for generic dataclasses. - - -.. code-block:: python - :linenos: - - - import dataclasses - from typing import Generic, TypeVar - - import tyro - - ScalarType = TypeVar("ScalarType", int, float) - ShapeType = TypeVar("ShapeType") - - - @dataclasses.dataclass(frozen=True) - class Point3(Generic[ScalarType]): - x: ScalarType - y: ScalarType - z: ScalarType - frame_id: str - - - @dataclasses.dataclass(frozen=True) - class Triangle: - a: Point3[float] - b: Point3[float] - c: Point3[float] - - - @dataclasses.dataclass(frozen=True) - class Args(Generic[ShapeType]): - shape: ShapeType - - - if __name__ == "__main__": - args = tyro.cli(Args[Triangle]) - print(args) - ------------- - -.. raw:: html - - python 04_additional/05_generics.py --help - -.. program-output:: python ../../examples/04_additional/05_generics.py --help diff --git a/docs/source/examples/04_additional/06_generics_py312.rst b/docs/source/examples/04_additional/06_generics_py312.rst deleted file mode 100644 index e075e758..00000000 --- a/docs/source/examples/04_additional/06_generics_py312.rst +++ /dev/null @@ -1,54 +0,0 @@ -.. Comment: this file is automatically generated by `update_example_docs.py`. - It should not be modified manually. - -Generic Types (Python 3.12+) -========================================== - -Example of parsing for generic dataclasses using syntax introduced in Python -3.12 (`PEP 695 `_). - -.. warning:: - - If used in conjunction with :code:`from __future__ import annotations`, the updated type parameter syntax requires Python 3.12.4 or newer. For technical details, see `this CPython PR `_. - - -.. code-block:: python - :linenos: - - - import dataclasses - - import tyro - - - @dataclasses.dataclass(frozen=True) - class Point3[ScalarType: (int, float)]: - x: ScalarType - y: ScalarType - z: ScalarType - frame_id: str - - - @dataclasses.dataclass(frozen=True) - class Triangle: - a: Point3[float] - b: Point3[float] - c: Point3[float] - - - @dataclasses.dataclass(frozen=True) - class Args[ShapeType]: - shape: ShapeType - - - if __name__ == "__main__": - args = tyro.cli(Args[Triangle]) - print(args) - ------------- - -.. raw:: html - - python 04_additional/06_generics_py312.py --help - -.. program-output:: python ../../examples/04_additional/06_generics_py312.py --help diff --git a/docs/source/examples/04_additional/07_conf.rst b/docs/source/examples/04_additional/07_conf.rst deleted file mode 100644 index 2faacdf4..00000000 --- a/docs/source/examples/04_additional/07_conf.rst +++ /dev/null @@ -1,63 +0,0 @@ -.. Comment: this file is automatically generated by `update_example_docs.py`. - It should not be modified manually. - -Configuration via typing.Annotated[] -========================================== - -The :mod:`tyro.conf` module contains utilities that can be used to configure -command-line interfaces beyond what is expressible via static type annotations. - -Features here are supported, but generally unnecessary and should be used sparingly. - - -.. code-block:: python - :linenos: - - - import dataclasses - - from typing_extensions import Annotated - - import tyro - - - @dataclasses.dataclass - class Args: - # A numeric field parsed as a positional argument. - positional: tyro.conf.Positional[int] - - # A boolean field with flag conversion turned off. - boolean: tyro.conf.FlagConversionOff[bool] = False - - # A numeric field that can't be changed via the CLI. - fixed: tyro.conf.Fixed[int] = 5 - - # A field with manually overridden properties. - manual: Annotated[ - str, - tyro.conf.arg( - name="renamed", - metavar="STRING", - help="A field with manually overridden properties!", - ), - ] = "Hello" - - - if __name__ == "__main__": - print(tyro.cli(Args)) - ------------- - -.. raw:: html - - python 04_additional/07_conf.py --help - -.. program-output:: python ../../examples/04_additional/07_conf.py --help - ------------- - -.. raw:: html - - python 04_additional/07_conf.py 5 --boolean True - -.. program-output:: python ../../examples/04_additional/07_conf.py 5 --boolean True diff --git a/docs/source/examples/04_additional/08_pydantic.rst b/docs/source/examples/04_additional/08_pydantic.rst deleted file mode 100644 index 9f529d17..00000000 --- a/docs/source/examples/04_additional/08_pydantic.rst +++ /dev/null @@ -1,54 +0,0 @@ -.. Comment: this file is automatically generated by `update_example_docs.py`. - It should not be modified manually. - -Pydantic Integration -========================================== - -In addition to standard dataclasses, :func:`tyro.cli()` also supports -`Pydantic `_ models. - - -.. code-block:: python - :linenos: - - - from pydantic import BaseModel, Field - - import tyro - - - class Args(BaseModel): - """Description. - This should show up in the helptext!""" - - field1: str - field2: int = Field(3, description="An integer field.") - - - if __name__ == "__main__": - args = tyro.cli(Args) - print(args) - ------------- - -.. raw:: html - - python 04_additional/08_pydantic.py --help - -.. program-output:: python ../../examples/04_additional/08_pydantic.py --help - ------------- - -.. raw:: html - - python 04_additional/08_pydantic.py --field1 hello - -.. program-output:: python ../../examples/04_additional/08_pydantic.py --field1 hello - ------------- - -.. raw:: html - - python 04_additional/08_pydantic.py --field1 hello --field2 5 - -.. program-output:: python ../../examples/04_additional/08_pydantic.py --field1 hello --field2 5 diff --git a/docs/source/examples/04_additional/09_attrs.rst b/docs/source/examples/04_additional/09_attrs.rst deleted file mode 100644 index e52a189f..00000000 --- a/docs/source/examples/04_additional/09_attrs.rst +++ /dev/null @@ -1,58 +0,0 @@ -.. Comment: this file is automatically generated by `update_example_docs.py`. - It should not be modified manually. - -Attrs Integration -========================================== - -In addition to standard dataclasses, :func:`tyro.cli()` also supports -`attrs `_ classes. - - -.. code-block:: python - :linenos: - - - import attr - - import tyro - - - @attr.s - class Args: - """Description. - This should show up in the helptext!""" - - field1: str = attr.ib() - """A string field.""" - - field2: int = attr.ib(factory=lambda: 5) - """A required integer field.""" - - - if __name__ == "__main__": - args = tyro.cli(Args) - print(args) - ------------- - -.. raw:: html - - python 04_additional/09_attrs.py --help - -.. program-output:: python ../../examples/04_additional/09_attrs.py --help - ------------- - -.. raw:: html - - python 04_additional/09_attrs.py --field1 hello - -.. program-output:: python ../../examples/04_additional/09_attrs.py --field1 hello - ------------- - -.. raw:: html - - python 04_additional/09_attrs.py --field1 hello --field2 5 - -.. program-output:: python ../../examples/04_additional/09_attrs.py --field1 hello --field2 5 diff --git a/docs/source/examples/04_additional/10_flax.rst b/docs/source/examples/04_additional/10_flax.rst deleted file mode 100644 index aec226d0..00000000 --- a/docs/source/examples/04_additional/10_flax.rst +++ /dev/null @@ -1,74 +0,0 @@ -.. Comment: this file is automatically generated by `update_example_docs.py`. - It should not be modified manually. - -JAX/Flax Integration -========================================== - -If you use `flax.linen `_, modules can be instantiated -directly from :func:`tyro.cli()`. - - -.. code-block:: python - :linenos: - - - from flax import linen as nn - from jax import numpy as jnp - - import tyro - - - class Classifier(nn.Module): - layers: int - """Layers in our network.""" - units: int = 32 - """Hidden unit count.""" - output_dim: int = 10 - """Number of classes.""" - - @nn.compact - def __call__(self, x: jnp.ndarray) -> jnp.ndarray: # type: ignore - for i in range(self.layers - 1): - x = nn.Dense( - self.units, - kernel_init=nn.initializers.kaiming_normal(), - )(x) - x = nn.relu(x) - - x = nn.Dense( - self.output_dim, - kernel_init=nn.initializers.xavier_normal(), - )(x) - x = nn.sigmoid(x) - return x - - - def train(model: Classifier, num_iterations: int = 1000) -> None: - """Train a model. - - Args: - model: Model to train. - num_iterations: Number of training iterations. - """ - print(f"{model=}") - print(f"{num_iterations=}") - - - if __name__ == "__main__": - tyro.cli(train) - ------------- - -.. raw:: html - - python 04_additional/10_flax.py --help - -.. program-output:: python ../../examples/04_additional/10_flax.py --help - ------------- - -.. raw:: html - - python 04_additional/10_flax.py --model.layers 4 - -.. program-output:: python ../../examples/04_additional/10_flax.py --model.layers 4 diff --git a/docs/source/examples/04_additional/11_aliases.rst b/docs/source/examples/04_additional/11_aliases.rst deleted file mode 100644 index ad728274..00000000 --- a/docs/source/examples/04_additional/11_aliases.rst +++ /dev/null @@ -1,96 +0,0 @@ -.. Comment: this file is automatically generated by `update_example_docs.py`. - It should not be modified manually. - -Argument Aliases -========================================== - -:func:`tyro.conf.arg()` can be used to attach aliases to arguments. - - -.. code-block:: python - :linenos: - - - from typing_extensions import Annotated - - import tyro - - - def checkout( - branch: Annotated[str, tyro.conf.arg(aliases=["-b"])], - ) -> None: - """Check out a branch.""" - print(f"{branch=}") - - - def commit( - message: Annotated[str, tyro.conf.arg(aliases=["-m"])], - all: Annotated[bool, tyro.conf.arg(aliases=["-a"])] = False, - ) -> None: - """Make a commit.""" - print(f"{message=} {all=}") - - - if __name__ == "__main__": - tyro.extras.subcommand_cli_from_dict( - { - "checkout": checkout, - "commit": commit, - } - ) - ------------- - -.. raw:: html - - python 04_additional/11_aliases.py --help - -.. program-output:: python ../../examples/04_additional/11_aliases.py --help - ------------- - -.. raw:: html - - python 04_additional/11_aliases.py commit --help - -.. program-output:: python ../../examples/04_additional/11_aliases.py commit --help - ------------- - -.. raw:: html - - python 04_additional/11_aliases.py commit --message hello --all - -.. program-output:: python ../../examples/04_additional/11_aliases.py commit --message hello --all - ------------- - -.. raw:: html - - python 04_additional/11_aliases.py commit -m hello -a - -.. program-output:: python ../../examples/04_additional/11_aliases.py commit -m hello -a - ------------- - -.. raw:: html - - python 04_additional/11_aliases.py checkout --help - -.. program-output:: python ../../examples/04_additional/11_aliases.py checkout --help - ------------- - -.. raw:: html - - python 04_additional/11_aliases.py checkout --branch main - -.. program-output:: python ../../examples/04_additional/11_aliases.py checkout --branch main - ------------- - -.. raw:: html - - python 04_additional/11_aliases.py checkout -b main - -.. program-output:: python ../../examples/04_additional/11_aliases.py checkout -b main diff --git a/docs/source/examples/04_additional/12_type_statement.rst b/docs/source/examples/04_additional/12_type_statement.rst deleted file mode 100644 index 63975f1e..00000000 --- a/docs/source/examples/04_additional/12_type_statement.rst +++ /dev/null @@ -1,50 +0,0 @@ -.. Comment: this file is automatically generated by `update_example_docs.py`. - It should not be modified manually. - -Type Aliases (Python 3.12+) -========================================== - -In Python 3.12, the :code:`type` statement is introduced to create type aliases. - - -.. code-block:: python - :linenos: - - - import dataclasses - - import tyro - - # Lazily-evaluated type alias. - type Field1Type = Inner - - - @dataclasses.dataclass - class Inner: - a: int - b: str - - - @dataclasses.dataclass - class Args: - """Description. - This should show up in the helptext!""" - - field1: Field1Type - """A field.""" - - field2: int = 3 - """A numeric field, with a default value.""" - - - if __name__ == "__main__": - args = tyro.cli(Args) - print(args) - ------------- - -.. raw:: html - - python 04_additional/12_type_statement.py --help - -.. program-output:: python ../../examples/04_additional/12_type_statement.py --help diff --git a/docs/source/examples/04_additional/13_counters.rst b/docs/source/examples/04_additional/13_counters.rst deleted file mode 100644 index 4068f137..00000000 --- a/docs/source/examples/04_additional/13_counters.rst +++ /dev/null @@ -1,67 +0,0 @@ -.. Comment: this file is automatically generated by `update_example_docs.py`. - It should not be modified manually. - -Counters -========================================== - -Repeatable 'counter' arguments can be specified via :data:`tyro.conf.UseCounterAction`. - - -.. code-block:: python - :linenos: - - - from typing_extensions import Annotated - - import tyro - from tyro.conf import UseCounterAction - - - def main( - verbosity: UseCounterAction[int], - aliased_verbosity: Annotated[UseCounterAction[int], tyro.conf.arg(aliases=["-v"])], - ) -> None: - """Example showing how to use counter actions. - - Args: - verbosity: Verbosity level. - aliased_verbosity: Same as above, but can also be specified with -v, -vv, -vvv, etc. - """ - print("Verbosity level:", verbosity) - print("Verbosity level (aliased):", aliased_verbosity) - - - if __name__ == "__main__": - tyro.cli(main) - ------------- - -.. raw:: html - - python 04_additional/13_counters.py --help - -.. program-output:: python ../../examples/04_additional/13_counters.py --help - ------------- - -.. raw:: html - - python 04_additional/13_counters.py --verbosity - -.. program-output:: python ../../examples/04_additional/13_counters.py --verbosity - ------------- - -.. raw:: html - - python 04_additional/13_counters.py --verbosity --verbosity - -.. program-output:: python ../../examples/04_additional/13_counters.py --verbosity --verbosity - ------------- - -.. raw:: html - - python 04_additional/13_counters.py -vvv - -.. program-output:: python ../../examples/04_additional/13_counters.py -vvv diff --git a/docs/source/examples/04_additional/14_suppress_console_outputs.rst b/docs/source/examples/04_additional/14_suppress_console_outputs.rst deleted file mode 100644 index e6056ea6..00000000 --- a/docs/source/examples/04_additional/14_suppress_console_outputs.rst +++ /dev/null @@ -1,57 +0,0 @@ -.. Comment: this file is automatically generated by `update_example_docs.py`. - It should not be modified manually. - -Cleaner Console Outputs for Scripts with Multiple Workers -========================================== - -The :code:`console_outputs=` argument can be set to :code:`False` to suppress helptext and -error message printing. - -This is useful in PyTorch for distributed training scripts, where you only want -to print the helptext from the main process: - - -.. code-block:: python - - # HuggingFace Accelerate. - args = tyro.cli(Args, console_outputs=accelerator.is_main_process) - - # PyTorch DDP. - args = tyro.cli(Args, console_outputs=(rank == 0)) - - # PyTorch Lightning. - args = tyro.cli(Args, console_outputs=trainer.is_global_zero) - - -.. code-block:: python - :linenos: - - - import dataclasses - - import tyro - - - @dataclasses.dataclass - class Args: - """Description. - This should show up in the helptext!""" - - field1: int - """A field.""" - - field2: int = 3 - """A numeric field, with a default value.""" - - - if __name__ == "__main__": - args = tyro.cli(Args, console_outputs=False) - print(args) - ------------- - -.. raw:: html - - python 04_additional/14_suppress_console_outputs.py --help - -.. program-output:: python ../../examples/04_additional/14_suppress_console_outputs.py --help diff --git a/docs/source/examples/04_additional/15_decorator_subcommands.rst b/docs/source/examples/04_additional/15_decorator_subcommands.rst deleted file mode 100644 index 1def51af..00000000 --- a/docs/source/examples/04_additional/15_decorator_subcommands.rst +++ /dev/null @@ -1,84 +0,0 @@ -.. Comment: this file is automatically generated by `update_example_docs.py`. - It should not be modified manually. - -Decorator-based Subcommands -========================================== - -:func:`tyro.extras.SubcommandApp()` provides a decorator-based API for -subcommands, which is inspired by `click `_. - - -.. code-block:: python - :linenos: - - - from tyro.extras import SubcommandApp - - app = SubcommandApp() - - - @app.command - def greet(name: str, loud: bool = False) -> None: - """Greet someone.""" - greeting = f"Hello, {name}!" - if loud: - greeting = greeting.upper() - print(greeting) - - - @app.command(name="addition") - def add(a: int, b: int) -> None: - """Add two numbers.""" - print(f"{a} + {b} = {a + b}") - - - if __name__ == "__main__": - app.cli() - ------------- - -.. raw:: html - - python 04_additional/15_decorator_subcommands.py --help - -.. program-output:: python ../../examples/04_additional/15_decorator_subcommands.py --help - ------------- - -.. raw:: html - - python 04_additional/15_decorator_subcommands.py greet --help - -.. program-output:: python ../../examples/04_additional/15_decorator_subcommands.py greet --help - ------------- - -.. raw:: html - - python 04_additional/15_decorator_subcommands.py greet --name Alice - -.. program-output:: python ../../examples/04_additional/15_decorator_subcommands.py greet --name Alice - ------------- - -.. raw:: html - - python 04_additional/15_decorator_subcommands.py greet --name Bob --loud - -.. program-output:: python ../../examples/04_additional/15_decorator_subcommands.py greet --name Bob --loud - ------------- - -.. raw:: html - - python 04_additional/15_decorator_subcommands.py addition --help - -.. program-output:: python ../../examples/04_additional/15_decorator_subcommands.py addition --help - ------------- - -.. raw:: html - - python 04_additional/15_decorator_subcommands.py addition --a 5 --b 3 - -.. program-output:: python ../../examples/04_additional/15_decorator_subcommands.py addition --a 5 --b 3 diff --git a/docs/source/examples/05_custom_constructors/01_primitive_annotation.rst b/docs/source/examples/05_custom_constructors/01_primitive_annotation.rst deleted file mode 100644 index e1e196c7..00000000 --- a/docs/source/examples/05_custom_constructors/01_primitive_annotation.rst +++ /dev/null @@ -1,71 +0,0 @@ -.. Comment: this file is automatically generated by `update_example_docs.py`. - It should not be modified manually. - -Custom Primitive -========================================== - -For additional flexibility, :mod:`tyro.constructors` exposes tyro's API for -defining behavior for different types. There are two categories of types: -primitive types can be instantiated from a single commandline argument, while -struct types are broken down into multiple. - -In this example, we attach a custom constructor via a runtime annotation. - - -.. code-block:: python - :linenos: - - - import json - - from typing_extensions import Annotated - - import tyro - - # A dictionary type, but `tyro` will expect a JSON string from the CLI. - JsonDict = Annotated[ - dict, - tyro.constructors.PrimitiveConstructorSpec( - nargs=1, - metavar="JSON", - instance_from_str=lambda args: json.loads(args[0]), - is_instance=lambda instance: isinstance(instance, dict), - str_from_instance=lambda instance: [json.dumps(instance)], - ), - ] - - - def main( - dict1: JsonDict, - dict2: JsonDict = {"default": None}, - ) -> None: - print(f"{dict1=}") - print(f"{dict2=}") - - - if __name__ == "__main__": - tyro.cli(main) - ------------- - -.. raw:: html - - python 05_custom_constructors/01_primitive_annotation.py --help - -.. program-output:: python ../../examples/05_custom_constructors/01_primitive_annotation.py --help - ------------- - -.. raw:: html - - python 05_custom_constructors/01_primitive_annotation.py --dict1 '{"hello": "world"}' - -.. program-output:: python ../../examples/05_custom_constructors/01_primitive_annotation.py --dict1 '{"hello": "world"}' - ------------- - -.. raw:: html - - python 05_custom_constructors/01_primitive_annotation.py --dict1 '{"hello": "world"}' --dict2 '{"hello": "world"}' - -.. program-output:: python ../../examples/05_custom_constructors/01_primitive_annotation.py --dict1 '{"hello": "world"}' --dict2 '{"hello": "world"}' diff --git a/docs/source/examples/05_custom_constructors/02_primitive_registry.rst b/docs/source/examples/05_custom_constructors/02_primitive_registry.rst deleted file mode 100644 index a6bd8bca..00000000 --- a/docs/source/examples/05_custom_constructors/02_primitive_registry.rst +++ /dev/null @@ -1,81 +0,0 @@ -.. Comment: this file is automatically generated by `update_example_docs.py`. - It should not be modified manually. - -Custom Primitive (Registry) -========================================== - -For additional flexibility, :mod:`tyro.constructors` exposes tyro's API for -defining behavior for different types. There are two categories of types: -primitive types can be instantiated from a single commandline argument, while -struct types are broken down into multiple. - - -In this example, we attach a custom constructor by defining a rule that applies -to all types that match ``dict[str, Any]``. - - -.. code-block:: python - :linenos: - - - import json - from typing import Any - - import tyro - - custom_registry = tyro.constructors.ConstructorRegistry() - - - @custom_registry.primitive_rule - def _( - type_info: tyro.constructors.PrimitiveTypeInfo, - ) -> tyro.constructors.PrimitiveConstructorSpec | None: - # We return `None` if the rule does not apply. - if type_info.type != dict[str, Any]: - return None - - # If the rule applies, we return the constructor spec. - return tyro.constructors.PrimitiveConstructorSpec( - nargs=1, - metavar="JSON", - instance_from_str=lambda args: json.loads(args[0]), - is_instance=lambda instance: isinstance(instance, dict), - str_from_instance=lambda instance: [json.dumps(instance)], - ) - - - def main( - dict1: dict[str, Any], - dict2: dict[str, Any] = {"default": None}, - ) -> None: - print(f"{dict1=}") - print(f"{dict2=}") - - - if __name__ == "__main__": - with custom_registry: - tyro.cli(main) - ------------- - -.. raw:: html - - python 05_custom_constructors/02_primitive_registry.py --help - -.. program-output:: python ../../examples/05_custom_constructors/02_primitive_registry.py --help - ------------- - -.. raw:: html - - python 05_custom_constructors/02_primitive_registry.py --dict1 '{"hello": "world"}' - -.. program-output:: python ../../examples/05_custom_constructors/02_primitive_registry.py --dict1 '{"hello": "world"}' - ------------- - -.. raw:: html - - python 05_custom_constructors/02_primitive_registry.py --dict1 '{"hello": "world"}' --dict2 '{"hello": "world"}' - -.. program-output:: python ../../examples/05_custom_constructors/02_primitive_registry.py --dict1 '{"hello": "world"}' --dict2 '{"hello": "world"}' diff --git a/docs/source/examples/basics.rst b/docs/source/examples/basics.rst new file mode 100644 index 00000000..ffa14321 --- /dev/null +++ b/docs/source/examples/basics.rst @@ -0,0 +1,990 @@ +.. Comment: this file is automatically generated by `update_example_docs.py`. + It should not be modified manually. + +.. _example-category-basics: + +Basics +====== + +In these examples, we show basic examples of using :func:`tyro.cli`: functions, +dataclasses, supported annotations, and configuration. + + +.. _example-01_functions: + +Functions +--------- + +In the simplest case, :func:`tyro.cli()` can be used to run a function with +arguments populated from the CLI. + + +.. code-block:: python + :linenos: + + # 01_functions.py + import tyro + + def main(field1: str, field2: int = 3) -> None: + """Function, whose arguments will be populated from a CLI interface. + + Args: + field1: A string field. + field2: A numeric field, with a default value. + """ + print(field1, field2) + + if __name__ == "__main__": + tyro.cli(main) + + +We can use ``--help`` to show the help message, or ``--field1`` and +``--field2`` to set the arguments: + +.. raw:: html + +
+    $ python ./01_functions.py --help
+    usage: 01_functions.py [-h] --field1 STR [--field2 INT]
+    
+    Function, whose arguments will be populated from a CLI interface.
+    
+    ╭─ options ───────────────────────────────────────────────────────────────╮
+    │ -h, --help          show this help message and exit                     │
+    │ --field1 STR        A string field. (required)                          │
+    │ --field2 INT        A numeric field, with a default value. (default: 3) │
+    ╰─────────────────────────────────────────────────────────────────────────╯
+    
+ + + +.. raw:: html + +
+    $ python ./01_functions.py --field1 hello
+    hello 3
+    
+ + + +.. raw:: html + +
+    $ python ./01_functions.py --field1 hello --field2 10
+    hello 10
+    
+.. _example-02_dataclasses: + +Dataclasses +----------- + +In addition to functions, :func:`tyro.cli()` can also take dataclasses as input. + + +.. code-block:: python + :linenos: + + # 02_dataclasses.py + from dataclasses import dataclass + from pprint import pprint + + import tyro + + @dataclass + class Args: + """Description. + This should show up in the helptext!""" + + field1: str + """A string field.""" + + field2: int = 3 + """A numeric field, with a default value.""" + + if __name__ == "__main__": + args = tyro.cli(Args) + pprint(args) + + +To show the help message, we can use the ``--help`` flag: + +.. raw:: html + +
+    $ python ./02_dataclasses.py --help
+    usage: 02_dataclasses.py [-h] --field1 STR [--field2 INT]
+    
+    Description. This should show up in the helptext!
+    
+    ╭─ options ───────────────────────────────────────────────────────────────╮
+    │ -h, --help          show this help message and exit                     │
+    │ --field1 STR        A string field. (required)                          │
+    │ --field2 INT        A numeric field, with a default value. (default: 3) │
+    ╰─────────────────────────────────────────────────────────────────────────╯
+    
+ +We can override ``field1`` and ``field2``: + +.. raw:: html + +
+    $ python ./02_dataclasses.py --field1 hello
+    Args(field1='hello', field2=3)
+    
+ + + +.. raw:: html + +
+    $ python ./02_dataclasses.py --field1 hello --field2 5
+    Args(field1='hello', field2=5)
+    
+.. _example-03_multivalue: + +Multi-value Arguments +--------------------- + +Arguments of both fixed and variable lengths can be annotated with standard +Python collection types. For Python 3.7 and 3.8, we can use either ``from +__future__ import annotations`` to support ``list[T]`` and ``tuple[T]``, +or the older :py:class:`typing.List` and :py:data:`typing.Tuple`. + + +.. code-block:: python + :linenos: + + # 03_multivalue.py + import pathlib + from dataclasses import dataclass + from pprint import pprint + + import tyro + + @dataclass + class Config: + # Example of a variable-length tuple. `list[T]`, `set[T]`, + # `dict[K, V]`, etc are supported as well. + source_paths: tuple[pathlib.Path, ...] + """This can be multiple!""" + + # Fixed-length tuples are also okay. + dimensions: tuple[int, int] = (32, 32) + """Height and width.""" + + if __name__ == "__main__": + config = tyro.cli(Config) + pprint(config) + + +To print help: + +.. raw:: html + +
+    $ python ./03_multivalue.py --help
+    usage: 03_multivalue.py [-h] --source-paths [PATH
+                            [PATH ...]] [--dimensions INT INT]
+    
+    ╭─ options ──────────────────────────────────────────────────╮
+    │ -h, --help              show this help message and exit    │
+    │ --source-paths [PATH [PATH ...]]                           │
+    │                         This can be multiple! (required)   │
+    │ --dimensions INT INT    Height and width. (default: 32 32) │
+    ╰────────────────────────────────────────────────────────────╯
+    
+ +We can override arguments: + +.. raw:: html + +
+    $ python ./03_multivalue.py --source-paths ./data --dimensions 16 16
+    Config(source_paths=(PosixPath('data'),), dimensions=(16, 16))
+    
+ + + +.. raw:: html + +
+    $ python ./03_multivalue.py --source-paths ./data1 ./data2
+    Config(source_paths=(PosixPath('data1'), PosixPath('data2')),
+           dimensions=(32, 32))
+    
+.. _example-04_classes: + +Instantiating Classes +--------------------- + +In addition to functions and dataclasses, we can also generate CLIs from the +constructors of standard Python classes. + + +.. code-block:: python + :linenos: + + # 04_classes.py + import tyro + + class Args: + def __init__( + self, + field1: str, + field2: int, + flag: bool = False, + ): + """Arguments. + + Args: + field1: A string field. + field2: A numeric field. + flag: A boolean flag. + """ + self.data = [field1, field2, flag] + + if __name__ == "__main__": + args = tyro.cli(Args) + print(args.data) + + + + +.. raw:: html + +
+    $ python ./04_classes.py --help
+    usage: 04_classes.py [-h] --field1 STR --field2 INT [--flag | --no-flag]
+    
+    Arguments.
+    
+    ╭─ options ───────────────────────────────────────────────╮
+    │ -h, --help              show this help message and exit │
+    │ --field1 STR            A string field. (required)      │
+    │ --field2 INT            A numeric field. (required)     │
+    │ --flag, --no-flag       A boolean flag. (default: None) │
+    ╰─────────────────────────────────────────────────────────╯
+    
+ + + +.. raw:: html + +
+    $ python ./04_classes.py --field1 hello --field2 7
+    ['hello', 7, False]
+    
+.. _example-04_flags: + +Booleans and Flags +------------------ + +Booleans can either be expected to be explicitly passed in, or, if given a default +value, automatically converted to flags. + +To turn off conversion, see :class:`tyro.conf.FlagConversionOff`. + + +.. code-block:: python + :linenos: + + # 04_flags.py + from dataclasses import dataclass + from pprint import pprint + + import tyro + + @dataclass + class Args: + # Boolean. This expects an explicit "True" or "False". + boolean: bool + + # Optional boolean. Same as above, but can be omitted. + optional_boolean: bool | None = None + + # Pass --flag-a in to set this value to True. + flag_a: bool = False + + # Pass --no-flag-b in to set this value to False. + flag_b: bool = True + + if __name__ == "__main__": + args = tyro.cli(Args) + pprint(args) + + + + +.. raw:: html + +
+    $ python ./04_flags.py --help
+    usage: 04_flags.py [-h] [OPTIONS]
+    
+    ╭─ options ────────────────────────────────────────────────────────────────╮
+    │ -h, --help                                                               │
+    │     show this help message and exit                                      │
+    │ --boolean {True,False}                                                   │
+    │     Boolean. This expects an explicit "True" or "False". (required)      │
+    │ --optional-boolean {None,True,False}                                     │
+    │     Optional boolean. Same as above, but can be omitted. (default: None) │
+    │ --flag-a, --no-flag-a                                                    │
+    │     Pass --flag-a in to set this value to True. (default: None)          │
+    │ --flag-b, --no-flag-b                                                    │
+    │     Pass --no-flag-b in to set this value to False. (default: None)      │
+    ╰──────────────────────────────────────────────────────────────────────────╯
+    
+ + + +.. raw:: html + +
+    $ python ./04_flags.py --boolean True
+    Args(boolean=True, optional_boolean=None, flag_a=False, flag_b=True)
+    
+ + + +.. raw:: html + +
+    $ python ./04_flags.py --boolean False --flag-a
+    Args(boolean=False, optional_boolean=None, flag_a=True, flag_b=True)
+    
+ + + +.. raw:: html + +
+    $ python ./04_flags.py --boolean False --no-flag-b
+    Args(boolean=False, optional_boolean=None, flag_a=False, flag_b=False)
+    
+.. _example-05_choices: + +Choices +------- + +:code:`typing.Literal[]` can be used to restrict inputs to a fixed set of literal choices. + + +.. code-block:: python + :linenos: + + # 05_choices.py + import dataclasses + from pprint import pprint + from typing import Literal + + import tyro + + @dataclasses.dataclass + class Args: + # We can use Literal[] to restrict the set of allowable inputs, for example, over + # a set of strings. + string: Literal["red", "green"] = "red" + + # Integers also work. (as well as booleans, enums, etc) + number: Literal[0, 1, 2] = 0 + + if __name__ == "__main__": + args = tyro.cli(Args) + pprint(args) + + + + +.. raw:: html + +
+    $ python ./05_choices.py --help
+    usage: 05_choices.py [-h] [--string {red,green}] [--number {0,1,2}]
+    
+    ╭─ options ──────────────────────────────────────────────────────────────────╮
+    │ -h, --help              show this help message and exit                    │
+    │ --string {red,green}    We can use Literal[] to restrict the set of        │
+    │                         allowable inputs, for example, over a set of       │
+    │                         strings. (default: red)                            │
+    │ --number {0,1,2}        Integers also work. (as well as booleans, enums,   │
+    │                         etc) (default: 0)                                  │
+    ╰────────────────────────────────────────────────────────────────────────────╯
+    
+ + + +.. raw:: html + +
+    $ python ./05_choices.py --string red
+    Args(string='red', number=0)
+    
+ + + +.. raw:: html + +
+    $ python ./05_choices.py --string blue
+    ╭─ Parsing error ────────────────────────────────────────────────────────╮
+    │ Argument --string: invalid choice: 'blue' (choose from 'red', 'green') │
+    │ ────────────────────────────────────────────────────────────────────── │
+    │ For full helptext, run 05_choices.py --help                            │
+    ╰────────────────────────────────────────────────────────────────────────╯
+    
+.. _example-06_enums: + +Enums +----- + +In addition to literals, enums can also be used to provide a fixed set of +choices. + + +.. code-block:: python + :linenos: + + # 06_enums.py + import enum + from dataclasses import dataclass + from pprint import pprint + + import tyro + + class Color(enum.Enum): + RED = enum.auto() + BLUE = enum.auto() + + @dataclass + class Config: + color: Color = Color.RED + """Color argument.""" + + opacity: float = 0.5 + """Opacity argument.""" + + if __name__ == "__main__": + config = tyro.cli(Config) + pprint(config) + + + + +.. raw:: html + +
+    $ python ./06_enums.py --help
+    usage: 06_enums.py [-h] [--color {RED,BLUE}] [--opacity FLOAT]
+    
+    ╭─ options ────────────────────────────────────────────────╮
+    │ -h, --help              show this help message and exit  │
+    │ --color {RED,BLUE}      Color argument. (default: RED)   │
+    │ --opacity FLOAT         Opacity argument. (default: 0.5) │
+    ╰──────────────────────────────────────────────────────────╯
+    
+ + + +.. raw:: html + +
+    $ python ./06_enums.py --color RED
+    Config(color=<Color.RED: 1>, opacity=0.5)
+    
+ + + +.. raw:: html + +
+    $ python ./06_enums.py --color BLUE --opacity 0.75
+    Config(color=<Color.BLUE: 2>, opacity=0.75)
+    
+.. _example-07_unions: + +Unions +------ + +:code:`X | Y` or :code:`typing.Union[X, Y]` can be used to expand inputs to +multiple types. + + +.. code-block:: python + :linenos: + + # 07_unions.py + import dataclasses + import enum + from pprint import pprint + from typing import Literal, Optional + + import tyro + + class Color(enum.Enum): + RED = enum.auto() + GREEN = enum.auto() + BLUE = enum.auto() + + @dataclasses.dataclass(frozen=True) + class Args: + # Unions can be used to specify multiple allowable types. + union_over_types: int | str = 0 + string_or_enum: Literal["red", "green"] | Color = "red" + + # Unions also work over more complex nested types. + union_over_tuples: tuple[int, int] | tuple[str] = ("1",) + + # And can be nested in other types. + tuple_of_string_or_enum: tuple[Literal["red", "green"] | Color, ...] = ( + "red", + Color.RED, + ) + + # Optional[T] is equivalent to `T | None`. + integer: Optional[Literal[0, 1, 2, 3]] = None + + if __name__ == "__main__": + args = tyro.cli(Args) + pprint(args) + + + + +.. raw:: html + +
+    $ python ./07_unions.py --help
+    usage: 07_unions.py [-h] [OPTIONS]
+    
+    ╭─ options ──────────────────────────────────────────────────────────────────╮
+    │ -h, --help                                                                 │
+    │     show this help message and exit                                        │
+    │ --union-over-types INT|STR                                                 │
+    │     Unions can be used to specify multiple allowable types. (default: 0)   │
+    │ --string-or-enum {red,green,RED,GREEN,BLUE}                                │
+    │     Unions can be used to specify multiple allowable types. (default: red) │
+    │ --union-over-tuples {INT INT}|STR                                          │
+    │     Unions also work over more complex nested types. (default: 1)          │
+    │ --tuple-of-string-or-enum [{red,green,RED,GREEN,BLUE}                      │
+    │ [{red,green,RED,GREEN,BLUE} ...]]                                          │
+    │     And can be nested in other types. (default: red RED)                   │
+    │ --integer {None,0,1,2,3}                                                   │
+    │     Optional[T] is equivalent to `T | None`. (default: None)               │
+    ╰────────────────────────────────────────────────────────────────────────────╯
+    
+ + + +.. raw:: html + +
+    $ python ./07_unions.py --union-over-types 3
+    Args(union_over_types=3,
+         string_or_enum='red',
+         union_over_tuples=('1',),
+         tuple_of_string_or_enum=('red', <Color.RED: 1>),
+         integer=None)
+    
+ + + +.. raw:: html + +
+    $ python ./07_unions.py --union-over-types three
+    Args(union_over_types='three',
+         string_or_enum='red',
+         union_over_tuples=('1',),
+         tuple_of_string_or_enum=('red', <Color.RED: 1>),
+         integer=None)
+    
+ + + +.. raw:: html + +
+    $ python ./07_unions.py --integer None
+    Args(union_over_types=0,
+         string_or_enum='red',
+         union_over_tuples=('1',),
+         tuple_of_string_or_enum=('red', <Color.RED: 1>),
+         integer=None)
+    
+ + + +.. raw:: html + +
+    $ python ./07_unions.py --integer 0
+    Args(union_over_types=0,
+         string_or_enum='red',
+         union_over_tuples=('1',),
+         tuple_of_string_or_enum=('red', <Color.RED: 1>),
+         integer=0)
+    
+.. _example-08_positional: + +Positional Arguments +-------------------- + +Positional-only arguments in functions are converted to positional CLI arguments. + +For more general positional arguments, see :class:`tyro.conf.Positional`. + + +.. code-block:: python + :linenos: + + # 08_positional.py + from __future__ import annotations + + import pathlib + + import tyro + + def main( + source: pathlib.Path, + dest: pathlib.Path, + /, # Mark the end of positional arguments. + verbose: bool = False, + ) -> None: + """Command-line interface defined using a function signature. Note that this + docstring is parsed to generate helptext. + + Args: + source: Source path. + dest: Destination path. + verbose: Explain what is being done. + """ + print(f"{source=}\n{dest=}\n{verbose=}") + + if __name__ == "__main__": + tyro.cli(main) + + + + +.. raw:: html + +
+    $ python 08_positional.py --help
+    usage: 08_positional.py [-h] [--verbose | --no-verbose] PATH PATH
+    
+    Command-line interface defined using a function signature. Note that this 
+    docstring is parsed to generate helptext.
+    
+    ╭─ positional arguments ────────────────────────────────────────╮
+    │ PATH              Source path. (required)                     │
+    │ PATH              Destination path. (required)                │
+    ╰───────────────────────────────────────────────────────────────╯
+    ╭─ options ─────────────────────────────────────────────────────╮
+    │ -h, --help        show this help message and exit             │
+    │ --verbose, --no-verbose                                       │
+    │                   Explain what is being done. (default: None) │
+    ╰───────────────────────────────────────────────────────────────╯
+    
+ + + +.. raw:: html + +
+    $ python 08_positional.py ./a ./b
+    source=PosixPath('a')
+    dest=PosixPath('b')
+    verbose=False
+    
+ + + +.. raw:: html + +
+    $ python 08_positional.py ./test1 ./test2 --verbose
+    source=PosixPath('test1')
+    dest=PosixPath('test2')
+    verbose=True
+    
+.. _example-09_conf: + +Configuration via typing.Annotated[] +------------------------------------ + +The :mod:`tyro.conf` module contains utilities that can be used in conjunction +with :py:data:`typing.Annotated` to configure command-line interfaces beyond +what is expressible via static type annotations. + +Features here are supported, but generally unnecessary and should be used sparingly. + + +.. code-block:: python + :linenos: + + # 09_conf.py + import dataclasses + + from typing_extensions import Annotated + + import tyro + + @dataclasses.dataclass + class Args: + # A numeric field parsed as a positional argument. + positional: tyro.conf.Positional[int] + + # A boolean field with flag conversion turned off. + boolean: tyro.conf.FlagConversionOff[bool] = False + + # A numeric field that can't be changed via the CLI. + fixed: tyro.conf.Fixed[int] = 5 + + # A field with manually overridden properties. + manual: Annotated[ + str, + tyro.conf.arg( + name="renamed", + metavar="STRING", + help="A field with manually overridden properties!", + ), + ] = "Hello" + + if __name__ == "__main__": + print(tyro.cli(Args)) + + + + +.. raw:: html + +
+    $ python ./09_conf.py --help
+    usage: 09_conf.py [-h] [OPTIONS] INT
+    
+    ╭─ positional arguments ─────────────────────────────────────────────────────╮
+    │ INT                     A numeric field parsed as a positional argument.   │
+    │                         (required)                                         │
+    ╰────────────────────────────────────────────────────────────────────────────╯
+    ╭─ options ──────────────────────────────────────────────────────────────────╮
+    │ -h, --help              show this help message and exit                    │
+    │ --boolean {True,False}  A boolean field with flag conversion turned off.   │
+    │                         (default: False)                                   │
+    │ --fixed {fixed}         A numeric field that can't be changed via the CLI. │
+    │                         (fixed to: 5)                                      │
+    │ --renamed STRING        A field with manually overridden properties!       │
+    │                         (default: Hello)                                   │
+    ╰────────────────────────────────────────────────────────────────────────────╯
+    
+ + + +.. raw:: html + +
+    $ python ./09_conf.py 5 --boolean True
+    Args(positional=5, boolean=True, fixed=5, manual='Hello')
+    
+.. _example-10_aliases: + +Argument Aliases +---------------- + +:func:`tyro.conf.arg()` can be used to attach aliases to arguments. + + +.. code-block:: python + :linenos: + + # 10_aliases.py + from typing import Annotated + + import tyro + + def checkout( + branch: Annotated[str, tyro.conf.arg(aliases=["-b"])], + ) -> None: + """Check out a branch.""" + print(f"{branch=}") + + if __name__ == "__main__": + tyro.cli(checkout) + + + + +.. raw:: html + +
+    $ python ./10_aliases.py --help
+    usage: 10_aliases.py [-h] --branch STR
+    
+    Check out a branch.
+    
+    ╭─ options ───────────────────────────────────────────────╮
+    │ -h, --help              show this help message and exit │
+    │ --branch STR, -b STR    (required)                      │
+    ╰─────────────────────────────────────────────────────────╯
+    
+ + + +.. raw:: html + +
+    $ python ./10_aliases.py --branch main
+    branch='main'
+    
+ + + +.. raw:: html + +
+    $ python ./10_aliases.py -b main
+    branch='main'
+    
+.. _example-11_type_aliases_py312: + +Type Aliases (3.12+) +-------------------- + +In Python 3.12, the :code:`type` statement is introduced to create type aliases. + + +.. code-block:: python + :linenos: + + # 11_type_aliases_py312.py + import dataclasses + + import tyro + + # Lazily-evaluated type alias. + type Field1Type = Inner + + @dataclasses.dataclass + class Inner: + a: int + b: str + + @dataclasses.dataclass + class Args: + """Description. + This should show up in the helptext!""" + + field1: Field1Type + """A field.""" + + field2: int = 3 + """A numeric field, with a default value.""" + + if __name__ == "__main__": + args = tyro.cli(Args) + print(args) + + + + +.. raw:: html + +
+    $ python ./11_type_aliases_py312.py --help
+    usage: 11_type_aliases_py312.py [-h] [--field2 INT] --field1.a INT --field1.b
+                                    STR
+    
+    Description. This should show up in the helptext!
+    
+    ╭─ options ─────────────────────────────────────────────────────────────────╮
+    │ -h, --help            show this help message and exit                     │
+    │ --field2 INT          A numeric field, with a default value. (default: 3) │
+    ╰───────────────────────────────────────────────────────────────────────────╯
+    ╭─ field1 options ──────────────────────────────────────────────────────────╮
+    │ A field.                                                                  │
+    │ ────────────────────────────────                                          │
+    │ --field1.a INT        (required)                                          │
+    │ --field1.b STR        (required)                                          │
+    ╰───────────────────────────────────────────────────────────────────────────╯
+    
+.. _example-12_counters: + +Counters +-------- + +Repeatable 'counter' arguments can be specified via :data:`tyro.conf.UseCounterAction`. + + +.. code-block:: python + :linenos: + + # 12_counters.py + from typing_extensions import Annotated + + import tyro + from tyro.conf import UseCounterAction + + def main( + verbosity: UseCounterAction[int], + aliased_verbosity: Annotated[UseCounterAction[int], tyro.conf.arg(aliases=["-v"])], + ) -> None: + """Example showing how to use counter actions. + + Args: + verbosity: Verbosity level. + aliased_verbosity: Same as above, but can also be specified with -v, -vv, -vvv, etc. + """ + print("Verbosity level:", verbosity) + print("Verbosity level (aliased):", aliased_verbosity) + + if __name__ == "__main__": + tyro.cli(main) + + + + +.. raw:: html + +
+    $ python ./12_counters.py --help
+    usage: 12_counters.py [-h] [--verbosity] [--aliased-verbosity]
+    
+    Example showing how to use counter actions.
+    
+    ╭─ options ──────────────────────────────────────────────────────────────────╮
+    │ -h, --help         show this help message and exit                         │
+    │ --verbosity        Verbosity level. (repeatable)                           │
+    │ --aliased-verbosity, -v                                                    │
+    │                    Same as above, but can also be specified with -v, -vv,  │
+    │                    -vvv, etc. (repeatable)                                 │
+    ╰────────────────────────────────────────────────────────────────────────────╯
+    
+ + + +.. raw:: html + +
+    $ python ./12_counters.py --verbosity
+    Verbosity level: 1
+    Verbosity level (aliased): 0
+    
+ + + +.. raw:: html + +
+    $ python ./12_counters.py --verbosity --verbosity
+    Verbosity level: 2
+    Verbosity level (aliased): 0
+    
+ + + +.. raw:: html + +
+    $ python ./12_counters.py -vvv
+    Verbosity level: 0
+    Verbosity level (aliased): 3
+    
\ No newline at end of file diff --git a/docs/source/examples/custom_constructors.rst b/docs/source/examples/custom_constructors.rst new file mode 100644 index 00000000..070cba69 --- /dev/null +++ b/docs/source/examples/custom_constructors.rst @@ -0,0 +1,190 @@ +.. Comment: this file is automatically generated by `update_example_docs.py`. + It should not be modified manually. + +.. _example-category-custom_constructors: + +Custom constructors +=================== + +In these examples, we show how custom types can be parsed by and registered with :func:`tyro.cli`. + + +.. _example-01_primitive_annotation: + +Custom Primitive +---------------- + +.. note:: + + This is an advanced feature, which should not be needed for the vast + majority of use cases. If :mod:`tyro` is missing support for a built-in + Python type, please open an issue on `GitHub `_. + +For additional flexibility, :mod:`tyro.constructors` exposes tyro's API for +defining behavior for different types. There are two categories of types: +primitive types can be instantiated from a single commandline argument, while +struct types are broken down into multiple. + +In this example, we attach a custom constructor via a runtime annotation. + + +.. code-block:: python + :linenos: + + # 01_primitive_annotation.py + import json + + from typing_extensions import Annotated + + import tyro + + # A dictionary type, but `tyro` will expect a JSON string from the CLI. + JsonDict = Annotated[ + dict, + tyro.constructors.PrimitiveConstructorSpec( + # Number of arguments to consume. + nargs=1, + # Argument name in usage messages. + metavar="JSON", + # Convert a list of strings to an instance. The length of the list + # should match `nargs`. + instance_from_str=lambda args: json.loads(args[0]), + # Check if an instance is of the expected type. This is only used for + # helptext formatting in the presence of union types. + is_instance=lambda instance: isinstance(instance, dict), + # Convert an instance to a list of strings. This is used for handling + # default values that are set in Python. The length of the list should + # match `nargs`. + str_from_instance=lambda instance: [json.dumps(instance)], + ), + ] + + def main( + dict1: JsonDict, + dict2: JsonDict = {"default": None}, + ) -> None: + print(f"{dict1=}") + print(f"{dict2=}") + + if __name__ == "__main__": + tyro.cli(main) + + + + +.. raw:: html + +
+    $ python ./01_primitive_annotation.py --help
+    usage: 01_primitive_annotation.py [-h] --dict1 JSON [--dict2 JSON]
+    
+    ╭─ options ───────────────────────────────────────────╮
+    │ -h, --help          show this help message and exit │
+    │ --dict1 JSON        (required)                      │
+    │ --dict2 JSON        (default: '{"default": null}')  │
+    ╰─────────────────────────────────────────────────────╯
+    
+ + + +.. raw:: html + +
+    $ python ./01_primitive_annotation.py --dict1 '{"hello": "world"}'
+    dict1={'hello': 'world'}
+    dict2={'default': None}
+    
+ + + +.. raw:: html + +
+    $ python ./01_primitive_annotation.py --dict1 '{"hello": "world"}' --dict2 '{"hello": "world"}'
+    dict1={'hello': 'world'}
+    dict2={'hello': 'world'}
+    
+.. _example-02_primitive_registry: + +Custom Primitive (Registry) +--------------------------- + +In this example, we use :class:`tyro.constructors.PrimitiveConstructorSpec` to +define a rule that applies to all types that match ``dict[str, Any]``. + + +.. code-block:: python + :linenos: + + # 02_primitive_registry.py + import json + from typing import Any + + import tyro + + custom_registry = tyro.constructors.ConstructorRegistry() + + @custom_registry.primitive_rule + def _( + type_info: tyro.constructors.PrimitiveTypeInfo, + ) -> tyro.constructors.PrimitiveConstructorSpec | None: + # We return `None` if the rule does not apply. + if type_info.type != dict[str, Any]: + return None + + # If the rule applies, we return the constructor spec. + return tyro.constructors.PrimitiveConstructorSpec( + nargs=1, + metavar="JSON", + instance_from_str=lambda args: json.loads(args[0]), + is_instance=lambda instance: isinstance(instance, dict), + str_from_instance=lambda instance: [json.dumps(instance)], + ) + + def main( + dict1: dict[str, Any], + dict2: dict[str, Any] = {"default": None}, + ) -> None: + print(f"{dict1=}") + print(f"{dict2=}") + + if __name__ == "__main__": + # The custom registry is used as a context. + with custom_registry: + tyro.cli(main) + + + + +.. raw:: html + +
+    $ python ./02_primitive_registry.py --help
+    usage: 02_primitive_registry.py [-h] --dict1 JSON [--dict2 JSON]
+    
+    ╭─ options ───────────────────────────────────────────╮
+    │ -h, --help          show this help message and exit │
+    │ --dict1 JSON        (required)                      │
+    │ --dict2 JSON        (default: '{"default": null}')  │
+    ╰─────────────────────────────────────────────────────╯
+    
+ + + +.. raw:: html + +
+    $ python ./02_primitive_registry.py --dict1 '{"hello": "world"}'
+    dict1={'hello': 'world'}
+    dict2={'default': None}
+    
+ + + +.. raw:: html + +
+    $ python ./02_primitive_registry.py --dict1 '{"hello": "world"}' --dict2 '{"hello": "world"}'
+    dict1={'hello': 'world'}
+    dict2={'hello': 'world'}
+    
\ No newline at end of file diff --git a/docs/source/examples/generics.rst b/docs/source/examples/generics.rst new file mode 100644 index 00000000..22163858 --- /dev/null +++ b/docs/source/examples/generics.rst @@ -0,0 +1,157 @@ +.. Comment: this file is automatically generated by `update_example_docs.py`. + It should not be modified manually. + +.. _example-category-generics: + +Generics +======== + +:mod:`tyro`'s understanding of Python's type system includes user-defined +parameterized types, which can reduce boilerplate and improve type safety. + + +.. _example-01_generics_py312: + +Generics (3.12+) +---------------- + +This example uses syntax introduced in Python 3.12 (`PEP 695 `_). + +.. note:: + + If used in conjunction with :code:`from __future__ import annotations`, the updated type parameter syntax requires Python 3.12.4 or newer. For technical details, see `this CPython PR `_. + + +.. code-block:: python + :linenos: + + # 01_generics_py312.py + import dataclasses + + import tyro + + @dataclasses.dataclass + class Point3[ScalarType: (int, float)]: + x: ScalarType + y: ScalarType + z: ScalarType + frame_id: str + + @dataclasses.dataclass + class Triangle: + a: Point3[float] + b: Point3[float] + c: Point3[float] + + @dataclasses.dataclass + class Args[ShapeType]: + shape: ShapeType + + if __name__ == "__main__": + args = tyro.cli(Args[Triangle]) + print(args) + + + + +.. raw:: html + +
+    $ python ./01_generics_py312.py --help
+    usage: 01_generics_py312.py [-h] [OPTIONS]
+    
+    ╭─ options ───────────────────────────────────────────────╮
+    │ -h, --help              show this help message and exit │
+    ╰─────────────────────────────────────────────────────────╯
+    ╭─ shape.a options ───────────────────────────────────────╮
+    │ --shape.a.x FLOAT       (required)                      │
+    │ --shape.a.y FLOAT       (required)                      │
+    │ --shape.a.z FLOAT       (required)                      │
+    │ --shape.a.frame-id STR  (required)                      │
+    ╰─────────────────────────────────────────────────────────╯
+    ╭─ shape.b options ───────────────────────────────────────╮
+    │ --shape.b.x FLOAT       (required)                      │
+    │ --shape.b.y FLOAT       (required)                      │
+    │ --shape.b.z FLOAT       (required)                      │
+    │ --shape.b.frame-id STR  (required)                      │
+    ╰─────────────────────────────────────────────────────────╯
+    ╭─ shape.c options ───────────────────────────────────────╮
+    │ --shape.c.x FLOAT       (required)                      │
+    │ --shape.c.y FLOAT       (required)                      │
+    │ --shape.c.z FLOAT       (required)                      │
+    │ --shape.c.frame-id STR  (required)                      │
+    ╰─────────────────────────────────────────────────────────╯
+    
+.. _example-02_generics: + +Generics (Legacy) +----------------- + +The legacy :py:class:`typing.Generic` and :py:class:`typing.TypeVar` syntax for +generic types is also supported. + + +.. code-block:: python + :linenos: + + # 02_generics.py + import dataclasses + from typing import Generic, TypeVar + + import tyro + + ScalarType = TypeVar("ScalarType", int, float) + ShapeType = TypeVar("ShapeType") + + @dataclasses.dataclass(frozen=True) + class Point3(Generic[ScalarType]): + x: ScalarType + y: ScalarType + z: ScalarType + frame_id: str + + @dataclasses.dataclass(frozen=True) + class Triangle: + a: Point3[float] + b: Point3[float] + c: Point3[float] + + @dataclasses.dataclass(frozen=True) + class Args(Generic[ShapeType]): + shape: ShapeType + + if __name__ == "__main__": + args = tyro.cli(Args[Triangle]) + print(args) + + + + +.. raw:: html + +
+    $ python ./02_generics.py --help
+    usage: 02_generics.py [-h] [OPTIONS]
+    
+    ╭─ options ───────────────────────────────────────────────╮
+    │ -h, --help              show this help message and exit │
+    ╰─────────────────────────────────────────────────────────╯
+    ╭─ shape.a options ───────────────────────────────────────╮
+    │ --shape.a.x FLOAT       (required)                      │
+    │ --shape.a.y FLOAT       (required)                      │
+    │ --shape.a.z FLOAT       (required)                      │
+    │ --shape.a.frame-id STR  (required)                      │
+    ╰─────────────────────────────────────────────────────────╯
+    ╭─ shape.b options ───────────────────────────────────────╮
+    │ --shape.b.x FLOAT       (required)                      │
+    │ --shape.b.y FLOAT       (required)                      │
+    │ --shape.b.z FLOAT       (required)                      │
+    │ --shape.b.frame-id STR  (required)                      │
+    ╰─────────────────────────────────────────────────────────╯
+    ╭─ shape.c options ───────────────────────────────────────╮
+    │ --shape.c.x FLOAT       (required)                      │
+    │ --shape.c.y FLOAT       (required)                      │
+    │ --shape.c.z FLOAT       (required)                      │
+    │ --shape.c.frame-id STR  (required)                      │
+    ╰─────────────────────────────────────────────────────────╯
+    
\ No newline at end of file diff --git a/docs/source/examples/nested_structures.rst b/docs/source/examples/nested_structures.rst new file mode 100644 index 00000000..a64a046a --- /dev/null +++ b/docs/source/examples/nested_structures.rst @@ -0,0 +1,594 @@ +.. Comment: this file is automatically generated by `update_example_docs.py`. + It should not be modified manually. + +.. _example-category-nested_structures: + +Nested Structures +================= + +In these examples, we show how :func:`tyro.cli` can be used to instantiate +nested structures. This can enable modular, reusable, and composable CLI +interfaces. + + +.. _example-01_nesting: + +Nested Dataclasses +------------------ + +Structures (typically :py:func:`dataclasses.dataclass`) can be nested to build hierarchical configuration +objects. This helps with modularity and grouping in larger projects. + + +.. code-block:: python + :linenos: + + # 01_nesting.py + import dataclasses + + import tyro + + @dataclasses.dataclass + class OptimizerConfig: + learning_rate: float = 3e-4 + weight_decay: float = 1e-2 + + @dataclasses.dataclass + class Config: + # Optimizer options. + opt: OptimizerConfig + + # Random seed. + seed: int = 0 + + if __name__ == "__main__": + config = tyro.cli(Config) + print(dataclasses.asdict(config)) + + + + +.. raw:: html + +
+    $ python ./01_nesting.py --help
+    usage: 01_nesting.py [-h] [--seed INT] [--opt.learning-rate FLOAT]
+                         [--opt.weight-decay FLOAT]
+    
+    ╭─ options ─────────────────────────────────────────╮
+    │ -h, --help        show this help message and exit │
+    │ --seed INT        Random seed. (default: 0)       │
+    ╰───────────────────────────────────────────────────╯
+    ╭─ opt options ─────────────────────────────────────╮
+    │ Optimizer options.                                │
+    │ ───────────────────────────────────               │
+    │ --opt.learning-rate FLOAT                         │
+    │                   (default: 0.0003)               │
+    │ --opt.weight-decay FLOAT                          │
+    │                   (default: 0.01)                 │
+    ╰───────────────────────────────────────────────────╯
+    
+ + + +.. raw:: html + +
+    $ python ./01_nesting.py --opt.learning-rate 1e-3
+    {'opt': {'learning_rate': 0.001, 'weight_decay': 0.01}, 'seed': 0}
+    
+ + + +.. raw:: html + +
+    $ python ./01_nesting.py --seed 4
+    {'opt': {'learning_rate': 0.0003, 'weight_decay': 0.01}, 'seed': 4}
+    
+.. _example-02_nesting_in_func: + +Structures as Function Arguments +-------------------------------- + +Structures can also be used as input to functions. + + +.. code-block:: python + :linenos: + + # 02_nesting_in_func.py + import dataclasses + import pathlib + + import tyro + + @dataclasses.dataclass + class OptimizerConfig: + learning_rate: float = 3e-4 + weight_decay: float = 1e-2 + + @dataclasses.dataclass + class Config: + # Optimizer options. + optimizer: OptimizerConfig + + # Random seed. + seed: int = 0 + + def train( + out_dir: pathlib.Path, + config: Config, + ) -> None: + """Train a model. + + Args: + out_dir: Where to save logs and checkpoints. + config: Experiment configuration. + """ + print(f"Saving to: {out_dir}") + print(f"Config: f{config}") + + if __name__ == "__main__": + tyro.cli(train) + + + + +.. raw:: html + +
+    $ python ./02_nesting_in_func.py --help
+    usage: 02_nesting_in_func.py [-h] [OPTIONS]
+    
+    Train a model.
+    
+    ╭─ options ──────────────────────────────────────────────────────────────╮
+    │ -h, --help              show this help message and exit                │
+    │ --out-dir PATH          Where to save logs and checkpoints. (required) │
+    ╰────────────────────────────────────────────────────────────────────────╯
+    ╭─ config options ───────────────────────────────────────────────────────╮
+    │ Experiment configuration.                                              │
+    │ ─────────────────────────────────────────────────                      │
+    │ --config.seed INT       Random seed. (default: 0)                      │
+    ╰────────────────────────────────────────────────────────────────────────╯
+    ╭─ config.optimizer options ─────────────────────────────────────────────╮
+    │ Optimizer options.                                                     │
+    │ ─────────────────────────────────────────────────                      │
+    │ --config.optimizer.learning-rate FLOAT                                 │
+    │                         (default: 0.0003)                              │
+    │ --config.optimizer.weight-decay FLOAT                                  │
+    │                         (default: 0.01)                                │
+    ╰────────────────────────────────────────────────────────────────────────╯
+    
+ + + +.. raw:: html + +
+    $ python ./02_nesting_in_func.py --out-dir /tmp/test1
+    Saving to: /tmp/test1
+    Config: fConfig(optimizer=OptimizerConfig(learning_rate=0.0003, weight_decay=0.01), seed=0)
+    
+ + + +.. raw:: html + +
+    $ python ./02_nesting_in_func.py --out-dir /tmp/test2 --config.seed 4
+    Saving to: /tmp/test2
+    Config: fConfig(optimizer=OptimizerConfig(learning_rate=0.0003, weight_decay=0.01), seed=4)
+    
+.. _example-03_nesting_containers: + +Nesting in Containers +--------------------- + +Structures can be nested inside of standard containers. + +.. warning:: + + When placing structures inside of containers like lists or tuples, the + length of the container must be inferrable from the annotation or default + value. + + +.. code-block:: python + :linenos: + + # 03_nesting_containers.py + import dataclasses + + import tyro + + @dataclasses.dataclass + class RGB: + r: int + g: int + b: int + + @dataclasses.dataclass + class Args: + color_tuple: tuple[RGB, RGB] + color_dict: dict[str, RGB] = dataclasses.field( + # We can't use mutable values as defaults directly. + default_factory={ + "red": RGB(255, 0, 0), + "green": RGB(0, 255, 0), + "blue": RGB(0, 0, 255), + }.copy + ) + + if __name__ == "__main__": + args = tyro.cli(Args) + print(args) + + + + +.. raw:: html + +
+    $ python ./03_nesting_containers.py --help
+    usage: 03_nesting_containers.py [-h] [OPTIONS]
+    
+    ╭─ options ───────────────────────────────────────────────╮
+    │ -h, --help              show this help message and exit │
+    ╰─────────────────────────────────────────────────────────╯
+    ╭─ color-tuple.0 options ─────────────────────────────────╮
+    │ --color-tuple.0.r INT   (required)                      │
+    │ --color-tuple.0.g INT   (required)                      │
+    │ --color-tuple.0.b INT   (required)                      │
+    ╰─────────────────────────────────────────────────────────╯
+    ╭─ color-tuple.1 options ─────────────────────────────────╮
+    │ --color-tuple.1.r INT   (required)                      │
+    │ --color-tuple.1.g INT   (required)                      │
+    │ --color-tuple.1.b INT   (required)                      │
+    ╰─────────────────────────────────────────────────────────╯
+    ╭─ color-dict.red options ────────────────────────────────╮
+    │ --color-dict.red.r INT  (default: 255)                  │
+    │ --color-dict.red.g INT  (default: 0)                    │
+    │ --color-dict.red.b INT  (default: 0)                    │
+    ╰─────────────────────────────────────────────────────────╯
+    ╭─ color-dict.green options ──────────────────────────────╮
+    │ --color-dict.green.r INT                                │
+    │                         (default: 0)                    │
+    │ --color-dict.green.g INT                                │
+    │                         (default: 255)                  │
+    │ --color-dict.green.b INT                                │
+    │                         (default: 0)                    │
+    ╰─────────────────────────────────────────────────────────╯
+    ╭─ color-dict.blue options ───────────────────────────────╮
+    │ --color-dict.blue.r INT                                 │
+    │                         (default: 0)                    │
+    │ --color-dict.blue.g INT                                 │
+    │                         (default: 0)                    │
+    │ --color-dict.blue.b INT                                 │
+    │                         (default: 255)                  │
+    ╰─────────────────────────────────────────────────────────╯
+    
+.. _example-04_dictionaries: + +Dictionaries and TypedDict +-------------------------- + +Dictionary inputs can be specified using either a standard ``dict[K, V]`` +annotation, or a :code:`TypedDict` subclass. + +For configuring :code:`TypedDict`, we also support :code:`total={True/False}`, +:code:`typing.Required`, and :code:`typing.NotRequired`. See the `Python docs `_ for all :code:`TypedDict` features. + + +.. code-block:: python + :linenos: + + # 04_dictionaries.py + from typing import TypedDict + + from typing_extensions import NotRequired + + import tyro + + class DictionarySchemaA( + TypedDict, + # Setting `total=False` specifies that not all keys need to exist. + total=False, + ): + learning_rate: float + betas: tuple[float, float] + + class DictionarySchemaB(TypedDict): + learning_rate: NotRequired[float] + """NotRequired[] specifies that a particular key doesn't need to exist.""" + betas: tuple[float, float] + + def main( + typed_dict_a: DictionarySchemaA, + typed_dict_b: DictionarySchemaB, + standard_dict: dict[str, float] = { + "learning_rate": 3e-4, + "beta1": 0.9, + "beta2": 0.999, + }, + ) -> None: + assert isinstance(typed_dict_a, dict) + assert isinstance(typed_dict_b, dict) + assert isinstance(standard_dict, dict) + print("Typed dict A:", typed_dict_a) + print("Typed dict B:", typed_dict_b) + print("Standard dict:", standard_dict) + + if __name__ == "__main__": + tyro.cli(main) + + + + +.. raw:: html + +
+    $ python ./04_dictionaries.py --help
+    usage: 04_dictionaries.py [-h] [OPTIONS]
+    
+    ╭─ options ──────────────────────────────────────────────────────────────────╮
+    │ -h, --help        show this help message and exit                          │
+    ╰────────────────────────────────────────────────────────────────────────────╯
+    ╭─ typed-dict-a options ─────────────────────────────────────────────────────╮
+    │ --typed-dict-a.learning-rate FLOAT                                         │
+    │                   (unset by default)                                       │
+    │ --typed-dict-a.betas FLOAT FLOAT                                           │
+    │                   (unset by default)                                       │
+    ╰────────────────────────────────────────────────────────────────────────────╯
+    ╭─ typed-dict-b options ─────────────────────────────────────────────────────╮
+    │ --typed-dict-b.learning-rate FLOAT                                         │
+    │                   NotRequired[] specifies that a particular key doesn't    │
+    │                   need to exist. (unset by default)                        │
+    │ --typed-dict-b.betas FLOAT FLOAT                                           │
+    │                   (required)                                               │
+    ╰────────────────────────────────────────────────────────────────────────────╯
+    ╭─ standard-dict options ────────────────────────────────────────────────────╮
+    │ --standard-dict.learning-rate FLOAT                                        │
+    │                   (default: 0.0003)                                        │
+    │ --standard-dict.beta1 FLOAT                                                │
+    │                   (default: 0.9)                                           │
+    │ --standard-dict.beta2 FLOAT                                                │
+    │                   (default: 0.999)                                         │
+    ╰────────────────────────────────────────────────────────────────────────────╯
+    
+ + + +.. raw:: html + +
+    $ python ./04_dictionaries.py --typed-dict-a.learning-rate 3e-4 --typed-dict-b.betas 0.9 0.999
+    Typed dict A: {'learning_rate': 0.0003}
+    Typed dict B: {'betas': (0.9, 0.999)}
+    Standard dict: {'learning_rate': 0.0003, 'beta1': 0.9, 'beta2': 0.999}
+    
+ + + +.. raw:: html + +
+    $ python ./04_dictionaries.py --typed-dict-b.betas 0.9 0.999
+    Typed dict A: {}
+    Typed dict B: {'betas': (0.9, 0.999)}
+    Standard dict: {'learning_rate': 0.0003, 'beta1': 0.9, 'beta2': 0.999}
+    
+.. _example-05_tuples: + +Tuples and NamedTuple +--------------------- + +Example using :func:`tyro.cli()` to instantiate tuple types. :code:`tuple`, +:py:data:`typing.Tuple`, and :py:class:`typing.NamedTuple` are all supported. + + +.. code-block:: python + :linenos: + + # 05_tuples.py + from typing import NamedTuple + + import tyro + + # Named tuples are interpreted as nested structures. + class Color(NamedTuple): + r: int + g: int + b: int + + class TupleType(NamedTuple): + """Description. + This should show up in the helptext!""" + + # Tuple types can contain raw values. + color: tuple[int, int, int] = (255, 0, 0) + + # Tuple types can contain nested structures. + two_colors: tuple[Color, Color] = (Color(255, 0, 0), Color(0, 255, 0)) + + if __name__ == "__main__": + x = tyro.cli(TupleType) + assert isinstance(x, tuple) + print(x) + + + + +.. raw:: html + +
+    $ python ./05_tuples.py --help
+    usage: 05_tuples.py [-h] [OPTIONS]
+    
+    Description. This should show up in the helptext!
+    
+    ╭─ options ──────────────────────────────────────────────────────────────────╮
+    │ -h, --help              show this help message and exit                    │
+    │ --color INT INT INT     Tuple types can contain raw values. (default: 255  │
+    │                         0 0)                                               │
+    ╰────────────────────────────────────────────────────────────────────────────╯
+    ╭─ two-colors.0 options ─────────────────────────────────────────────────────╮
+    │ --two-colors.0.r INT    (default: 255)                                     │
+    │ --two-colors.0.g INT    (default: 0)                                       │
+    │ --two-colors.0.b INT    (default: 0)                                       │
+    ╰────────────────────────────────────────────────────────────────────────────╯
+    ╭─ two-colors.1 options ─────────────────────────────────────────────────────╮
+    │ --two-colors.1.r INT    (default: 0)                                       │
+    │ --two-colors.1.g INT    (default: 255)                                     │
+    │ --two-colors.1.b INT    (default: 0)                                       │
+    ╰────────────────────────────────────────────────────────────────────────────╯
+    
+ + + +.. raw:: html + +
+    $ python ./05_tuples.py --color 127 127 127
+    TupleType(color=(127, 127, 127), two_colors=(Color(r=255, g=0, b=0), Color(r=0, g=255, b=0)))
+    
+ + + +.. raw:: html + +
+    $ python ./05_tuples.py --two-colors.1.r 127 --two-colors.1.g 0 --two-colors.1.b 0
+    TupleType(color=(255, 0, 0), two_colors=(Color(r=255, g=0, b=0), Color(r=127, g=0, b=0)))
+    
+.. _example-06_pydantic: + +Pydantic Integration +-------------------- + +In addition to standard dataclasses, :func:`tyro.cli()` also supports +`Pydantic `_ models. + + +.. code-block:: python + :linenos: + + # 06_pydantic.py + from pydantic import BaseModel, Field + + import tyro + + class Args(BaseModel): + """Description. + This should show up in the helptext!""" + + field1: str + field2: int = Field(3, description="An integer field.") + + if __name__ == "__main__": + args = tyro.cli(Args) + print(args) + + + + +.. raw:: html + +
+    $ python ./06_pydantic.py --help
+    usage: 06_pydantic.py [-h] --field1 STR [--field2 INT]
+    
+    Description. This should show up in the helptext!
+    
+    ╭─ options ───────────────────────────────────────────╮
+    │ -h, --help          show this help message and exit │
+    │ --field1 STR        (required)                      │
+    │ --field2 INT        An integer field. (default: 3)  │
+    ╰─────────────────────────────────────────────────────╯
+    
+ + + +.. raw:: html + +
+    $ python ./06_pydantic.py --field1 hello
+    field1='hello' field2=3
+    
+ + + +.. raw:: html + +
+    $ python ./06_pydantic.py --field1 hello --field2 5
+    field1='hello' field2=5
+    
+.. _example-07_attrs: + +Attrs Integration +----------------- + +In addition to standard dataclasses, :func:`tyro.cli()` also supports +`attrs `_ classes. + + +.. code-block:: python + :linenos: + + # 07_attrs.py + import attr + + import tyro + + @attr.s + class Args: + """Description. + This should show up in the helptext!""" + + field1: str = attr.ib() + """A string field.""" + + field2: int = attr.ib(factory=lambda: 5) + """A required integer field.""" + + if __name__ == "__main__": + args = tyro.cli(Args) + print(args) + + + + +.. raw:: html + +
+    $ python ./07_attrs.py --help
+    usage: 07_attrs.py [-h] --field1 STR [--field2 INT]
+    
+    Description. This should show up in the helptext!
+    
+    ╭─ options ──────────────────────────────────────────────────╮
+    │ -h, --help          show this help message and exit        │
+    │ --field1 STR        A string field. (required)             │
+    │ --field2 INT        A required integer field. (default: 5) │
+    ╰────────────────────────────────────────────────────────────╯
+    
+ + + +.. raw:: html + +
+    $ python ./07_attrs.py --field1 hello
+    Args(field1='hello', field2=5)
+    
+ + + +.. raw:: html + +
+    $ python ./07_attrs.py --field1 hello --field2 5
+    Args(field1='hello', field2=5)
+    
\ No newline at end of file diff --git a/docs/source/examples/overriding_configs.rst b/docs/source/examples/overriding_configs.rst new file mode 100644 index 00000000..3f28f2ac --- /dev/null +++ b/docs/source/examples/overriding_configs.rst @@ -0,0 +1,375 @@ +.. Comment: this file is automatically generated by `update_example_docs.py`. + It should not be modified manually. + +.. _example-category-overriding_configs: + +Overriding Configs +================== + +In these examples, we show how :func:`tyro.cli` can be used to override values +in pre-instantiated configuration objects. + + +.. _example-01_dataclasses_defaults: + +Dataclasses + Defaults +---------------------- + +The :code:`default=` argument can be used to override default values in dataclass +types. + + +.. note:: + + When ``default=`` is used, we advise against mutation of configuration + objects from a dataclass's :code:`__post_init__` method [#f1]_. In the + example below, :code:`__post_init__` would be called twice: once for the + :code:`Args()` object provided as a default value and another time for the + :code:`Args()` objected instantiated by :func:`tyro.cli()`. This can cause + confusing behavior! Instead, we show below one example of how derived + fields can be defined immutably. + + .. [#f1] Official Python docs for ``__post_init__`` can be found `here `_. + + +.. code-block:: python + :linenos: + + # 01_dataclasses_defaults.py + import dataclasses + + import tyro + + @dataclasses.dataclass + class Args: + """Description. + This should show up in the helptext!""" + + string: str + """A string field.""" + + reps: int = 3 + """A numeric field, with a default value.""" + + @property + def derived_field(self) -> str: + return ", ".join([self.string] * self.reps) + + if __name__ == "__main__": + args = tyro.cli( + Args, + default=Args( + string="default string", + reps=tyro.MISSING, + ), + ) + print(args.derived_field) + + + + +.. raw:: html + +
+    $ python ./01_dataclasses_defaults.py --help
+    usage: 01_dataclasses_defaults.py [-h] [--string STR] --reps INT
+    
+    Description. This should show up in the helptext!
+    
+    ╭─ options ─────────────────────────────────────────────────────────────╮
+    │ -h, --help          show this help message and exit                   │
+    │ --string STR        A string field. (default: 'default string')       │
+    │ --reps INT          A numeric field, with a default value. (required) │
+    ╰───────────────────────────────────────────────────────────────────────╯
+    
+ + + +.. raw:: html + +
+    $ python ./01_dataclasses_defaults.py --reps 3
+    default string, default string, default string
+    
+ + + +.. raw:: html + +
+    $ python ./01_dataclasses_defaults.py --string hello --reps 5
+    hello, hello, hello, hello, hello
+    
+.. _example-02_overriding_yaml: + +Overriding YAML Configs +----------------------- + +:mod:`tyro` understands a wide range of data structures, including standard dictionaries +and lists. + +If you have a library of existing YAML files that you want to use, `tyro` can +help override values within them. + +.. note:: + + We recommend dataclass configs for new projects. + + +.. code-block:: python + :linenos: + + # 02_overriding_yaml.py + import yaml + + import tyro + + # YAML configuration. Note that this could also be loaded from a file! Environment + # variables are an easy way to select between different YAML files. + default_yaml = r""" + exp_name: test + optimizer: + learning_rate: 0.0001 + type: adam + training: + batch_size: 32 + num_steps: 10000 + checkpoint_steps: + - 500 + - 1000 + - 1500 + """.strip() + + if __name__ == "__main__": + # Convert our YAML config into a nested dictionary. + default_config = yaml.safe_load(default_yaml) + + # Override fields in the dictionary. + overridden_config = tyro.cli(dict, default=default_config) + + # Print the overridden config. + overridden_yaml = yaml.safe_dump(overridden_config) + print(overridden_yaml) + + + + +.. raw:: html + +
+    $ python ./02_overriding_yaml.py --help
+    usage: 02_overriding_yaml.py [-h] [OPTIONS]
+    
+    ╭─ options ───────────────────────────────────────────────╮
+    │ -h, --help              show this help message and exit │
+    │ --exp-name STR          (default: test)                 │
+    ╰─────────────────────────────────────────────────────────╯
+    ╭─ optimizer options ─────────────────────────────────────╮
+    │ --optimizer.learning-rate FLOAT                         │
+    │                         (default: 0.0001)               │
+    │ --optimizer.type STR    (default: adam)                 │
+    ╰─────────────────────────────────────────────────────────╯
+    ╭─ training options ──────────────────────────────────────╮
+    │ --training.batch-size INT                               │
+    │                         (default: 32)                   │
+    │ --training.num-steps INT                                │
+    │                         (default: 10000)                │
+    │ --training.checkpoint-steps [INT [INT ...]]             │
+    │                         (default: 500 1000 1500)        │
+    ╰─────────────────────────────────────────────────────────╯
+    
+ + + +.. raw:: html + +
+    $ python ./02_overriding_yaml.py --training.checkpoint-steps 300 1000 9000
+    exp_name: test
+    optimizer:
+      learning_rate: 0.0001
+      type: adam
+    training:
+      batch_size: 32
+      checkpoint_steps:
+      - 300
+      - 1000
+      - 9000
+      num_steps: 10000
+    
+    
+.. _example-03_choosing_base_configs: + +Choosing Base Configs +--------------------- + +One common pattern is to have a set of "base" configurations, which can be +selected from and then overridden. + +This is often implemented with a set of configuration files (e.g., YAML files). +With :mod:`tyro`, we can instead define each base configuration as a separate +Python object. + +After creating the base configurations, we can use the CLI to select one of +them and then override (existing) or fill in (missing) values. + +The helper function used here, :func:`tyro.extras.overridable_config_cli()`, is +a lightweight wrapper over :func:`tyro.cli()` and its Union-based subcommand +syntax. + + +.. code-block:: python + :linenos: + + # 03_choosing_base_configs.py + from dataclasses import dataclass + from typing import Callable, Literal + + from torch import nn + + import tyro + + @dataclass(frozen=True) + class ExperimentConfig: + # Dataset to run experiment on. + dataset: Literal["mnist", "imagenet-50"] + + # Model size. + num_layers: int + units: int + + # Batch size. + batch_size: int + + # Total number of training steps. + train_steps: int + + # Random seed. This is helpful for making sure that our experiments are all + # reproducible! + seed: int + + # Activation to use. Not specifiable via the commandline. + activation: Callable[[], nn.Module] + + # Note that we could also define this library using separate YAML files (similar to + # `config_path`/`config_name` in Hydra), but staying in Python enables seamless type + # checking + IDE support. + default_configs = { + "small": ( + "Small experiment.", + ExperimentConfig( + dataset="mnist", + batch_size=2048, + num_layers=4, + units=64, + train_steps=30_000, + seed=0, + activation=nn.ReLU, + ), + ), + "big": ( + "Big experiment.", + ExperimentConfig( + dataset="imagenet-50", + batch_size=32, + num_layers=8, + units=256, + train_steps=100_000, + seed=0, + activation=nn.GELU, + ), + ), + } + if __name__ == "__main__": + config = tyro.extras.overridable_config_cli(default_configs) + print(config) + + +Overall helptext: + +.. raw:: html + +
+    $ python ./03_choosing_base_configs.py --help
+    usage: 03_choosing_base_configs.py [-h] {small,big}
+    
+    ╭─ options ──────────────────────────────────────────╮
+    │ -h, --help         show this help message and exit │
+    ╰────────────────────────────────────────────────────╯
+    ╭─ subcommands ──────────────────────────────────────╮
+    │ {small,big}                                        │
+    │     small          Small experiment.               │
+    │     big            Big experiment.                 │
+    ╰────────────────────────────────────────────────────╯
+    
+ +The "small" subcommand: + +.. raw:: html + +
+    $ python ./03_choosing_base_configs.py small --help
+    usage: 03_choosing_base_configs.py small [-h] [SMALL OPTIONS]
+    
+    Small experiment.
+    
+    ╭─ options ──────────────────────────────────────────────────────────────────╮
+    │ -h, --help              show this help message and exit                    │
+    │ --dataset {mnist,imagenet-50}                                              │
+    │                         Dataset to run experiment on. (default: mnist)     │
+    │ --num-layers INT        Model size. (default: 4)                           │
+    │ --units INT             Model size. (default: 64)                          │
+    │ --batch-size INT        Batch size. (default: 2048)                        │
+    │ --train-steps INT       Total number of training steps. (default: 30000)   │
+    │ --seed INT              Random seed. This is helpful for making sure that  │
+    │                         our experiments are all reproducible! (default: 0) │
+    │ --activation {fixed}    Activation to use. Not specifiable via the         │
+    │                         commandline. (fixed to: <class                     │
+    │                         'torch.nn.modules.activation.ReLU'>)               │
+    ╰────────────────────────────────────────────────────────────────────────────╯
+    
+ + + +.. raw:: html + +
+    $ python ./03_choosing_base_configs.py small --seed 94720
+    ExperimentConfig(dataset='mnist', num_layers=4, units=64, batch_size=2048, train_steps=30000, seed=94720, activation=<class 'torch.nn.modules.activation.ReLU'>)
+    
+ +The "big" subcommand: + +.. raw:: html + +
+    $ python ./03_choosing_base_configs.py big --help
+    usage: 03_choosing_base_configs.py big [-h] [BIG OPTIONS]
+    
+    Big experiment.
+    
+    ╭─ options ──────────────────────────────────────────────────────────────────╮
+    │ -h, --help              show this help message and exit                    │
+    │ --dataset {mnist,imagenet-50}                                              │
+    │                         Dataset to run experiment on. (default:            │
+    │                         imagenet-50)                                       │
+    │ --num-layers INT        Model size. (default: 8)                           │
+    │ --units INT             Model size. (default: 256)                         │
+    │ --batch-size INT        Batch size. (default: 32)                          │
+    │ --train-steps INT       Total number of training steps. (default: 100000)  │
+    │ --seed INT              Random seed. This is helpful for making sure that  │
+    │                         our experiments are all reproducible! (default: 0) │
+    │ --activation {fixed}    Activation to use. Not specifiable via the         │
+    │                         commandline. (fixed to: <class                     │
+    │                         'torch.nn.modules.activation.GELU'>)               │
+    ╰────────────────────────────────────────────────────────────────────────────╯
+    
+ + + +.. raw:: html + +
+    $ python ./03_choosing_base_configs.py big --seed 94720
+    ExperimentConfig(dataset='imagenet-50', num_layers=8, units=256, batch_size=32, train_steps=100000, seed=94720, activation=<class 'torch.nn.modules.activation.GELU'>)
+    
\ No newline at end of file diff --git a/docs/source/examples/pytorch_jax.rst b/docs/source/examples/pytorch_jax.rst new file mode 100644 index 00000000..fe9735c6 --- /dev/null +++ b/docs/source/examples/pytorch_jax.rst @@ -0,0 +1,159 @@ +.. Comment: this file is automatically generated by `update_example_docs.py`. + It should not be modified manually. + +.. _example-category-pytorch_jax: + +PyTorch / JAX +============= + +In these examples, we show some patterns for using :func:`tyro.cli` with PyTorch and JAX. + + +.. _example-01_pytorch_parallelism: + +PyTorch Parallelism +------------------- + +The :code:`console_outputs=` argument can be set to :code:`False` to suppress helptext and +error message printing. + +This is useful in PyTorch for distributed training scripts, where you only want +to print the helptext from the main process: + + +.. code-block:: python + + # HuggingFace Accelerate. + args = tyro.cli(Args, console_outputs=accelerator.is_main_process) + + # PyTorch DDP. + args = tyro.cli(Args, console_outputs=(rank == 0)) + + # PyTorch Lightning. + args = tyro.cli(Args, console_outputs=trainer.is_global_zero) + + +.. code-block:: python + :linenos: + + # 01_pytorch_parallelism.py + import dataclasses + + import tyro + + @dataclasses.dataclass + class Args: + """Description. + This should show up in the helptext!""" + + field1: int + """A field.""" + + field2: int = 3 + """A numeric field, with a default value.""" + + if __name__ == "__main__": + args = tyro.cli(Args, console_outputs=False) + print(args) + + + + +.. raw:: html + +
+    $ python ./01_pytorch_parallelism.py --help
+    
+.. _example-02_flax: + +JAX/Flax Integration +-------------------- + +If you use `flax.linen `_, modules can be instantiated +directly from :func:`tyro.cli()`. + + +.. code-block:: python + :linenos: + + # 02_flax.py + from flax import linen as nn + from jax import numpy as jnp + + import tyro + + class Classifier(nn.Module): + layers: int + """Layers in our network.""" + units: int = 32 + """Hidden unit count.""" + output_dim: int = 10 + """Number of classes.""" + + @nn.compact + def __call__(self, x: jnp.ndarray) -> jnp.ndarray: # type: ignore + for i in range(self.layers - 1): + x = nn.Dense( + self.units, + kernel_init=nn.initializers.kaiming_normal(), + )(x) + x = nn.relu(x) + + x = nn.Dense( + self.output_dim, + kernel_init=nn.initializers.xavier_normal(), + )(x) + x = nn.sigmoid(x) + return x + + def train(model: Classifier, num_iterations: int = 1000) -> None: + """Train a model. + + Args: + model: Model to train. + num_iterations: Number of training iterations. + """ + print(f"{model=}") + print(f"{num_iterations=}") + + if __name__ == "__main__": + tyro.cli(train) + + + + +.. raw:: html + +
+    $ python ./02_flax.py --help
+    usage: 02_flax.py [-h] [OPTIONS]
+    
+    Train a model.
+    
+    ╭─ options ──────────────────────────────────────────────────────────────╮
+    │ -h, --help              show this help message and exit                │
+    │ --num-iterations INT    Number of training iterations. (default: 1000) │
+    ╰────────────────────────────────────────────────────────────────────────╯
+    ╭─ model options ────────────────────────────────────────────────────────╮
+    │ Model to train.                                                        │
+    │ ─────────────────────────────────────────────────────────              │
+    │ --model.layers INT      Layers in our network. (required)              │
+    │ --model.units INT       Hidden unit count. (default: 32)               │
+    │ --model.output-dim INT  Number of classes. (default: 10)               │
+    ╰────────────────────────────────────────────────────────────────────────╯
+    
+ + + +.. raw:: html + +
+    $ python ./02_flax.py --model.layers 4
+    model=Classifier(
+        # attributes
+        layers = 4
+        units = 32
+        output_dim = 10
+    )
+    num_iterations=1000
+    
\ No newline at end of file diff --git a/docs/source/examples/subcommands.rst b/docs/source/examples/subcommands.rst new file mode 100644 index 00000000..e99d5a76 --- /dev/null +++ b/docs/source/examples/subcommands.rst @@ -0,0 +1,573 @@ +.. Comment: this file is automatically generated by `update_example_docs.py`. + It should not be modified manually. + +.. _example-category-subcommands: + +Subcommands +=========== + +In these examples, we show how :func:`tyro.cli` can be used to create CLI +interfaces with subcommands. + + +.. _example-01_subcommands: + +Subcommands are Unions +---------------------- + +All of :mod:`tyro`'s subcommand features are built using unions over struct +types (typically dataclasses). Subcommands are used to choose between types in +the union; arguments are then populated from the chosen type. + +.. note:: + + For configuring subcommands beyond what can be expressed with type annotations, see + :func:`tyro.conf.subcommand()`. + + +.. code-block:: python + :linenos: + + # 01_subcommands.py + from __future__ import annotations + + import dataclasses + + import tyro + + @dataclasses.dataclass(frozen=True) + class Checkout: + """Checkout a branch.""" + + branch: str + + @dataclasses.dataclass(frozen=True) + class Commit: + """Commit changes.""" + + message: str + + if __name__ == "__main__": + cmd = tyro.cli(Checkout | Commit) + print(cmd) + + +Print the helptext. This will show the available subcommands: + +.. raw:: html + +
+    $ python ./01_subcommands.py --help
+    usage: 01_subcommands.py [-h] {checkout,commit}
+    
+    ╭─ options ───────────────────────────────────────────────╮
+    │ -h, --help              show this help message and exit │
+    ╰─────────────────────────────────────────────────────────╯
+    ╭─ subcommands ───────────────────────────────────────────╮
+    │ {checkout,commit}                                       │
+    │     checkout            Checkout a branch.              │
+    │     commit              Commit changes.                 │
+    ╰─────────────────────────────────────────────────────────╯
+    
+ +The `commit` subcommand: + +.. raw:: html + +
+    $ python ./01_subcommands.py commit --help
+    usage: 01_subcommands.py commit [-h] --message STR
+    
+    Commit changes.
+    
+    ╭─ options ────────────────────────────────────────────╮
+    │ -h, --help           show this help message and exit │
+    │ --message STR        (required)                      │
+    ╰──────────────────────────────────────────────────────╯
+    
+ + + +.. raw:: html + +
+    $ python ./01_subcommands.py commit --message hello
+    Commit(message='hello')
+    
+ +The `checkout` subcommand: + +.. raw:: html + +
+    $ python ./01_subcommands.py checkout --help
+    usage: 01_subcommands.py checkout [-h] --branch STR
+    
+    Checkout a branch.
+    
+    ╭─ options ───────────────────────────────────────────╮
+    │ -h, --help          show this help message and exit │
+    │ --branch STR        (required)                      │
+    ╰─────────────────────────────────────────────────────╯
+    
+ + + +.. raw:: html + +
+    $ python ./01_subcommands.py checkout --branch main
+    Checkout(branch='main')
+    
+.. _example-02_subcommands_in_func: + +Subcommands as Function Arguments +--------------------------------- + +A subcommand will be created for each input annotated with a union over +struct types. + +.. note:: + + To prevent :func:`tyro.cli()` from converting a Union type into a subcommand, + use :class:`tyro.conf.AvoidSubcommands`. + +.. note:: + + Argument ordering for subcommands can be tricky. In the example below, + ``--shared-arg`` must always come *before* the subcommand. As an option for + alleviating this, see :class:`tyro.conf.ConsolidateSubcommandArgs`. + + +.. code-block:: python + :linenos: + + # 02_subcommands_in_func.py + from __future__ import annotations + + import dataclasses + + import tyro + + @dataclasses.dataclass(frozen=True) + class Checkout: + """Checkout a branch.""" + + branch: str + + @dataclasses.dataclass(frozen=True) + class Commit: + """Commit changes.""" + + message: str + + def main( + shared_arg: int, + cmd: Checkout | Commit = Checkout(branch="default"), + ): + print(f"{shared_arg=}") + print(cmd) + + if __name__ == "__main__": + tyro.cli(main) + + +Print the helptext. This will show the available subcommands: + +.. raw:: html + +
+    $ python ./02_subcommands_in_func.py --help
+    usage: 02_subcommands_in_func.py [-h] --shared-arg INT
+                                     [{cmd:checkout,cmd:commit}]
+    
+    ╭─ options ───────────────────────────────────────────────╮
+    │ -h, --help              show this help message and exit │
+    │ --shared-arg INT        (required)                      │
+    ╰─────────────────────────────────────────────────────────╯
+    ╭─ optional subcommands ──────────────────────────────────╮
+    │ (default: cmd:checkout)                                 │
+    │ ──────────────────────────────────────────              │
+    │ [{cmd:checkout,cmd:commit}]                             │
+    │     cmd:checkout        Checkout a branch.              │
+    │     cmd:commit          Commit changes.                 │
+    ╰─────────────────────────────────────────────────────────╯
+    
+ +Using the default subcommand: + +.. raw:: html + +
+    $ python ./02_subcommands_in_func.py --shared-arg 100
+    shared_arg=100
+    Checkout(branch='default')
+    
+ +Choosing a different subcommand: + +.. raw:: html + +
+    $ python ./02_subcommands_in_func.py --shared-arg 100 cmd:commit --cmd.message 'Hello!'
+    shared_arg=100
+    Commit(message='Hello!')
+    
+.. _example-03_multiple_subcommands: + +Sequenced Subcommands +--------------------- + +Multiple unions over struct types are populated using a series of subcommands. + + +.. code-block:: python + :linenos: + + # 03_multiple_subcommands.py + from __future__ import annotations + + import dataclasses + from typing import Literal + + import tyro + + # Possible dataset configurations. + + @dataclasses.dataclass + class Mnist: + binary: bool = False + """Set to load binary version of MNIST dataset.""" + + @dataclasses.dataclass + class ImageNet: + subset: Literal[50, 100, 1000] + """Choose between ImageNet-50, ImageNet-100, ImageNet-1000, etc.""" + + # Possible optimizer configurations. + + @dataclasses.dataclass + class Adam: + learning_rate: float = 1e-3 + betas: tuple[float, float] = (0.9, 0.999) + + @dataclasses.dataclass + class Sgd: + learning_rate: float = 3e-4 + + # Train script. + + def train( + dataset: Mnist | ImageNet = Mnist(), + optimizer: Adam | Sgd = Adam(), + ) -> None: + """Example training script. + + Args: + dataset: Dataset to train on. + optimizer: Optimizer to train with. + + Returns: + None: + """ + print(dataset) + print(optimizer) + + if __name__ == "__main__": + tyro.cli(train, config=(tyro.conf.ConsolidateSubcommandArgs,)) + + +Note that we apply the :class:`tyro.conf.ConsolidateSubcommandArgs` flag. +This pushes all arguments to the end of the command: + +.. raw:: html + +
+    $ python ./03_multiple_subcommands.py --help
+    usage: 03_multiple_subcommands.py [-h] {dataset:mnist,dataset:image-net}
+    
+    Example training script.
+    
+    ╭─ options ─────────────────────────────────────────╮
+    │ -h, --help        show this help message and exit │
+    ╰───────────────────────────────────────────────────╯
+    ╭─ subcommands ─────────────────────────────────────╮
+    │ Dataset to train on.                              │
+    │ ─────────────────────────────────                 │
+    │ {dataset:mnist,dataset:image-net}                 │
+    │     dataset:mnist                                 │
+    │     dataset:image-net                             │
+    ╰───────────────────────────────────────────────────╯
+    
+ + + +.. raw:: html + +
+    $ python ./03_multiple_subcommands.py dataset:mnist --help
+    usage: 03_multiple_subcommands.py dataset:mnist [-h]
+                                                    {optimizer:adam,optimizer:sgd}
+    
+    ╭─ options ─────────────────────────────────────────╮
+    │ -h, --help        show this help message and exit │
+    ╰───────────────────────────────────────────────────╯
+    ╭─ subcommands ─────────────────────────────────────╮
+    │ Optimizer to train with.                          │
+    │ ──────────────────────────────                    │
+    │ {optimizer:adam,optimizer:sgd}                    │
+    │     optimizer:adam                                │
+    │     optimizer:sgd                                 │
+    ╰───────────────────────────────────────────────────╯
+    
+ + + +.. raw:: html + +
+    $ python ./03_multiple_subcommands.py dataset:mnist optimizer:adam --help
+    usage: 03_multiple_subcommands.py dataset:mnist optimizer:adam
+           [-h] [--optimizer.learning-rate FLOAT] [--optimizer.betas FLOAT FLOAT]
+           [--dataset.binary | --dataset.no-binary]
+    
+    ╭─ options ────────────────────────────────────────────────────────╮
+    │ -h, --help                                                       │
+    │     show this help message and exit                              │
+    ╰──────────────────────────────────────────────────────────────────╯
+    ╭─ optimizer options ──────────────────────────────────────────────╮
+    │ --optimizer.learning-rate FLOAT                                  │
+    │     (default: 0.001)                                             │
+    │ --optimizer.betas FLOAT FLOAT                                    │
+    │     (default: 0.9 0.999)                                         │
+    ╰──────────────────────────────────────────────────────────────────╯
+    ╭─ dataset options ────────────────────────────────────────────────╮
+    │ --dataset.binary, --dataset.no-binary                            │
+    │     Set to load binary version of MNIST dataset. (default: None) │
+    
+    ╰──────────────────────────────────────────────────────────────────╯
+    
+ + + +.. raw:: html + +
+    $ python ./03_multiple_subcommands.py dataset:mnist optimizer:adam --optimizer.learning-rate 3e-4 --dataset.binary
+    Mnist(binary=True)
+    Adam(learning_rate=0.0003, betas=(0.9, 0.999))
+    
+.. _example-04_decorator_subcommands: + +Decorator-based Subcommands +--------------------------- + +:func:`tyro.extras.SubcommandApp()` provides a decorator-based API for +subcommands, which is inspired by `click `_. + + +.. code-block:: python + :linenos: + + # 04_decorator_subcommands.py + from tyro.extras import SubcommandApp + + app = SubcommandApp() + + @app.command + def greet(name: str, loud: bool = False) -> None: + """Greet someone.""" + greeting = f"Hello, {name}!" + if loud: + greeting = greeting.upper() + print(greeting) + + @app.command(name="addition") + def add(a: int, b: int) -> None: + """Add two numbers.""" + print(f"{a} + {b} = {a + b}") + + if __name__ == "__main__": + app.cli() + + + + +.. raw:: html + +
+    $ python 04_decorator_subcommands.py --help
+    usage: 04_decorator_subcommands.py [-h] {greet,addition}
+    
+    ╭─ options ───────────────────────────────────────────────╮
+    │ -h, --help              show this help message and exit │
+    ╰─────────────────────────────────────────────────────────╯
+    ╭─ subcommands ───────────────────────────────────────────╮
+    │ {greet,addition}                                        │
+    │     greet               Greet someone.                  │
+    │     addition            Add two numbers.                │
+    ╰─────────────────────────────────────────────────────────╯
+    
+ + + +.. raw:: html + +
+    $ python 04_decorator_subcommands.py greet --help
+    usage: 04_decorator_subcommands.py greet [-h] --name STR [--loud | --no-loud]
+    
+    Greet someone.
+    
+    ╭─ options ───────────────────────────────────────────────╮
+    │ -h, --help              show this help message and exit │
+    │ --name STR              (required)                      │
+    │ --loud, --no-loud       (default: None)                 │
+    ╰─────────────────────────────────────────────────────────╯
+    
+ + + +.. raw:: html + +
+    $ python 04_decorator_subcommands.py greet --name Alice
+    Hello, Alice!
+    
+ + + +.. raw:: html + +
+    $ python 04_decorator_subcommands.py greet --name Bob --loud
+    HELLO, BOB!
+    
+ + + +.. raw:: html + +
+    $ python 04_decorator_subcommands.py addition --help
+    usage: 04_decorator_subcommands.py addition [-h] --a INT --b INT
+    
+    Add two numbers.
+    
+    ╭─ options ─────────────────────────────────────────╮
+    │ -h, --help        show this help message and exit │
+    │ --a INT           (required)                      │
+    │ --b INT           (required)                      │
+    ╰───────────────────────────────────────────────────╯
+    
+ + + +.. raw:: html + +
+    $ python 04_decorator_subcommands.py addition --a 5 --b 3
+    5 + 3 = 8
+    
+.. _example-05_subcommands_func: + +Subcommands from Functions +-------------------------- + +We provide a shorthand for generating a subcommand CLI from a dictionary. This +is a thin wrapper around :func:`tyro.cli()`'s more verbose, type-based API. If +more generality is needed, the internal working are explained in the docs for +:func:`tyro.extras.subcommand_cli_from_dict()`. + + +.. code-block:: python + :linenos: + + # 05_subcommands_func.py + import tyro + + def checkout(branch: str) -> None: + """Check out a branch.""" + print(f"{branch=}") + + def commit(message: str, all: bool = False) -> None: + """Make a commit.""" + print(f"{message=} {all=}") + + if __name__ == "__main__": + tyro.extras.subcommand_cli_from_dict( + { + "checkout": checkout, + "commit": commit, + } + ) + + + + +.. raw:: html + +
+    $ python ./05_subcommands_func.py --help
+    usage: 05_subcommands_func.py [-h] {checkout,commit}
+    
+    ╭─ options ───────────────────────────────────────────────╮
+    │ -h, --help              show this help message and exit │
+    ╰─────────────────────────────────────────────────────────╯
+    ╭─ subcommands ───────────────────────────────────────────╮
+    │ {checkout,commit}                                       │
+    │     checkout            Check out a branch.             │
+    │     commit              Make a commit.                  │
+    ╰─────────────────────────────────────────────────────────╯
+    
+ + + +.. raw:: html + +
+    $ python ./05_subcommands_func.py commit --help
+    usage: 05_subcommands_func.py commit [-h] --message STR [--all | --no-all]
+    
+    Make a commit.
+    
+    ╭─ options ──────────────────────────────────────────────╮
+    │ -h, --help             show this help message and exit │
+    │ --message STR          (required)                      │
+    │ --all, --no-all        (default: None)                 │
+    ╰────────────────────────────────────────────────────────╯
+    
+ + + +.. raw:: html + +
+    $ python ./05_subcommands_func.py commit --message hello --all
+    message='hello' all=True
+    
+ + + +.. raw:: html + +
+    $ python ./05_subcommands_func.py checkout --help
+    usage: 05_subcommands_func.py checkout [-h] --branch STR
+    
+    Check out a branch.
+    
+    ╭─ options ───────────────────────────────────────────╮
+    │ -h, --help          show this help message and exit │
+    │ --branch STR        (required)                      │
+    ╰─────────────────────────────────────────────────────╯
+    
+ + + +.. raw:: html + +
+    $ python ./05_subcommands_func.py checkout --branch main
+    branch='main'
+    
\ No newline at end of file diff --git a/docs/source/goals_and_alternatives.md b/docs/source/goals_and_alternatives.md index b8dc5484..8ab17807 100644 --- a/docs/source/goals_and_alternatives.md +++ b/docs/source/goals_and_alternatives.md @@ -14,7 +14,7 @@ Usage distinctions are the result of two API goals: - In contrast, similar libraries have more expansive APIs , and require more library-specific structures, decorators, or metadata formats for configuring parsing behavior. -- **Strict typing.** Any type that can be annotated and unambiguously parsed +- **Types.** Any type that can be annotated and unambiguously parsed with an `argparse`-style CLI interface should work out-of-the-box; any public API that isn't statically analyzable should be avoided. - In contrast, many similar libraries implement features that depend on diff --git a/docs/source/helptext_generation.md b/docs/source/helptext_generation.md index 8b46e588..5efffdef 100644 --- a/docs/source/helptext_generation.md +++ b/docs/source/helptext_generation.md @@ -12,10 +12,7 @@ and Epydoc docstrings are supported as well. Under the hood, all of these options use [docstring_parser](https://github.com/rr-/docstring_parser). ```python -def main( - field1: str, - field2: int = 3, -) -> None: +def main(field1: str, field2: int = 3) -> None: """Function, whose arguments will be populated from a CLI interface. Args: diff --git a/docs/source/index.md b/docs/source/index.md index dbcbae06..cb95fefd 100644 --- a/docs/source/index.md +++ b/docs/source/index.md @@ -2,8 +2,7 @@ |build| |nbsp| |ruff| |nbsp| |mypy| |nbsp| |pyright| |nbsp| |coverage| |nbsp| |versions| -:func:`tyro.cli()` is a tool for generating CLI -interfaces. +:func:`tyro.cli()` is a tool for generating CLI interfaces in Python. We can define configurable scripts using functions: @@ -93,53 +92,17 @@ shell completion. supported_types .. toctree:: - :caption: Basics + :caption: Examples :hidden: - :maxdepth: 1 - :titlesonly: - :glob: - - examples/01_basics/* - - -.. toctree:: - :caption: Hierarchies - :hidden: - :maxdepth: 1 - :titlesonly: - :glob: - - examples/02_nesting/* - - -.. toctree:: - :caption: Config Management - :hidden: - :maxdepth: 1 :titlesonly: - :glob: - - examples/03_config_systems/* - - -.. toctree:: - :caption: Additional Features - :hidden: - :maxdepth: 1 - :titlesonly: - :glob: - - examples/04_additional/* - - -.. toctree:: - :caption: Custom Constructors - :hidden: - :maxdepth: 1 - :titlesonly: - :glob: - examples/05_custom_constructors/* + ./examples/basics.rst + ./examples/nested_structures.rst + ./examples/subcommands.rst + ./examples/overriding_configs.rst + ./examples/generics.rst + ./examples/custom_constructors.rst + ./examples/pytorch_jax.rst .. toctree:: diff --git a/docs/source/supported_types.rst b/docs/source/supported_types.rst index 3fdc1707..772d1e95 100644 --- a/docs/source/supported_types.rst +++ b/docs/source/supported_types.rst @@ -29,7 +29,7 @@ What's not supported -------------------- -There are some limitations. We currently _do not_ support: +There are some limitations. We currently *do not* support: - Variable-length sequences over nested structures, unless a default is provided. For types like ``list[Dataclass]``, we require a default value to @@ -41,6 +41,5 @@ There are some limitations. We currently _do not_ support: ambiguous to parse and not supported. - Self-referential types, like ``type RecursiveList[T] = T | list[RecursiveList[T]]``. -In each of these cases, a `custom -constructor `_ -can be defined as a workaround. +In each of these cases, a :ref:`custom constructor +` can be defined as a workaround. diff --git a/docs/update_example_docs.py b/docs/update_example_docs.py index 475b9dbe..13b61c0c 100644 --- a/docs/update_example_docs.py +++ b/docs/update_example_docs.py @@ -1,14 +1,111 @@ +# mypy: ignore-errors """Helper script for updating the auto-generated examples pages in the documentation.""" from __future__ import annotations import dataclasses +import io +import os import pathlib +import pty +import re import shlex import shutil -from typing import Iterable +import subprocess +from pathlib import Path +from typing import Iterable, Optional, Tuple, Union import tyro +from tqdm import tqdm + + +def command_to_rst( + command: list[str], + cwd: Optional[Union[str, Path]] = None, + shell: bool = False, +) -> Tuple[str, int]: + """ + Run a command and format its output (including ANSI codes) as RST with HTML colors. + Uses a pseudo-terminal to ensure ANSI color codes are output. + + Args: + command: The command to run (string or list of strings) + cwd: Working directory to run the command in (str or Path) + shell: Whether to run the command through the shell + + Returns: + Tuple of (formatted RST string, return code) + """ + # Convert cwd to Path if provided + if cwd is not None: + cwd = Path(cwd).expanduser().resolve() + if not cwd.exists(): + raise FileNotFoundError(f"Working directory does not exist: {cwd}") + if not cwd.is_dir(): + raise NotADirectoryError( + f"Working directory path is not a directory: {cwd}" + ) + + # Create a pseudo-terminal to capture colored output + m, s = pty.openpty() + + # Set terminal type to ensure color support + env = os.environ.copy() + env["TERM"] = "xterm-256color" + process = subprocess.Popen( + command, + stdout=s, + stderr=s, + cwd=cwd, + shell=shell, + env=env, + close_fds=True, + ) + + os.close(s) + + # Read output from the master end of the pseudo-terminal + output = io.BytesIO() + while True: + try: + data = os.read(m, 1024) + if not data: + break + output.write(data) + except OSError: + break + + os.close(m) + process.wait() + + # Decode the captured output + output_str = output.getvalue().decode("utf-8", errors="replace") + + from ansi2html import Ansi2HTMLConverter + + # Create an Ansi2HTMLConverter instance + converter = Ansi2HTMLConverter(inline=True, scheme="osx-basic") + + # Convert ANSI codes to HTML + html_output = converter.convert(output_str, full=False) + + # Clean up any remaining ANSI codes (just in case) + output_str = re.sub("\x1b\\[[0-9;]*[mGKH]", "", html_output) + + # Format as RST code block with HTML + rst_output = ".. raw:: html\n\n" + rst_output += '
\n'
+    rst_output += f'    $ {shlex.join(command)}\n'
+
+    # Indent and HTML-escape the content
+    for line in output_str.splitlines():
+        # We don't escape < and > because we want to preserve HTML tags
+        # but we do escape & to prevent XML entities from being interpreted
+        rst_output += f"    {line}\n"
+
+    rst_output += "    
\n" + + return rst_output, process.returncode @dataclasses.dataclass @@ -17,7 +114,7 @@ class ExampleMetadata: index_with_zero: str source: str title: str - usages: Iterable[str] + usages: tuple[tuple[str, str]] # (comment, command) description: str @staticmethod @@ -37,19 +134,24 @@ def from_path(path: pathlib.Path) -> ExampleMetadata: title, _, description = docstring.partition("\n") description, _, usage_text = description.partition("Usage:") - example_usages = map( - lambda x: x[1:-1], - filter( - lambda line: line.startswith("`") and line.endswith("`"), - usage_text.split("\n"), - ), - ) + example_usages: list[tuple[str, str]] = [] + comment = "" + for usage_line in usage_text.splitlines(): + usage_line = usage_line.strip() + if usage_line.startswith("#"): + comment += usage_line.strip().lstrip("#").strip() + "\n" + elif usage_line.startswith("python"): + example_usages.append((comment.strip(), usage_line)) + comment = "" + # else: + # assert len(usage_line) == 0, usage_line + return ExampleMetadata( index=index, index_with_zero=index_with_zero, source=source.partition('"""')[2].partition('"""')[2].strip(), title=title, - usages=example_usages, + usages=tuple(example_usages), description=description.strip(), ) @@ -67,46 +169,97 @@ def main( examples_dir: pathlib.Path = REPO_ROOT / "examples", sphinx_source_dir: pathlib.Path = REPO_ROOT / "docs" / "source", ) -> None: + print("\nStarting documentation update...") + print(f"Examples directory: {examples_dir}") + print(f"Sphinx source directory: {sphinx_source_dir}") + example_doc_dir = sphinx_source_dir / "examples" + print(f"Cleaning up old docs directory: {example_doc_dir}") shutil.rmtree(example_doc_dir) example_doc_dir.mkdir() - for path in get_example_paths(examples_dir): - ex = ExampleMetadata.from_path(path) - path_for_sphinx = pathlib.Path("..") / ".." / path.relative_to(REPO_ROOT) + category_set: set[str] = set() - usage_lines = [] - for usage in ex.usages: - args = shlex.split(usage) - python_index = args.index("python") - sphinx_usage = shlex.join( - args[:python_index] - + ["python", path_for_sphinx.as_posix()] - + args[python_index + 2 :] - ) + from concurrent.futures import ThreadPoolExecutor - # Note that :kbd: in Sphinx does unnecessary stuff we want to avoid, see: - # https://github.com/sphinx-doc/sphinx/issues/7530 - # - # Instead, we just use raw HTML. - assert "../../examples/" in sphinx_usage - command = sphinx_usage.replace("../../examples/", "") + def process_example(path, examples_dir, REPO_ROOT): + print(f"Processing example: {path.name}") + ex = ExampleMetadata.from_path(path) + usage_lines = [] + for comment, usage in ex.usages: usage_lines += [ - "------------", "", - ".. raw:: html", + f"{comment}", + "", + ] + command_to_rst(shlex.split(usage), cwd=path.parent)[0].splitlines() + + category = path.parent.name + + example_content = "\n".join( + [ + f".. _example-{path.stem}:", + "", + f"{ex.title}", + "-" * len(ex.title), "", - f" {command}", + ex.description, "", - f".. program-output:: {sphinx_usage}", + "", + ".. code-block:: python", + " :linenos:", + "", + f" # {path.name}", + "\n".join( + f" {line}".rstrip() + for line in ex.source.replace("\n\n\n", "\n\n").splitlines() + ), "", ] + + usage_lines + ) + return category, example_content + + with ThreadPoolExecutor() as executor: + futures = [] + for path in get_example_paths(examples_dir): + futures.append( + executor.submit(process_example, path, examples_dir, REPO_ROOT) + ) + + category_set = set() + example_contents: dict[str, str] = {} + for future in tqdm(futures, total=len(futures)): + category, content = future.result() + category_set.add(category) + if category not in example_contents: + example_contents[category] = [] + example_contents[category].append(content) - relative_dir = path.parent.relative_to(examples_dir) - target_dir = example_doc_dir / relative_dir - target_dir.mkdir(exist_ok=True, parents=True) + print(f"\nProcessing {len(category_set)} categories...") + for category in category_set: + print(f"\nProcessing category: {category}") - (target_dir / f"{path.stem}.rst").write_text( + example_category_dir = examples_dir / category + readme_path = example_category_dir / "README.rst" + if readme_path.exists(): + print("Found", readme_path) + readme_content = readme_path.read_text() + else: + category_title = " ".join(category.split("_")[1:]).title() + readme_content = "\n".join( + [ + f"{category_title}", + "=" * len(category_title), + ] + ) + + # 0_basics => basics + number, _, category_clean = category.partition("_") + int(number) + + output_file = example_doc_dir / f"{category_clean}.rst" + print(f"Writing documentation to: {output_file}") + output_file.write_text( "\n".join( [ ( @@ -115,25 +268,19 @@ def main( ), " It should not be modified manually.", "", - f"{ex.title}", - "==========================================", - "", - ex.description, + f".. _example-category-{category_clean}:", "", - "", - ".. code-block:: python", - " :linenos:", - "", - "", - "\n".join( - f" {line}".rstrip() for line in ex.source.split("\n") - ), + readme_content, "", ] - + usage_lines - ) + + example_contents[category] + ), + encoding="utf-8", ) + output_file.parent.mkdir(parents=True, exist_ok=True) if __name__ == "__main__": + print("Starting example documentation generator...") tyro.cli(main, description=__doc__) + print("\nDocumentation generation complete!") diff --git a/examples/01_basics/01_functions.py b/examples/01_basics/01_functions.py index d9fe1741..b12013bc 100644 --- a/examples/01_basics/01_functions.py +++ b/examples/01_basics/01_functions.py @@ -4,18 +4,19 @@ arguments populated from the CLI. Usage: -`python ./01_functions.py --help` -`python ./01_functions.py --field1 hello` -`python ./01_functions.py --field1 hello --field2 10` + + # We can use ``--help`` to show the help message, or ``--field1`` and + # ``--field2`` to set the arguments: + python ./01_functions.py --help + python ./01_functions.py --field1 hello + python ./01_functions.py --field1 hello --field2 10 + """ import tyro -def main( - field1: str, - field2: int = 3, -) -> None: +def main(field1: str, field2: int = 3) -> None: """Function, whose arguments will be populated from a CLI interface. Args: diff --git a/examples/01_basics/02_dataclasses.py b/examples/01_basics/02_dataclasses.py index 09e94bbf..4c41da27 100644 --- a/examples/01_basics/02_dataclasses.py +++ b/examples/01_basics/02_dataclasses.py @@ -1,19 +1,24 @@ """Dataclasses -Common pattern: use :func:`tyro.cli()` to instantiate a dataclass. +In addition to functions, :func:`tyro.cli()` can also take dataclasses as input. Usage: -`python ./02_dataclasses.py --help` -`python ./02_dataclasses.py --field1 hello` -`python ./02_dataclasses.py --field1 hello --field2 5` + + # To show the help message, we can use the ``--help`` flag: + python ./02_dataclasses.py --help + + # We can override ``field1`` and ``field2``: + python ./02_dataclasses.py --field1 hello + python ./02_dataclasses.py --field1 hello --field2 5 """ -import dataclasses +from dataclasses import dataclass +from pprint import pprint import tyro -@dataclasses.dataclass +@dataclass class Args: """Description. This should show up in the helptext!""" @@ -27,4 +32,4 @@ class Args: if __name__ == "__main__": args = tyro.cli(Args) - print(args) + pprint(args) diff --git a/examples/01_basics/03_dataclasses_defaults.py b/examples/01_basics/03_dataclasses_defaults.py deleted file mode 100644 index 9b207e8c..00000000 --- a/examples/01_basics/03_dataclasses_defaults.py +++ /dev/null @@ -1,55 +0,0 @@ -"""Dataclasses + Defaults - -The :code:`default=` argument can be used to override default values in dataclass -types. - - -.. warning:: - - We advise against mutation of configuration objects from a dataclass's - :code:`__post_init__` method [#f1]_. In the example below, - :code:`__post_init__` would be called twice: once for the :code:`Args()` - object provided as a default value and another time for the :code:`Args()` - objected instantiated by :func:`tyro.cli()`. This can cause confusing - behavior! Instead, we show below one example of how derived fields can be - defined immutably. - - .. [#f1] Official Python docs for ``__post_init__`` can be found `here `_. - - -Usage: -`python ./03_dataclasses_defaults.py --help` -`python ./03_dataclasses_defaults.py --field2 3` -`python ./03_dataclasses_defaults.py --field1 hello --field2 5` -""" - -import dataclasses - -import tyro - - -@dataclasses.dataclass -class Args: - """Description. - This should show up in the helptext!""" - - field1: str - """A string field.""" - - field2: int = 3 - """A numeric field, with a default value.""" - - @property - def derived_field(self) -> str: - return ", ".join([self.field1] * self.field2) - - -if __name__ == "__main__": - args = tyro.cli( - Args, - default=Args( - field1="default string", - field2=tyro.MISSING, - ), - ) - print(args.derived_field) diff --git a/examples/01_basics/03_multivalue.py b/examples/01_basics/03_multivalue.py new file mode 100644 index 00000000..10d05b3b --- /dev/null +++ b/examples/01_basics/03_multivalue.py @@ -0,0 +1,39 @@ +"""Multi-value Arguments + +Arguments of both fixed and variable lengths can be annotated with standard +Python collection types. For Python 3.7 and 3.8, we can use either ``from +__future__ import annotations`` to support ``list[T]`` and ``tuple[T]``, +or the older :py:class:`typing.List` and :py:data:`typing.Tuple`. + +Usage: + + # To print help: + python ./03_multivalue.py --help + + # We can override arguments: + python ./03_multivalue.py --source-paths ./data --dimensions 16 16 + python ./03_multivalue.py --source-paths ./data1 ./data2 +""" + +import pathlib +from dataclasses import dataclass +from pprint import pprint + +import tyro + + +@dataclass +class Config: + # Example of a variable-length tuple. `list[T]`, `set[T]`, + # `dict[K, V]`, etc are supported as well. + source_paths: tuple[pathlib.Path, ...] + """This can be multiple!""" + + # Fixed-length tuples are also okay. + dimensions: tuple[int, int] = (32, 32) + """Height and width.""" + + +if __name__ == "__main__": + config = tyro.cli(Config) + pprint(config) diff --git a/examples/04_additional/04_classes.py b/examples/01_basics/04_classes.py similarity index 86% rename from examples/04_additional/04_classes.py rename to examples/01_basics/04_classes.py index 5dbae463..5c977e14 100644 --- a/examples/04_additional/04_classes.py +++ b/examples/01_basics/04_classes.py @@ -4,8 +4,9 @@ constructors of standard Python classes. Usage: -`python ./04_classes.py --help` -`python ./04_classes.py --field1 hello --field2 7` + + python ./04_classes.py --help + python ./04_classes.py --field1 hello --field2 7 """ import tyro diff --git a/examples/01_basics/04_collections.py b/examples/01_basics/04_collections.py deleted file mode 100644 index a3f3b836..00000000 --- a/examples/01_basics/04_collections.py +++ /dev/null @@ -1,34 +0,0 @@ -"""Multi-value Arguments - -Arguments of both fixed and variable lengths can be annotated with standard -Python collection types. For Python 3.7 and 3.8, we can use either :code:`from -__future__ import annotations` to support :code:`list[T]` and :code:`tuple[T]`, -or the older API :code:`typing.List[T]` and :code:`typing.Tuple[T1, T2]`. - -Usage: -`python ./03_collections.py --help` -`python ./03_collections.py --dataset-sources ./data --image-dimensions 16 16` -`python ./03_collections.py --dataset-sources ./data` -""" - -import dataclasses -import pathlib - -import tyro - - -@dataclasses.dataclass(frozen=True) -class TrainConfig: - # Example of a variable-length tuple. `list[T]`, `set[T]`, - # `dict[K, V]`, etc are supported as well. - dataset_sources: tuple[pathlib.Path, ...] - """Paths to load training data from. This can be multiple!""" - - # Fixed-length tuples are also okay. - image_dimensions: tuple[int, int] = (32, 32) - """Height and width of some image data.""" - - -if __name__ == "__main__": - config = tyro.cli(TrainConfig) - print(config) diff --git a/examples/01_basics/05_flags.py b/examples/01_basics/04_flags.py similarity index 70% rename from examples/01_basics/05_flags.py rename to examples/01_basics/04_flags.py index 11c07b8b..ff5f45ca 100644 --- a/examples/01_basics/05_flags.py +++ b/examples/01_basics/04_flags.py @@ -6,18 +6,20 @@ To turn off conversion, see :class:`tyro.conf.FlagConversionOff`. Usage: -`python ./05_flags.py --help` -`python ./05_flags.py --boolean True` -`python ./05_flags.py --boolean False --flag-a` -`python ./05_flags.py --boolean False --no-flag-b` + + python ./04_flags.py --help + python ./04_flags.py --boolean True + python ./04_flags.py --boolean False --flag-a + python ./04_flags.py --boolean False --no-flag-b """ -import dataclasses +from dataclasses import dataclass +from pprint import pprint import tyro -@dataclasses.dataclass +@dataclass class Args: # Boolean. This expects an explicit "True" or "False". boolean: bool @@ -34,4 +36,4 @@ class Args: if __name__ == "__main__": args = tyro.cli(Args) - print(args) + pprint(args) diff --git a/examples/01_basics/06_literals.py b/examples/01_basics/05_choices.py similarity index 61% rename from examples/01_basics/06_literals.py rename to examples/01_basics/05_choices.py index 5f3017d8..602eaef9 100644 --- a/examples/01_basics/06_literals.py +++ b/examples/01_basics/05_choices.py @@ -3,25 +3,29 @@ :code:`typing.Literal[]` can be used to restrict inputs to a fixed set of literal choices. Usage: -`python ./06_literals.py --help` + + python ./05_choices.py --help + python ./05_choices.py --string red + python ./05_choices.py --string blue """ import dataclasses +from pprint import pprint from typing import Literal import tyro -@dataclasses.dataclass(frozen=True) +@dataclasses.dataclass class Args: # We can use Literal[] to restrict the set of allowable inputs, for example, over # a set of strings. - strings: Literal["red", "green"] = "red" + string: Literal["red", "green"] = "red" # Integers also work. (as well as booleans, enums, etc) - numbers: Literal[0, 1, 2] = 0 + number: Literal[0, 1, 2] = 0 if __name__ == "__main__": args = tyro.cli(Args) - print(args) + pprint(args) diff --git a/examples/01_basics/06_enums.py b/examples/01_basics/06_enums.py new file mode 100644 index 00000000..749e26bf --- /dev/null +++ b/examples/01_basics/06_enums.py @@ -0,0 +1,36 @@ +"""Enums + +In addition to literals, enums can also be used to provide a fixed set of +choices. + +Usage: + + python ./06_enums.py --help + python ./06_enums.py --color RED + python ./06_enums.py --color BLUE --opacity 0.75 +""" + +import enum +from dataclasses import dataclass +from pprint import pprint + +import tyro + + +class Color(enum.Enum): + RED = enum.auto() + BLUE = enum.auto() + + +@dataclass +class Config: + color: Color = Color.RED + """Color argument.""" + + opacity: float = 0.5 + """Opacity argument.""" + + +if __name__ == "__main__": + config = tyro.cli(Config) + pprint(config) diff --git a/examples/01_basics/07_unions.py b/examples/01_basics/07_unions.py index f99ffb04..3289d0e0 100644 --- a/examples/01_basics/07_unions.py +++ b/examples/01_basics/07_unions.py @@ -4,11 +4,17 @@ multiple types. Usage: -`python ./07_unions.py --help` + + python ./07_unions.py --help + python ./07_unions.py --union-over-types 3 + python ./07_unions.py --union-over-types three + python ./07_unions.py --integer None + python ./07_unions.py --integer 0 """ import dataclasses import enum +from pprint import pprint from typing import Literal, Optional import tyro @@ -41,4 +47,4 @@ class Args: if __name__ == "__main__": args = tyro.cli(Args) - print(args) + pprint(args) diff --git a/examples/01_basics/08_enums.py b/examples/01_basics/08_enums.py deleted file mode 100644 index 0fa47eed..00000000 --- a/examples/01_basics/08_enums.py +++ /dev/null @@ -1,35 +0,0 @@ -"""Enums - -In addition to literals, enums can also be used to provide a fixed set of -choices. - -Usage: -`python ./04_enums.py --help` -`python ./04_enums.py --optimizer-type SGD` -`python ./04_enums.py --optimizer-type ADAM --learning-rate 3e-4` -""" - -import dataclasses -import enum - -import tyro - - -class OptimizerType(enum.Enum): - ADAM = enum.auto() - SGD = enum.auto() - - -@dataclasses.dataclass(frozen=True) -class TrainConfig: - # Enums are handled seamlessly. - optimizer_type: OptimizerType = OptimizerType.ADAM - """Gradient-based optimizer to use.""" - - learning_rate: float = 1e-4 - """Learning rate for optimizer.""" - - -if __name__ == "__main__": - config = tyro.cli(TrainConfig) - print(config) diff --git a/examples/01_basics/08_positional.py b/examples/01_basics/08_positional.py new file mode 100644 index 00000000..6380b90b --- /dev/null +++ b/examples/01_basics/08_positional.py @@ -0,0 +1,39 @@ +"""Positional Arguments + +Positional-only arguments in functions are converted to positional CLI arguments. + +For more general positional arguments, see :class:`tyro.conf.Positional`. + +Usage: + + python 08_positional.py --help + python 08_positional.py ./a ./b + python 08_positional.py ./test1 ./test2 --verbose +""" + +from __future__ import annotations + +import pathlib + +import tyro + + +def main( + source: pathlib.Path, + dest: pathlib.Path, + /, # Mark the end of positional arguments. + verbose: bool = False, +) -> None: + """Command-line interface defined using a function signature. Note that this + docstring is parsed to generate helptext. + + Args: + source: Source path. + dest: Destination path. + verbose: Explain what is being done. + """ + print(f"{source=}\n{dest=}\n{verbose=}") + + +if __name__ == "__main__": + tyro.cli(main) diff --git a/examples/04_additional/07_conf.py b/examples/01_basics/09_conf.py similarity index 81% rename from examples/04_additional/07_conf.py rename to examples/01_basics/09_conf.py index 479a888a..d87f8809 100644 --- a/examples/04_additional/07_conf.py +++ b/examples/01_basics/09_conf.py @@ -1,13 +1,15 @@ """Configuration via typing.Annotated[] -The :mod:`tyro.conf` module contains utilities that can be used to configure -command-line interfaces beyond what is expressible via static type annotations. +The :mod:`tyro.conf` module contains utilities that can be used in conjunction +with :py:data:`typing.Annotated` to configure command-line interfaces beyond +what is expressible via static type annotations. Features here are supported, but generally unnecessary and should be used sparingly. Usage: -`python ./06_conf.py --help` -`python ./06_conf.py 5 --boolean True` + + python ./09_conf.py --help + python ./09_conf.py 5 --boolean True """ import dataclasses diff --git a/examples/01_basics/10_aliases.py b/examples/01_basics/10_aliases.py new file mode 100644 index 00000000..3c534be9 --- /dev/null +++ b/examples/01_basics/10_aliases.py @@ -0,0 +1,25 @@ +"""Argument Aliases + +:func:`tyro.conf.arg()` can be used to attach aliases to arguments. + +Usage: + + python ./10_aliases.py --help + python ./10_aliases.py --branch main + python ./10_aliases.py -b main +""" + +from typing import Annotated + +import tyro + + +def checkout( + branch: Annotated[str, tyro.conf.arg(aliases=["-b"])], +) -> None: + """Check out a branch.""" + print(f"{branch=}") + + +if __name__ == "__main__": + tyro.cli(checkout) diff --git a/examples/04_additional/12_type_statement.py b/examples/01_basics/11_type_aliases_py312.py similarity index 89% rename from examples/04_additional/12_type_statement.py rename to examples/01_basics/11_type_aliases_py312.py index fa814029..c8daad52 100644 --- a/examples/04_additional/12_type_statement.py +++ b/examples/01_basics/11_type_aliases_py312.py @@ -1,12 +1,13 @@ # mypy: ignore-errors # # PEP 695 isn't yet supported in mypy. (April 4, 2024) -"""Type Aliases (Python 3.12+) +"""Type Aliases (3.12+) In Python 3.12, the :code:`type` statement is introduced to create type aliases. Usage: -`python ./12_type_statement.py --help` + + python ./11_type_aliases_py312.py --help """ import dataclasses diff --git a/examples/04_additional/13_counters.py b/examples/01_basics/12_counters.py similarity index 81% rename from examples/04_additional/13_counters.py rename to examples/01_basics/12_counters.py index 83562b1a..7d9b6374 100644 --- a/examples/04_additional/13_counters.py +++ b/examples/01_basics/12_counters.py @@ -3,10 +3,11 @@ Repeatable 'counter' arguments can be specified via :data:`tyro.conf.UseCounterAction`. Usage: -`python ./13_counters.py --help` -`python ./13_counters.py --verbosity` -`python ./13_counters.py --verbosity --verbosity` -`python ./13_counters.py -vvv` + + python ./12_counters.py --help + python ./12_counters.py --verbosity + python ./12_counters.py --verbosity --verbosity + python ./12_counters.py -vvv """ from typing_extensions import Annotated diff --git a/examples/01_basics/README.rst b/examples/01_basics/README.rst new file mode 100644 index 00000000..bc3b1d23 --- /dev/null +++ b/examples/01_basics/README.rst @@ -0,0 +1,5 @@ +Basics +====== + +In these examples, we show basic examples of using :func:`tyro.cli`: functions, +dataclasses, supported annotations, and configuration. diff --git a/examples/02_nested_structures/01_nesting.py b/examples/02_nested_structures/01_nesting.py new file mode 100644 index 00000000..32128d29 --- /dev/null +++ b/examples/02_nested_structures/01_nesting.py @@ -0,0 +1,35 @@ +"""Nested Dataclasses + +Structures (typically :py:func:`dataclasses.dataclass`) can be nested to build hierarchical configuration +objects. This helps with modularity and grouping in larger projects. + +Usage: + + python ./01_nesting.py --help + python ./01_nesting.py --opt.learning-rate 1e-3 + python ./01_nesting.py --seed 4 +""" + +import dataclasses + +import tyro + + +@dataclasses.dataclass +class OptimizerConfig: + learning_rate: float = 3e-4 + weight_decay: float = 1e-2 + + +@dataclasses.dataclass +class Config: + # Optimizer options. + opt: OptimizerConfig + + # Random seed. + seed: int = 0 + + +if __name__ == "__main__": + config = tyro.cli(Config) + print(dataclasses.asdict(config)) diff --git a/examples/02_nested_structures/02_nesting_in_func.py b/examples/02_nested_structures/02_nesting_in_func.py new file mode 100644 index 00000000..da0c81f0 --- /dev/null +++ b/examples/02_nested_structures/02_nesting_in_func.py @@ -0,0 +1,48 @@ +"""Structures as Function Arguments + +Structures can also be used as input to functions. + +Usage: + + python ./02_nesting_in_func.py --help + python ./02_nesting_in_func.py --out-dir /tmp/test1 + python ./02_nesting_in_func.py --out-dir /tmp/test2 --config.seed 4 +""" + +import dataclasses +import pathlib + +import tyro + + +@dataclasses.dataclass +class OptimizerConfig: + learning_rate: float = 3e-4 + weight_decay: float = 1e-2 + + +@dataclasses.dataclass +class Config: + # Optimizer options. + optimizer: OptimizerConfig + + # Random seed. + seed: int = 0 + + +def train( + out_dir: pathlib.Path, + config: Config, +) -> None: + """Train a model. + + Args: + out_dir: Where to save logs and checkpoints. + config: Experiment configuration. + """ + print(f"Saving to: {out_dir}") + print(f"Config: f{config}") + + +if __name__ == "__main__": + tyro.cli(train) diff --git a/examples/02_nested_structures/03_nesting_containers.py b/examples/02_nested_structures/03_nesting_containers.py new file mode 100644 index 00000000..9f3ecd08 --- /dev/null +++ b/examples/02_nested_structures/03_nesting_containers.py @@ -0,0 +1,44 @@ +"""Nesting in Containers + +Structures can be nested inside of standard containers. + +.. warning:: + + When placing structures inside of containers like lists or tuples, the + length of the container must be inferrable from the annotation or default + value. + + +Usage: + + python ./03_nesting_containers.py --help +""" + +import dataclasses + +import tyro + + +@dataclasses.dataclass +class RGB: + r: int + g: int + b: int + + +@dataclasses.dataclass +class Args: + color_tuple: tuple[RGB, RGB] + color_dict: dict[str, RGB] = dataclasses.field( + # We can't use mutable values as defaults directly. + default_factory={ + "red": RGB(255, 0, 0), + "green": RGB(0, 255, 0), + "blue": RGB(0, 0, 255), + }.copy + ) + + +if __name__ == "__main__": + args = tyro.cli(Args) + print(args) diff --git a/examples/04_additional/02_dictionaries.py b/examples/02_nested_structures/04_dictionaries.py similarity index 82% rename from examples/04_additional/02_dictionaries.py rename to examples/02_nested_structures/04_dictionaries.py index 9b0bac6f..b1fa5108 100644 --- a/examples/04_additional/02_dictionaries.py +++ b/examples/02_nested_structures/04_dictionaries.py @@ -1,15 +1,16 @@ """Dictionaries and TypedDict -Dictionary inputs can be specified using either a standard `Dict[K, V]` +Dictionary inputs can be specified using either a standard ``dict[K, V]`` annotation, or a :code:`TypedDict` subclass. For configuring :code:`TypedDict`, we also support :code:`total={True/False}`, :code:`typing.Required`, and :code:`typing.NotRequired`. See the `Python docs `_ for all :code:`TypedDict` features. Usage: -`python ./02_dictionaries.py --help` -`python ./02_dictionaries.py --typed-dict-a.learning-rate 3e-4 --typed-dict-b.betas 0.9 0.999` -`python ./02_dictionaries.py --typed-dict-b.betas 0.9 0.999` + + python ./04_dictionaries.py --help + python ./04_dictionaries.py --typed-dict-a.learning-rate 3e-4 --typed-dict-b.betas 0.9 0.999 + python ./04_dictionaries.py --typed-dict-b.betas 0.9 0.999 """ from typing import TypedDict diff --git a/examples/04_additional/03_tuples.py b/examples/02_nested_structures/05_tuples.py similarity index 71% rename from examples/04_additional/03_tuples.py rename to examples/02_nested_structures/05_tuples.py index ef40280c..d9e83981 100644 --- a/examples/04_additional/03_tuples.py +++ b/examples/02_nested_structures/05_tuples.py @@ -1,12 +1,13 @@ -"""Tuples +"""Tuples and NamedTuple Example using :func:`tyro.cli()` to instantiate tuple types. :code:`tuple`, -:code:`typing.Tuple`, and :code:`NamedTuple` are all supported. +:py:data:`typing.Tuple`, and :py:class:`typing.NamedTuple` are all supported. Usage: -`python ./03_tuples.py --help` -`python ./03_tuples.py --color 127 127 127` -`python ./03_tuples.py --two-colors.1.r 127 --two-colors.1.g 0 --two-colors.1.b 0` + + python ./05_tuples.py --help + python ./05_tuples.py --color 127 127 127 + python ./05_tuples.py --two-colors.1.r 127 --two-colors.1.g 0 --two-colors.1.b 0 """ from typing import NamedTuple diff --git a/examples/04_additional/08_pydantic.py b/examples/02_nested_structures/06_pydantic.py similarity index 77% rename from examples/04_additional/08_pydantic.py rename to examples/02_nested_structures/06_pydantic.py index e9c5ee45..740eeac4 100644 --- a/examples/04_additional/08_pydantic.py +++ b/examples/02_nested_structures/06_pydantic.py @@ -4,9 +4,10 @@ `Pydantic `_ models. Usage: -`python ./08_pydantic.py --help` -`python ./08_pydantic.py --field1 hello` -`python ./08_pydantic.py --field1 hello --field2 5` + + python ./06_pydantic.py --help + python ./06_pydantic.py --field1 hello + python ./06_pydantic.py --field1 hello --field2 5 """ from pydantic import BaseModel, Field diff --git a/examples/04_additional/09_attrs.py b/examples/02_nested_structures/07_attrs.py similarity index 79% rename from examples/04_additional/09_attrs.py rename to examples/02_nested_structures/07_attrs.py index effe864e..30d93488 100644 --- a/examples/04_additional/09_attrs.py +++ b/examples/02_nested_structures/07_attrs.py @@ -4,9 +4,10 @@ `attrs `_ classes. Usage: -`python ./09_attrs.py --help` -`python ./09_attrs.py --field1 hello` -`python ./09_attrs.py --field1 hello --field2 5` + + python ./07_attrs.py --help + python ./07_attrs.py --field1 hello + python ./07_attrs.py --field1 hello --field2 5 """ import attr diff --git a/examples/02_nested_structures/README.rst b/examples/02_nested_structures/README.rst new file mode 100644 index 00000000..8ec6ad9b --- /dev/null +++ b/examples/02_nested_structures/README.rst @@ -0,0 +1,6 @@ +Nested Structures +================= + +In these examples, we show how :func:`tyro.cli` can be used to instantiate +nested structures. This can enable modular, reusable, and composable CLI +interfaces. diff --git a/examples/02_nesting/01_nesting.py b/examples/02_nesting/01_nesting.py deleted file mode 100644 index b6e67305..00000000 --- a/examples/02_nesting/01_nesting.py +++ /dev/null @@ -1,72 +0,0 @@ -"""Hierarchical Configs - -Structures (typically dataclasses) can be nested to build hierarchical configuration -objects. This helps with modularity and grouping in larger projects. - -Usage: -`python ./01_nesting.py --help` -`python ./01_nesting.py --out-dir . --config.optimizer.algorithm SGD` -`python ./01_nesting.py --out-dir . --restore-checkpoint` -""" - -import dataclasses -import enum -import pathlib - -import tyro - - -class OptimizerType(enum.Enum): - ADAM = enum.auto() - SGD = enum.auto() - - -@dataclasses.dataclass -class OptimizerConfig: - # Gradient-based optimizer to use. - algorithm: OptimizerType = OptimizerType.ADAM - - # Learning rate to use. - learning_rate: float = 3e-4 - - # Coefficient for L2 regularization. - weight_decay: float = 1e-2 - - -@dataclasses.dataclass -class ExperimentConfig: - # Various configurable options for our optimizer. - optimizer: OptimizerConfig - - # Batch size. - batch_size: int = 32 - - # Total number of training steps. - train_steps: int = 100_000 - - # Random seed. This is helpful for making sure that our experiments are all - # reproducible! - seed: int = 0 - - -def train( - out_dir: pathlib.Path, - config: ExperimentConfig, - restore_checkpoint: bool = False, - checkpoint_interval: int = 1000, -) -> None: - """Train a model. - - Args: - out_dir: Where to save logs and checkpoints. - config: Experiment configuration. - restore_checkpoint: Set to restore an existing checkpoint. - checkpoint_interval: Training steps between each checkpoint save. - """ - print(f"{out_dir=}, {restore_checkpoint=}, {checkpoint_interval=}") - print() - print(f"{config=}") - - -if __name__ == "__main__": - tyro.cli(train) diff --git a/examples/02_nesting/02_subcommands.py b/examples/02_nesting/02_subcommands.py deleted file mode 100644 index 5ccc2661..00000000 --- a/examples/02_nesting/02_subcommands.py +++ /dev/null @@ -1,46 +0,0 @@ -"""Subcommands - -Unions over nested types (classes or dataclasses) are populated using subcommands. - -For configuring subcommands beyond what can be expressed with type annotations, see -:func:`tyro.conf.subcommand()`. - -Usage: -`python ./02_subcommands.py --help` -`python ./02_subcommands.py cmd:commit --help` -`python ./02_subcommands.py cmd:commit --cmd.message hello --cmd.all` -`python ./02_subcommands.py cmd:checkout --help` -`python ./02_subcommands.py cmd:checkout --cmd.branch main` -""" - -from __future__ import annotations - -import dataclasses - -import tyro - - -@dataclasses.dataclass(frozen=True) -class Checkout: - """Checkout a branch.""" - - branch: str - - -@dataclasses.dataclass(frozen=True) -class Commit: - """Commit changes.""" - - message: str - all: bool = False - - -def main(cmd: Checkout | Commit) -> None: - print(cmd) - - -if __name__ == "__main__": - # Note that we can also pass `Checkout | Command` directly into - # `tyro.cli()`; this is understood by tyro and pyright, but unfortunately not by - # mypy. - tyro.cli(main) diff --git a/examples/02_nesting/04_nesting_in_containers.py b/examples/02_nesting/04_nesting_in_containers.py deleted file mode 100644 index 18892696..00000000 --- a/examples/02_nesting/04_nesting_in_containers.py +++ /dev/null @@ -1,60 +0,0 @@ -"""Nesting in Containers - -Structures can be nested inside of standard containers. - -Note that lengths must be inferable, either via a fixed-length tuple annotation or by -parsing default values. - - -Usage: -`python ./04_nesting_in_containers.py.py --help` -""" - -import dataclasses - -import tyro - - -class Color: - pass - - -@dataclasses.dataclass -class RGB(Color): - r: int - g: int - b: int - - -@dataclasses.dataclass -class HSL(Color): - h: int - s: int - l: int - - -@dataclasses.dataclass -class Args: - # Example of specifying nested structures via a fixed-length tuple. - color_tuple: tuple[RGB, HSL] - - # Examples of nested structures in variable-length containers. These need a default - # provided for length inference; we don't currently support specifying dynamic - # container lengths directly from the commandline. - color_tuple_alt: tuple[Color, ...] = ( - RGB(255, 0, 0), - HSL(0, 255, 0), - ) - color_map: dict[str, RGB] = dataclasses.field( - # We can't use mutable values as defaults directly. - default_factory={ - "red": RGB(255, 0, 0), - "green": RGB(0, 255, 0), - "blue": RGB(0, 0, 255), - }.copy - ) - - -if __name__ == "__main__": - args = tyro.cli(Args) - print(args) diff --git a/examples/03_subcommands/01_subcommands.py b/examples/03_subcommands/01_subcommands.py new file mode 100644 index 00000000..0f184d34 --- /dev/null +++ b/examples/03_subcommands/01_subcommands.py @@ -0,0 +1,53 @@ +# mypy: ignore-errors +# +# Passing a Union type directly to tyro.cli() doesn't type-check correctly in +# mypy. This will be fixed by `typing.TypeForm`: https://peps.python.org/pep-0747/ +"""Subcommands are Unions + +All of :mod:`tyro`'s subcommand features are built using unions over struct +types (typically dataclasses). Subcommands are used to choose between types in +the union; arguments are then populated from the chosen type. + +.. note:: + + For configuring subcommands beyond what can be expressed with type annotations, see + :func:`tyro.conf.subcommand()`. + +Usage: + + # Print the helptext. This will show the available subcommands: + python ./01_subcommands.py --help + + # The `commit` subcommand: + python ./01_subcommands.py commit --help + python ./01_subcommands.py commit --message hello + + # The `checkout` subcommand: + python ./01_subcommands.py checkout --help + python ./01_subcommands.py checkout --branch main +""" + +from __future__ import annotations + +import dataclasses + +import tyro + + +@dataclasses.dataclass(frozen=True) +class Checkout: + """Checkout a branch.""" + + branch: str + + +@dataclasses.dataclass(frozen=True) +class Commit: + """Commit changes.""" + + message: str + + +if __name__ == "__main__": + cmd = tyro.cli(Checkout | Commit) + print(cmd) diff --git a/examples/03_subcommands/02_subcommands_in_func.py b/examples/03_subcommands/02_subcommands_in_func.py new file mode 100644 index 00000000..727d336e --- /dev/null +++ b/examples/03_subcommands/02_subcommands_in_func.py @@ -0,0 +1,60 @@ +"""Subcommands as Function Arguments + +A subcommand will be created for each input annotated with a union over +struct types. + +.. note:: + + To prevent :func:`tyro.cli()` from converting a Union type into a subcommand, + use :class:`tyro.conf.AvoidSubcommands`. + +.. note:: + + Argument ordering for subcommands can be tricky. In the example below, + ``--shared-arg`` must always come *before* the subcommand. As an option for + alleviating this, see :class:`tyro.conf.ConsolidateSubcommandArgs`. + + +Usage: + + # Print the helptext. This will show the available subcommands: + python ./02_subcommands_in_func.py --help + + # Using the default subcommand: + python ./02_subcommands_in_func.py --shared-arg 100 + + # Choosing a different subcommand: + python ./02_subcommands_in_func.py --shared-arg 100 cmd:commit --cmd.message Hello! +""" + +from __future__ import annotations + +import dataclasses + +import tyro + + +@dataclasses.dataclass(frozen=True) +class Checkout: + """Checkout a branch.""" + + branch: str + + +@dataclasses.dataclass(frozen=True) +class Commit: + """Commit changes.""" + + message: str + + +def main( + shared_arg: int, + cmd: Checkout | Commit = Checkout(branch="default"), +): + print(f"{shared_arg=}") + print(cmd) + + +if __name__ == "__main__": + tyro.cli(main) diff --git a/examples/02_nesting/03_multiple_subcommands.py b/examples/03_subcommands/03_multiple_subcommands.py similarity index 68% rename from examples/02_nesting/03_multiple_subcommands.py rename to examples/03_subcommands/03_multiple_subcommands.py index 561232b5..86cfcd5c 100644 --- a/examples/02_nesting/03_multiple_subcommands.py +++ b/examples/03_subcommands/03_multiple_subcommands.py @@ -1,12 +1,15 @@ """Sequenced Subcommands -Multiple unions over nested types are populated using a series of subcommands. +Multiple unions over struct types are populated using a series of subcommands. Usage: -`python ./03_multiple_subcommands.py --help` -`python ./03_multiple_subcommands.py dataset:mnist --help` -`python ./03_multiple_subcommands.py dataset:mnist optimizer:adam --help` -`python ./03_multiple_subcommands.py dataset:mnist optimizer:adam --optimizer.learning-rate 3e-4 --dataset.binary` + + # Note that we apply the :class:`tyro.conf.ConsolidateSubcommandArgs` flag. + # This pushes all arguments to the end of the command: + python ./03_multiple_subcommands.py --help + python ./03_multiple_subcommands.py dataset:mnist --help + python ./03_multiple_subcommands.py dataset:mnist optimizer:adam --help + python ./03_multiple_subcommands.py dataset:mnist optimizer:adam --optimizer.learning-rate 3e-4 --dataset.binary """ from __future__ import annotations diff --git a/examples/04_additional/15_decorator_subcommands.py b/examples/03_subcommands/04_decorator_subcommands.py similarity index 64% rename from examples/04_additional/15_decorator_subcommands.py rename to examples/03_subcommands/04_decorator_subcommands.py index e9fc77cc..297a6ab9 100644 --- a/examples/04_additional/15_decorator_subcommands.py +++ b/examples/03_subcommands/04_decorator_subcommands.py @@ -4,12 +4,13 @@ subcommands, which is inspired by `click `_. Usage: -`python my_script.py --help` -`python my_script.py greet --help` -`python my_script.py greet --name Alice` -`python my_script.py greet --name Bob --loud` -`python my_script.py addition --help` -`python my_script.py addition --a 5 --b 3` + + python 04_decorator_subcommands.py --help + python 04_decorator_subcommands.py greet --help + python 04_decorator_subcommands.py greet --name Alice + python 04_decorator_subcommands.py greet --name Bob --loud + python 04_decorator_subcommands.py addition --help + python 04_decorator_subcommands.py addition --a 5 --b 3 """ from tyro.extras import SubcommandApp diff --git a/examples/02_nesting/05_subcommands_func.py b/examples/03_subcommands/05_subcommands_func.py similarity index 72% rename from examples/02_nesting/05_subcommands_func.py rename to examples/03_subcommands/05_subcommands_func.py index ca2f202f..a65c0bb5 100644 --- a/examples/02_nesting/05_subcommands_func.py +++ b/examples/03_subcommands/05_subcommands_func.py @@ -7,11 +7,12 @@ Usage: -`python ./05_subcommands_func.py --help` -`python ./05_subcommands_func.py commit --help` -`python ./05_subcommands_func.py commit --message hello --all` -`python ./05_subcommands_func.py checkout --help` -`python ./05_subcommands_func.py checkout --branch main` + + python ./05_subcommands_func.py --help + python ./05_subcommands_func.py commit --help + python ./05_subcommands_func.py commit --message hello --all + python ./05_subcommands_func.py checkout --help + python ./05_subcommands_func.py checkout --branch main """ import tyro diff --git a/examples/03_subcommands/README.rst b/examples/03_subcommands/README.rst new file mode 100644 index 00000000..156c89fe --- /dev/null +++ b/examples/03_subcommands/README.rst @@ -0,0 +1,5 @@ +Subcommands +=========== + +In these examples, we show how :func:`tyro.cli` can be used to create CLI +interfaces with subcommands. diff --git a/examples/04_additional/01_positional_args.py b/examples/04_additional/01_positional_args.py deleted file mode 100644 index 66048cd5..00000000 --- a/examples/04_additional/01_positional_args.py +++ /dev/null @@ -1,63 +0,0 @@ -"""Positional Arguments - -Positional-only arguments in functions are converted to positional CLI arguments. - -For more general positional arguments, see :class:`tyro.conf.Positional`. - -Usage: -`python ./01_positional_args.py --help` -`python ./01_positional_args.py ./a ./b --optimizer.learning-rate 1e-5` -""" - -from __future__ import annotations - -import dataclasses -import enum -import pathlib -from typing import Tuple - -import tyro - - -def main( - source: pathlib.Path, - dest: pathlib.Path, - /, # Mark the end of positional arguments. - optimizer: OptimizerConfig, - force: bool = False, - verbose: bool = False, - background_rgb: Tuple[float, float, float] = (1.0, 0.0, 0.0), -) -> None: - """Command-line interface defined using a function signature. Note that this - docstring is parsed to generate helptext. - - Args: - source: Source path. - dest: Destination path. - optimizer: Configuration for our optimizer object. - force: Do not prompt before overwriting. - verbose: Explain what is being done. - background_rgb: Background color. Red by default. - """ - print(f"{source=}\n{dest=}\n{optimizer=}\n{force=}\n{verbose=}\n{background_rgb=}") - - -class OptimizerType(enum.Enum): - ADAM = enum.auto() - SGD = enum.auto() - - -@dataclasses.dataclass(frozen=True) -class OptimizerConfig: - algorithm: OptimizerType = OptimizerType.ADAM - """Gradient-based optimizer to use.""" - - learning_rate: float = 3e-4 - """Learning rate to use.""" - - weight_decay: float = 1e-2 - """Coefficient for L2 regularization.""" - - -if __name__ == "__main__": - tyro.cli(main) diff --git a/examples/04_additional/11_aliases.py b/examples/04_additional/11_aliases.py deleted file mode 100644 index 5ee33de9..00000000 --- a/examples/04_additional/11_aliases.py +++ /dev/null @@ -1,41 +0,0 @@ -"""Argument Aliases - -:func:`tyro.conf.arg()` can be used to attach aliases to arguments. - -Usage: -`python ./11_aliases.py --help` -`python ./11_aliases.py commit --help` -`python ./11_aliases.py commit --message hello --all` -`python ./11_aliases.py commit -m hello -a` -`python ./11_aliases.py checkout --help` -`python ./11_aliases.py checkout --branch main` -`python ./11_aliases.py checkout -b main` -""" - -from typing_extensions import Annotated - -import tyro - - -def checkout( - branch: Annotated[str, tyro.conf.arg(aliases=["-b"])], -) -> None: - """Check out a branch.""" - print(f"{branch=}") - - -def commit( - message: Annotated[str, tyro.conf.arg(aliases=["-m"])], - all: Annotated[bool, tyro.conf.arg(aliases=["-a"])] = False, -) -> None: - """Make a commit.""" - print(f"{message=} {all=}") - - -if __name__ == "__main__": - tyro.extras.subcommand_cli_from_dict( - { - "checkout": checkout, - "commit": commit, - } - ) diff --git a/examples/04_overriding_configs/01_dataclasses_defaults.py b/examples/04_overriding_configs/01_dataclasses_defaults.py new file mode 100644 index 00000000..c52aa66f --- /dev/null +++ b/examples/04_overriding_configs/01_dataclasses_defaults.py @@ -0,0 +1,56 @@ +"""Dataclasses + Defaults + +The :code:`default=` argument can be used to override default values in dataclass +types. + + +.. note:: + + When ``default=`` is used, we advise against mutation of configuration + objects from a dataclass's :code:`__post_init__` method [#f1]_. In the + example below, :code:`__post_init__` would be called twice: once for the + :code:`Args()` object provided as a default value and another time for the + :code:`Args()` objected instantiated by :func:`tyro.cli()`. This can cause + confusing behavior! Instead, we show below one example of how derived + fields can be defined immutably. + + .. [#f1] Official Python docs for ``__post_init__`` can be found `here `_. + + +Usage: + + python ./01_dataclasses_defaults.py --help + python ./01_dataclasses_defaults.py --reps 3 + python ./01_dataclasses_defaults.py --string hello --reps 5 +""" + +import dataclasses + +import tyro + + +@dataclasses.dataclass +class Args: + """Description. + This should show up in the helptext!""" + + string: str + """A string field.""" + + reps: int = 3 + """A numeric field, with a default value.""" + + @property + def derived_field(self) -> str: + return ", ".join([self.string] * self.reps) + + +if __name__ == "__main__": + args = tyro.cli( + Args, + default=Args( + string="default string", + reps=tyro.MISSING, + ), + ) + print(args.derived_field) diff --git a/examples/03_config_systems/02_overriding_yaml.py b/examples/04_overriding_configs/02_overriding_yaml.py similarity index 72% rename from examples/03_config_systems/02_overriding_yaml.py rename to examples/04_overriding_configs/02_overriding_yaml.py index 5db0cfd8..6ef7bcce 100644 --- a/examples/03_config_systems/02_overriding_yaml.py +++ b/examples/04_overriding_configs/02_overriding_yaml.py @@ -1,12 +1,19 @@ """Overriding YAML Configs +:mod:`tyro` understands a wide range of data structures, including standard dictionaries +and lists. + If you have a library of existing YAML files that you want to use, `tyro` can -be used to override values in them. We generally recommend dataclass configs -for new projects. +help override values within them. + +.. note:: + + We recommend dataclass configs for new projects. Usage: -`python ./02_overriding_yaml.py --help` -`python ./02_overriding_yaml.py --training.checkpoint-steps 300 1000 9000` + + python ./02_overriding_yaml.py --help + python ./02_overriding_yaml.py --training.checkpoint-steps 300 1000 9000 """ import yaml diff --git a/examples/03_config_systems/01_base_configs.py b/examples/04_overriding_configs/03_choosing_base_configs.py similarity index 63% rename from examples/03_config_systems/01_base_configs.py rename to examples/04_overriding_configs/03_choosing_base_configs.py index 50ee71fa..fb26e068 100644 --- a/examples/03_config_systems/01_base_configs.py +++ b/examples/04_overriding_configs/03_choosing_base_configs.py @@ -1,19 +1,32 @@ -"""Base Configurations +"""Choosing Base Configs -We can integrate `tyro` into common configuration patterns: here, we select -one of multiple possible base configurations, create a subcommand for each one, and then -use the CLI to either override (existing) or fill in (missing) values. +One common pattern is to have a set of "base" configurations, which can be +selected from and then overridden. -The helper function used here, :func:`tyro.extras.overridable_config_cli()`, is a -lightweight wrapper over :func:`tyro.cli()`. +This is often implemented with a set of configuration files (e.g., YAML files). +With :mod:`tyro`, we can instead define each base configuration as a separate +Python object. + +After creating the base configurations, we can use the CLI to select one of +them and then override (existing) or fill in (missing) values. + +The helper function used here, :func:`tyro.extras.overridable_config_cli()`, is +a lightweight wrapper over :func:`tyro.cli()` and its Union-based subcommand +syntax. Usage: -`python ./01_base_configs.py --help` -`python ./01_base_configs.py small --help` -`python ./01_base_configs.py small --seed 94720` -`python ./01_base_configs.py big --help` -`python ./01_base_configs.py big --seed 94720` + + # Overall helptext: + python ./03_choosing_base_configs.py --help + + # The "small" subcommand: + python ./03_choosing_base_configs.py small --help + python ./03_choosing_base_configs.py small --seed 94720 + + # The "big" subcommand: + python ./03_choosing_base_configs.py big --help + python ./03_choosing_base_configs.py big --seed 94720 """ from dataclasses import dataclass @@ -24,20 +37,11 @@ import tyro -@dataclass(frozen=True) -class AdamOptimizer: - learning_rate: float = 1e-3 - betas: tuple[float, float] = (0.9, 0.999) - - @dataclass(frozen=True) class ExperimentConfig: # Dataset to run experiment on. dataset: Literal["mnist", "imagenet-50"] - # Optimizer parameters. - optimizer: AdamOptimizer - # Model size. num_layers: int units: int @@ -64,7 +68,6 @@ class ExperimentConfig: "Small experiment.", ExperimentConfig( dataset="mnist", - optimizer=AdamOptimizer(), batch_size=2048, num_layers=4, units=64, @@ -77,7 +80,6 @@ class ExperimentConfig: "Big experiment.", ExperimentConfig( dataset="imagenet-50", - optimizer=AdamOptimizer(), batch_size=32, num_layers=8, units=256, diff --git a/examples/04_overriding_configs/README.rst b/examples/04_overriding_configs/README.rst new file mode 100644 index 00000000..2ea9e690 --- /dev/null +++ b/examples/04_overriding_configs/README.rst @@ -0,0 +1,5 @@ +Overriding Configs +================== + +In these examples, we show how :func:`tyro.cli` can be used to override values +in pre-instantiated configuration objects. diff --git a/examples/04_additional/06_generics_py312.py b/examples/05_generics/01_generics_py312.py similarity index 68% rename from examples/04_additional/06_generics_py312.py rename to examples/05_generics/01_generics_py312.py index 416d75e2..bb4ef868 100644 --- a/examples/04_additional/06_generics_py312.py +++ b/examples/05_generics/01_generics_py312.py @@ -1,17 +1,17 @@ # mypy: ignore-errors # # PEP 695 isn't yet supported in mypy. (April 4, 2024) -"""Generic Types (Python 3.12+) +"""Generics (3.12+) -Example of parsing for generic dataclasses using syntax introduced in Python -3.12 (`PEP 695 `_). +This example uses syntax introduced in Python 3.12 (`PEP 695 `_). -.. warning:: +.. note:: If used in conjunction with :code:`from __future__ import annotations`, the updated type parameter syntax requires Python 3.12.4 or newer. For technical details, see `this CPython PR `_. Usage: -`python ./05_generics.py --help` + + python ./01_generics_py312.py --help """ import dataclasses @@ -19,7 +19,7 @@ import tyro -@dataclasses.dataclass(frozen=True) +@dataclasses.dataclass class Point3[ScalarType: (int, float)]: x: ScalarType y: ScalarType @@ -27,14 +27,14 @@ class Point3[ScalarType: (int, float)]: frame_id: str -@dataclasses.dataclass(frozen=True) +@dataclasses.dataclass class Triangle: a: Point3[float] b: Point3[float] c: Point3[float] -@dataclasses.dataclass(frozen=True) +@dataclasses.dataclass class Args[ShapeType]: shape: ShapeType diff --git a/examples/04_additional/05_generics.py b/examples/05_generics/02_generics.py similarity index 77% rename from examples/04_additional/05_generics.py rename to examples/05_generics/02_generics.py index 89f96b24..698d15e8 100644 --- a/examples/04_additional/05_generics.py +++ b/examples/05_generics/02_generics.py @@ -1,9 +1,11 @@ -"""Generic Types +"""Generics (Legacy) -Example of parsing for generic dataclasses. +The legacy :py:class:`typing.Generic` and :py:class:`typing.TypeVar` syntax for +generic types is also supported. Usage: -`python ./05_generics.py --help` + + python ./02_generics.py --help """ import dataclasses diff --git a/examples/05_generics/README.rst b/examples/05_generics/README.rst new file mode 100644 index 00000000..b6e388af --- /dev/null +++ b/examples/05_generics/README.rst @@ -0,0 +1,5 @@ +Generics +======== + +:mod:`tyro`'s understanding of Python's type system includes user-defined +parameterized types, which can reduce boilerplate and improve type safety. diff --git a/examples/05_custom_constructors/01_primitive_annotation.py b/examples/06_custom_constructors/01_primitive_annotation.py similarity index 50% rename from examples/05_custom_constructors/01_primitive_annotation.py rename to examples/06_custom_constructors/01_primitive_annotation.py index 505e8b9d..e22fbca4 100644 --- a/examples/05_custom_constructors/01_primitive_annotation.py +++ b/examples/06_custom_constructors/01_primitive_annotation.py @@ -1,5 +1,11 @@ """Custom Primitive +.. note:: + + This is an advanced feature, which should not be needed for the vast + majority of use cases. If :mod:`tyro` is missing support for a built-in + Python type, please open an issue on `GitHub `_. + For additional flexibility, :mod:`tyro.constructors` exposes tyro's API for defining behavior for different types. There are two categories of types: primitive types can be instantiated from a single commandline argument, while @@ -8,9 +14,10 @@ In this example, we attach a custom constructor via a runtime annotation. Usage: -`python ./01_primitive_annotation.py --help` -`python ./01_primitive_annotation.py --dict1 '{"hello": "world"}'` -`python ./01_primitive_annotation.py --dict1 '{"hello": "world"}' --dict2 '{"hello": "world"}'` + + python ./01_primitive_annotation.py --help + python ./01_primitive_annotation.py --dict1 '{"hello": "world"}' + python ./01_primitive_annotation.py --dict1 '{"hello": "world"}' --dict2 '{"hello": "world"}' """ import json @@ -23,10 +30,19 @@ JsonDict = Annotated[ dict, tyro.constructors.PrimitiveConstructorSpec( + # Number of arguments to consume. nargs=1, + # Argument name in usage messages. metavar="JSON", + # Convert a list of strings to an instance. The length of the list + # should match `nargs`. instance_from_str=lambda args: json.loads(args[0]), + # Check if an instance is of the expected type. This is only used for + # helptext formatting in the presence of union types. is_instance=lambda instance: isinstance(instance, dict), + # Convert an instance to a list of strings. This is used for handling + # default values that are set in Python. The length of the list should + # match `nargs`. str_from_instance=lambda instance: [json.dumps(instance)], ), ] diff --git a/examples/05_custom_constructors/02_primitive_registry.py b/examples/06_custom_constructors/02_primitive_registry.py similarity index 62% rename from examples/05_custom_constructors/02_primitive_registry.py rename to examples/06_custom_constructors/02_primitive_registry.py index 767765f0..43589c23 100644 --- a/examples/05_custom_constructors/02_primitive_registry.py +++ b/examples/06_custom_constructors/02_primitive_registry.py @@ -1,17 +1,13 @@ """Custom Primitive (Registry) -For additional flexibility, :mod:`tyro.constructors` exposes tyro's API for -defining behavior for different types. There are two categories of types: -primitive types can be instantiated from a single commandline argument, while -struct types are broken down into multiple. - -In this example, we attach a custom constructor by defining a rule that applies -to all types that match ``dict[str, Any]``. +In this example, we use :class:`tyro.constructors.PrimitiveConstructorSpec` to +define a rule that applies to all types that match ``dict[str, Any]``. Usage: -`python ./02_primitive_registry.py --help` -`python ./02_primitive_registry.py --dict1 '{"hello": "world"}'` -`python ./02_primitive_registry.py --dict1 '{"hello": "world"}' --dict2 '{"hello": "world"}'` + + python ./02_primitive_registry.py --help + python ./02_primitive_registry.py --dict1 '{"hello": "world"}' + python ./02_primitive_registry.py --dict1 '{"hello": "world"}' --dict2 '{"hello": "world"}' """ import json @@ -49,5 +45,6 @@ def main( if __name__ == "__main__": + # The custom registry is used as a context. with custom_registry: tyro.cli(main) diff --git a/examples/06_custom_constructors/README.rst b/examples/06_custom_constructors/README.rst new file mode 100644 index 00000000..f954f2ce --- /dev/null +++ b/examples/06_custom_constructors/README.rst @@ -0,0 +1,4 @@ +Custom constructors +=================== + +In these examples, we show how custom types can be parsed by and registered with :func:`tyro.cli`. diff --git a/examples/04_additional/14_suppress_console_outputs.py b/examples/07_pytorch_jax/01_pytorch_parallelism.py similarity index 88% rename from examples/04_additional/14_suppress_console_outputs.py rename to examples/07_pytorch_jax/01_pytorch_parallelism.py index c8b48840..97ef6a74 100644 --- a/examples/04_additional/14_suppress_console_outputs.py +++ b/examples/07_pytorch_jax/01_pytorch_parallelism.py @@ -1,4 +1,4 @@ -"""Cleaner Console Outputs for Scripts with Multiple Workers +"""PyTorch Parallelism The :code:`console_outputs=` argument can be set to :code:`False` to suppress helptext and error message printing. @@ -20,7 +20,8 @@ Usage: -`python ./14_suppress_console_outputs.py --help` + + python ./01_pytorch_parallelism.py --help """ import dataclasses diff --git a/examples/04_additional/10_flax.py b/examples/07_pytorch_jax/02_flax.py similarity index 94% rename from examples/04_additional/10_flax.py rename to examples/07_pytorch_jax/02_flax.py index ecc5c50f..10bb0131 100644 --- a/examples/04_additional/10_flax.py +++ b/examples/07_pytorch_jax/02_flax.py @@ -4,8 +4,9 @@ directly from :func:`tyro.cli()`. Usage: -`python ./07_flax.py --help` -`python ./07_flax.py --model.layers 4` + + python ./02_flax.py --help + python ./02_flax.py --model.layers 4 """ from flax import linen as nn diff --git a/examples/07_pytorch_jax/README.rst b/examples/07_pytorch_jax/README.rst new file mode 100644 index 00000000..f7bb835f --- /dev/null +++ b/examples/07_pytorch_jax/README.rst @@ -0,0 +1,4 @@ +PyTorch / JAX +============= + +In these examples, we show some patterns for using :func:`tyro.cli` with PyTorch and JAX. diff --git a/src/tyro/conf/_markers.py b/src/tyro/conf/_markers.py index f96aa68e..5ac78ef9 100644 --- a/src/tyro/conf/_markers.py +++ b/src/tyro/conf/_markers.py @@ -68,17 +68,22 @@ By default, :mod:`tyro` will generate a traditional CLI interface where args are applied to the directly preceding subcommand. When we have two subcommands ``s1`` and ``s2``: -`` -python x.py {--root options} s1 {--s1 options} s2 {--s2 options} -`` + + +.. code-block:: bash + + python x.py {--root options} s1 {--s1 options} s2 {--s2 options} This can be frustrating because the resulting CLI is sensitive to the positioning of options. To consolidate subcommands, we push arguments to the end, after all subcommands: -`` -python x.py s1 s2 {--root, s1, and s2 options} -`` + + +.. code-block:: bash + + python x.py s1 s2 {--root, s1, and s2 options} + This is more robust to reordering of options, ensuring that any new options can simply be placed at the end of the command. @@ -103,6 +108,8 @@ If we have a structure with the field: +.. code-block:: python + cmd: NestedType By default, ``--cmd.arg`` may be generated as a flag. If prefixes are omitted, we would diff --git a/src/tyro/extras/_subcommand_app.py b/src/tyro/extras/_subcommand_app.py index 21e72be4..7ce21494 100644 --- a/src/tyro/extras/_subcommand_app.py +++ b/src/tyro/extras/_subcommand_app.py @@ -8,7 +8,9 @@ class SubcommandApp: - """This module provides a decorator-based API for subcommands in :mod:`tyro`, inspired by click. + """This class provides a decorator-based API for subcommands in + :mod:`tyro`, inspired by click. Under-the-hood, this is a light wrapper + over :func:`tyro.cli`. Example: