Skip to content

Commit

Permalink
[tuner] Update README (#532)
Browse files Browse the repository at this point in the history
Add a warning about the tuner being in early development. Update the
tuner readme and remove obsolete instructions.
  • Loading branch information
kuhar authored Nov 15, 2024
1 parent a5c1527 commit 7ff0534
Show file tree
Hide file tree
Showing 2 changed files with 16 additions and 42 deletions.
5 changes: 5 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,11 @@ conversion tools to produce inference-optimized programs.
The Tuner sub-project assists with tuning program performance by searching for
optimal parameter configurations to use during model compilation.

> [!WARNING]
> SHARK Tuner is still in early development. Interested users may want
> to try it out, but the tuner is not ready for general use yet. Check out
> [the readme](tuner/README.md) for more details.
## Support matrix

<!-- TODO: version requirements for Python, ROCm, Linux, etc. -->
Expand Down
53 changes: 11 additions & 42 deletions tuner/README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,8 @@
# IREE dispatch auto-tuning scripts
`libtuner.py` is the core Python script that provides the fundamental functions for the tuning loop. It imports `candidate_gen.py` for candidate generation. To implement the full tuning loop, `libtuner.py` requires a separate Python script that uses the provided `TuningClient` API from `libtuner.py`.
`libtuner.py` is the core Python script that provides the fundamental functions
for the tuning loop. It imports `candidate_gen.py` for candidate generation. To
implement the full tuning loop, `libtuner.py` requires a separate Python script
that uses the provided `TuningClient` API from `libtuner.py`.

## Prerequisites
[Optional] Using virtual environments:
Expand All @@ -22,47 +25,13 @@ Using the IREE's Python bindings:
- Set environment
```shell
source ../iree-build/.env && export PYTHONPATH
export PATH="$(realpath ../iree-build/tools):$PATH"
```
For more information, refer to the [IREE documentation](https://iree.dev/building-from-source/getting-started/#python-bindings)
For more information, refer to the [IREE
documentation](https://iree.dev/building-from-source/getting-started/#python-bindings).

### Overall flow
## Examples

1. Symlink all scripts and mlir/irpa files in your build dir.
- Symlink `iree-build-dir/tools` inside `tuning`.
- Symlink ML model MLIR and weights based on `unet.sh`.

2. Copy the attention/matmul spec as `config.mlir` in the tuning dir.

3. Temporarily comment out all the existing configs in `config.mlir`.
- Example:
```mlir
// , @match_mmt_2048x10240x1280 -> @apply_op_config
// , @match_mmt_2048x1280x5120 -> @apply_op_config
// , @match_mmt_2048x1280x1280 -> @apply_op_config
```

4. Compile a baseline unet
```shell
./unet.sh winograd unet.mlir -o unet_baseline.vmfb --iree-hal-dump-executable-files-to=dump-winograd
```

5. Find the matmul to tune and copy the `*_benchmark.mlir` file to the build dir.
```shell
cp dump-winograd/*_141_*benchmark.mlir ./141.mlir
```

6. Run the tuning script.
- Example:
```shell
python -m examples.punet 141.mlir --devices=hip://GPU-0,hip://GPU-4 --num-candidates=1024
```

7. Check the winner candidate in `result_summary.log`, find and copy the transform spec.

8. Paste the transform spec into the `config.mlir` and uncomment them.

9. Add the match function to the entry point in `config.mlir`
- Example:
```mlir
@match_something -> @apply_op_config
```
Check the `examples` directory for sample tuners implemented with `libtuner`.
The [`dispatch` example](https://github.com/nod-ai/SHARK-Platform/tree/main/tuner/examples/dispatch)
should be a good starting point for most users.

0 comments on commit 7ff0534

Please sign in to comment.