Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a tip on YAML anchors to docs on dataset layers #2181

Open
wants to merge 8 commits into
base: main
Choose a base branch
from
37 changes: 37 additions & 0 deletions docs/source/kedro-viz_visualisation.md
Original file line number Diff line number Diff line change
Expand Up @@ -194,6 +194,43 @@

![](./images/pipeline_visualisation_with_layers.png)

Duplicated definitions like:

```yaml
metadata:
kedro-viz:
layer: raw
```
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we also show the example here that you gave in the original issue on how to use it. thanks

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_raw_layer: &raw_layer
  metadata:
    kedro-viz:
      layer: 01_raw

And then reuse it like this:

companies:
  type: pandas.CSVDataset
  filepath: data/01_raw/companies.csv
  <<: *raw_layer

reviews:
  type: pandas.CSVDataset
  filepath: data/01_raw/reviews.csv
  <<: *raw_layer

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rashidakanchwala yeah let me add this

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


can be avoided by leveraging YAML native syntax for anchors and aliases.

Use an anchor (`&`) first, to create a reusable piece of configuration:

```yaml
_raw_layer: &raw_layer
metadata:
kedro-viz:
layer: 01_raw
```

And then leverage aliases (`*`) to reference it:

Check warning on line 216 in docs/source/kedro-viz_visualisation.md

View workflow job for this annotation

GitHub Actions / runner / vale

[vale] reported by reviewdog 🐶 [Kedro-viz.words] Use 'use' instead of 'leverage'. Raw Output: {"message": "[Kedro-viz.words] Use 'use' instead of 'leverage'.", "location": {"path": "docs/source/kedro-viz_visualisation.md", "range": {"start": {"line": 216, "column": 10}}}, "severity": "WARNING"}

```yaml
companies:
type: pandas.CSVDataset
filepath: data/01_raw/companies.csv
<<: *raw_layer

reviews:
type: pandas.CSVDataset
filepath: data/01_raw/reviews.csv
<<: *raw_layer

# Same for other datasets of the raw layer...
```

See [this example from the Kedro docs](https://docs.kedro.org/en/stable/data/data_catalog_yaml_examples.html#load-multiple-datasets-with-similar-configuration-using-yaml-anchors) for more details.

## Share a pipeline visualisation

You can save a pipeline structure within a Kedro-Viz visualisation directly from the terminal as follows:
Expand Down
Loading