Skip to content

Commit

Permalink
[dagster-tableau] Update docs with refreshable workbooks and material…
Browse files Browse the repository at this point in the history
…ization (#24864)

## Summary & Motivation

As title.

## How I Tested These Changes

Docs preview

## Changelog

NOCHANGELOG

---------

Co-authored-by: Alex Noonan <[email protected]>
Co-authored-by: colton <[email protected]>
Co-authored-by: Colton Padden <[email protected]>
  • Loading branch information
4 people authored Oct 18, 2024
1 parent f7a57f3 commit 9811d73
Show file tree
Hide file tree
Showing 16 changed files with 197 additions and 113 deletions.
7 changes: 7 additions & 0 deletions docs/content/_apidocs.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -700,6 +700,13 @@ Dagster also provides a growing set of optional add-on libraries to integrate wi
posting files via SFTP.
</td>
</tr>
<tr>
<td>
<a href="/_apidocs/libraries/dagster-tableau">Tableau</a> (
<code>dagster-tableau</code>)
</td>
<td>Provides a resource for integrating Tableau Workspaces</td>
</tr>
<tr>
<td>
<a href="/_apidocs/libraries/dagster-twilio">Twilio</a> (
Expand Down
4 changes: 4 additions & 0 deletions docs/content/_navigation.json
Original file line number Diff line number Diff line change
Expand Up @@ -1572,6 +1572,10 @@
"title": "SSH / SFTP (dagster-ssh)",
"path": "/_apidocs/libraries/dagster-ssh"
},
{
"title": "Tableau (dagster-tableau)",
"path": "/_apidocs/libraries/dagster-tableau"
},
{
"title": "Twilio (dagster-twilio)",
"path": "/_apidocs/libraries/dagster-twilio"
Expand Down
Binary file modified docs/content/api/modules.json.gz
Binary file not shown.
Binary file modified docs/content/api/searchindex.json.gz
Binary file not shown.
Binary file modified docs/content/api/sections.json.gz
Binary file not shown.
150 changes: 91 additions & 59 deletions docs/content/integrations/tableau.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,10 @@ This guide provides instructions for using Dagster with Tableau. Your Tableau as

- How to customize asset definition metadata for these Tableau assets.

- How to refresh Tableau workbooks.

- How to materialize Tableau sheets and dashboards.

<details>
<summary>Prerequisites</summary>

Expand All @@ -29,28 +33,28 @@ This guide provides instructions for using Dagster with Tableau. Your Tableau as

## Represent Tableau assets in the asset graph

To load Tableau assets into the Dagster asset graph, you must first construct a Tableau resource, which allows Dagster to communicate with your Tableau workspace. The Tableau resource to create depends on your Tableau deployment type - use `TableauCloudWorkspace` if you are using Tableau Cloud or `TableauServerWorkspace` if you are using Tableau Server. To connect to the Tableau workspace, you'll need to [configure a connected app with direct trust](https://help.tableau.com/current/online/en-gb/connected_apps_direct.htm) in Tableau, then supply your Tableau site information and connected app credentials to the resource. The Tableau resource uses the JSON Web Token (JWT) authentication to connect to the Tableau workspace.
To load Tableau assets into the Dagster asset graph, you must first construct a Tableau resource, which allows Dagster to communicate with your Tableau workspace. The Tableau resource to create depends on your Tableau deployment type - use <PyObject module="dagster_tableau" object="TableauCloudWorkspace" /> if you are using Tableau Cloud or <PyObject module="dagster_tableau" object="TableauServerWorkspace" /> if you are using Tableau Server. To connect to the Tableau workspace, you'll need to [configure a connected app with direct trust](https://help.tableau.com/current/online/en-gb/connected_apps_direct.htm) in Tableau, then supply your Tableau site information and connected app credentials to the resource. The Tableau resource uses the JSON Web Token (JWT) authentication to connect to the Tableau workspace.

Dagster can automatically load all data sources, sheets, and dashboards from your Tableau workspace. Call the `build_defs()` function, which returns a `Definitions` object containing all the asset definitions for these Tableau assets.
Dagster can automatically load all data sources, sheets, and dashboards from your Tableau workspace. Call the <PyObject module="dagster_tableau" object="BaseTableauClient" method="build_defs" /> function, which returns a <PyObject object="Definitions" /> object containing all the asset definitions for these Tableau assets.

<TabGroup>
<TabItem name="Using Dagster with Tableau Cloud">

Use `TableauCloudWorkspace` to interact with your Tableau Cloud workspace:
Use <PyObject module="dagster_tableau" object="TableauCloudWorkspace" /> to interact with your Tableau Cloud workspace:

```python file=/integrations/tableau/representing-tableau-cloud-assets.py
from dagster_tableau import TableauCloudWorkspace

from dagster import EnvVar
import dagster as dg

# Connect to Tableau Cloud using the connected app credentials
workspace = TableauCloudWorkspace(
connected_app_client_id=EnvVar("TABLEAU_CONNECTED_APP_CLIENT_ID"),
connected_app_secret_id=EnvVar("TABLEAU_CONNECTED_APP_SECRET_ID"),
connected_app_secret_value=EnvVar("TABLEAU_CONNECTED_APP_SECRET_VALUE"),
username=EnvVar("TABLEAU_USERNAME"),
site_name=EnvVar("TABLEAU_SITE_NAME"),
pod_name=EnvVar("TABLEAU_POD_NAME"),
connected_app_client_id=dg.EnvVar("TABLEAU_CONNECTED_APP_CLIENT_ID"),
connected_app_secret_id=dg.EnvVar("TABLEAU_CONNECTED_APP_SECRET_ID"),
connected_app_secret_value=dg.EnvVar("TABLEAU_CONNECTED_APP_SECRET_VALUE"),
username=dg.EnvVar("TABLEAU_USERNAME"),
site_name=dg.EnvVar("TABLEAU_SITE_NAME"),
pod_name=dg.EnvVar("TABLEAU_POD_NAME"),
)

defs = workspace.build_defs()
Expand All @@ -61,21 +65,21 @@ defs = workspace.build_defs()
</TabItem>
<TabItem name="Using Dagster with Tableau Server">

Use `TableauServerWorkspace` to interact with your Tableau Server workspace:
Use <PyObject module="dagster_tableau" object="TableauServerWorkspace" /> to interact with your Tableau Server workspace:

```python file=/integrations/tableau/representing-tableau-server-assets.py
from dagster_tableau import TableauServerWorkspace

from dagster import EnvVar
import dagster as dg

# Connect to Tableau Server using the connected app credentials
workspace = TableauServerWorkspace(
connected_app_client_id=EnvVar("TABLEAU_CONNECTED_APP_CLIENT_ID"),
connected_app_secret_id=EnvVar("TABLEAU_CONNECTED_APP_SECRET_ID"),
connected_app_secret_value=EnvVar("TABLEAU_CONNECTED_APP_SECRET_VALUE"),
username=EnvVar("TABLEAU_USERNAME"),
site_name=EnvVar("TABLEAU_SITE_NAME"),
server_name=EnvVar("TABLEAU_SERVER_NAME"),
connected_app_client_id=dg.EnvVar("TABLEAU_CONNECTED_APP_CLIENT_ID"),
connected_app_secret_id=dg.EnvVar("TABLEAU_CONNECTED_APP_SECRET_ID"),
connected_app_secret_value=dg.EnvVar("TABLEAU_CONNECTED_APP_SECRET_VALUE"),
username=dg.EnvVar("TABLEAU_USERNAME"),
site_name=dg.EnvVar("TABLEAU_SITE_NAME"),
server_name=dg.EnvVar("TABLEAU_SERVER_NAME"),
)

defs = workspace.build_defs()
Expand All @@ -88,28 +92,28 @@ defs = workspace.build_defs()

### Customize asset definition metadata for Tableau assets

By default, Dagster will generate asset keys for each Tableau asset based on its type and name and populate default metadata. You can further customize asset properties by passing a custom `DagsterTableauTranslator` subclass to the `build_defs()` function. This subclass can implement methods to customize the asset keys or specs for each Tableau asset type.
By default, Dagster will generate asset keys for each Tableau asset based on its type and name and populate default metadata. You can further customize asset properties by passing a custom <PyObject module="dagster_tableau" object="DagsterTableauTranslator" /> subclass to the <PyObject module="dagster_tableau" object="BaseTableauClient" method="build_defs" /> function. This subclass can implement methods to customize the asset keys or specs for each Tableau asset type.

```python file=/integrations/tableau/customize-tableau-asset-defs.py
from dagster_tableau import DagsterTableauTranslator, TableauCloudWorkspace
from dagster_tableau.translator import TableauContentData

from dagster import AssetSpec, EnvVar
import dagster as dg

workspace = TableauCloudWorkspace(
connected_app_client_id=EnvVar("TABLEAU_CONNECTED_APP_CLIENT_ID"),
connected_app_secret_id=EnvVar("TABLEAU_CONNECTED_APP_SECRET_ID"),
connected_app_secret_value=EnvVar("TABLEAU_CONNECTED_APP_SECRET_VALUE"),
username=EnvVar("TABLEAU_USERNAME"),
site_name=EnvVar("TABLEAU_SITE_NAME"),
pod_name=EnvVar("TABLEAU_POD_NAME"),
connected_app_client_id=dg.EnvVar("TABLEAU_CONNECTED_APP_CLIENT_ID"),
connected_app_secret_id=dg.EnvVar("TABLEAU_CONNECTED_APP_SECRET_ID"),
connected_app_secret_value=dg.EnvVar("TABLEAU_CONNECTED_APP_SECRET_VALUE"),
username=dg.EnvVar("TABLEAU_USERNAME"),
site_name=dg.EnvVar("TABLEAU_SITE_NAME"),
pod_name=dg.EnvVar("TABLEAU_POD_NAME"),
)


# A translator class lets us customize properties of the built
# Tableau assets, such as the owners or asset key
class MyCustomTableauTranslator(DagsterTableauTranslator):
def get_sheet_spec(self, data: TableauContentData) -> AssetSpec:
def get_sheet_spec(self, data: TableauContentData) -> dg.AssetSpec:
# We add a custom team owner tag to all sheets
return super().get_sheet_spec(data)._replace(owners=["my_team"])

Expand All @@ -124,70 +128,91 @@ Definitions from multiple Tableau workspaces can be combined by instantiating mu
```python file=/integrations/tableau/multiple-tableau-workspaces.py
from dagster_tableau import TableauCloudWorkspace

from dagster import Definitions, EnvVar
import dagster as dg

sales_team_workspace = TableauCloudWorkspace(
connected_app_client_id=EnvVar("SALES_TABLEAU_CONNECTED_APP_CLIENT_ID"),
connected_app_secret_id=EnvVar("SALES_TABLEAU_CONNECTED_APP_SECRET_ID"),
connected_app_secret_value=EnvVar("SALES_TABLEAU_CONNECTED_APP_SECRET_VALUE"),
username=EnvVar("TABLEAU_USERNAME"),
site_name=EnvVar("SALES_TABLEAU_SITE_NAME"),
pod_name=EnvVar("SALES_TABLEAU_POD_NAME"),
connected_app_client_id=dg.EnvVar("SALES_TABLEAU_CONNECTED_APP_CLIENT_ID"),
connected_app_secret_id=dg.EnvVar("SALES_TABLEAU_CONNECTED_APP_SECRET_ID"),
connected_app_secret_value=dg.EnvVar("SALES_TABLEAU_CONNECTED_APP_SECRET_VALUE"),
username=dg.EnvVar("TABLEAU_USERNAME"),
site_name=dg.EnvVar("SALES_TABLEAU_SITE_NAME"),
pod_name=dg.EnvVar("SALES_TABLEAU_POD_NAME"),
)

marketing_team_workspace = TableauCloudWorkspace(
connected_app_client_id=EnvVar("MARKETING_TABLEAU_CONNECTED_APP_CLIENT_ID"),
connected_app_secret_id=EnvVar("MARKETING_TABLEAU_CONNECTED_APP_SECRET_ID"),
connected_app_secret_value=EnvVar("MARKETING_TABLEAU_CONNECTED_APP_SECRET_VALUE"),
username=EnvVar("TABLEAU_USERNAME"),
site_name=EnvVar("MARKETING_TABLEAU_SITE_NAME"),
pod_name=EnvVar("MARKETING_TABLEAU_POD_NAME"),
connected_app_client_id=dg.EnvVar("MARKETING_TABLEAU_CONNECTED_APP_CLIENT_ID"),
connected_app_secret_id=dg.EnvVar("MARKETING_TABLEAU_CONNECTED_APP_SECRET_ID"),
connected_app_secret_value=dg.EnvVar(
"MARKETING_TABLEAU_CONNECTED_APP_SECRET_VALUE"
),
username=dg.EnvVar("TABLEAU_USERNAME"),
site_name=dg.EnvVar("MARKETING_TABLEAU_SITE_NAME"),
pod_name=dg.EnvVar("MARKETING_TABLEAU_POD_NAME"),
)

# We use Definitions.merge to combine the definitions from both workspaces
# into a single set of definitions to load
defs = Definitions.merge(
defs = dg.Definitions.merge(
sales_team_workspace.build_defs(),
marketing_team_workspace.build_defs(),
)
```

### Refresh and materialize Tableau assets

You can use Dagster to refresh Tableau workbooks and materialize Tableau sheets and dashboards.

```python file=/integrations/tableau/refresh-and-materialize-tableau-assets.py
from dagster_tableau import TableauCloudWorkspace

import dagster as dg

workspace = TableauCloudWorkspace(
connected_app_client_id=dg.EnvVar("TABLEAU_CONNECTED_APP_CLIENT_ID"),
connected_app_secret_id=dg.EnvVar("TABLEAU_CONNECTED_APP_SECRET_ID"),
connected_app_secret_value=dg.EnvVar("TABLEAU_CONNECTED_APP_SECRET_VALUE"),
username=dg.EnvVar("TABLEAU_USERNAME"),
site_name=dg.EnvVar("TABLEAU_SITE_NAME"),
pod_name=dg.EnvVar("TABLEAU_POD_NAME"),
)

defs = workspace.build_defs(
refreshable_workbook_ids=["b75fc023-a7ca-4115-857b-4342028640d0"]
)
```

Note that only workbooks created with extracts can be refreshed using this method. See more about [refreshing data sources](https://help.tableau.com/current/pro/desktop/en-us/refreshing_data.htm) in Tableau documentation website.

### Add a Data Quality Warning in Tableau using a sensor

When an upstream dependency of a Tableau asset fails to materialize or to pass the asset checks, it is possible to add a [Data Quality Warning](https://help.tableau.com/current/online/en-us/dm_dqw.htm) to the corresponding data source in Tableau. This can be achieved by leveraging the `add_data_quality_warning_to_data_source` in a sensor.

```python file=/integrations/tableau/add-tableau-data-quality-warning.py
from dagster_tableau import TableauCloudWorkspace

from dagster import (
Definitions,
EnvVar,
RunFailureSensorContext,
asset,
run_failure_sensor,
)
import dagster as dg

# Connect to Tableau Cloud using the connected app credentials
workspace = TableauCloudWorkspace(
connected_app_client_id=EnvVar("TABLEAU_CONNECTED_APP_CLIENT_ID"),
connected_app_secret_id=EnvVar("TABLEAU_CONNECTED_APP_SECRET_ID"),
connected_app_secret_value=EnvVar("TABLEAU_CONNECTED_APP_SECRET_VALUE"),
username=EnvVar("TABLEAU_USERNAME"),
site_name=EnvVar("TABLEAU_SITE_NAME"),
pod_name=EnvVar("TABLEAU_POD_NAME"),
connected_app_client_id=dg.EnvVar("TABLEAU_CONNECTED_APP_CLIENT_ID"),
connected_app_secret_id=dg.EnvVar("TABLEAU_CONNECTED_APP_SECRET_ID"),
connected_app_secret_value=dg.EnvVar("TABLEAU_CONNECTED_APP_SECRET_VALUE"),
username=dg.EnvVar("TABLEAU_USERNAME"),
site_name=dg.EnvVar("TABLEAU_SITE_NAME"),
pod_name=dg.EnvVar("TABLEAU_POD_NAME"),
)


@asset(
@dg.asset(
# Define which Tableau data source this upstream asset corresponds to
metadata={"dagster/tableau_data_source_id": "f5660c7-2b05-4ff0-90ce-3199226956c6"}
)
def upstream_asset(): ...


@run_failure_sensor
@dg.run_failure_sensor
def tableau_run_failure_sensor(
context: RunFailureSensorContext, tableau: TableauCloudWorkspace
context: dg.RunFailureSensorContext, tableau: TableauCloudWorkspace
):
asset_keys = context.dagster_run.asset_selection or set()
for asset_key in asset_keys:
Expand All @@ -204,11 +229,18 @@ def tableau_run_failure_sensor(
# We use Definitions.merge to combine the definitions from the Tableau workspace
# and the Dagster definitions into a single set of definitions to load
tableau_defs = workspace.build_defs()
upstream_defs = Definitions(
upstream_defs = dg.Definitions(
assets=[upstream_asset],
sensors=[tableau_run_failure_sensor],
resources={"tableau": workspace},
)

defs = Definitions.merge(tableau_defs, upstream_defs)
defs = dg.Definitions.merge(tableau_defs, upstream_defs)
```

### Related

- [`dagster-tableau` API reference](/\_apidocs/libraries/dagster-tableau)
- [Asset definitions](/concepts/assets/software-defined-assets)
- [Resources](/concepts/resources)
- [Using environment variables and secrets](/guides/dagster/using-environment-variables-and-secrets)
Binary file modified docs/next/public/objects.inv
Binary file not shown.
1 change: 1 addition & 0 deletions docs/sphinx/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,7 @@
sections/api/apidocs/libraries/dagster-spark
sections/api/apidocs/libraries/dagster-ssh
sections/api/apidocs/libraries/dagster-twilio
sections/api/apidocs/libraries/dagster-tableau
sections/api/apidocs/libraries/dagstermill
sections/api/apidocs/libraries/dagster-graphql
sections/api/apidocs/libraries/dagster-wandb
26 changes: 26 additions & 0 deletions docs/sphinx/sections/api/apidocs/libraries/dagster-tableau.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
#########################
Tableau (dagster-tableau)
#########################

Dagster allows you to represent your Tableau workspace as assets, alongside other your other
technologies like dbt and Sling. This allows you to see how your Tableau assets are connected to
your other data assets, and how changes to other data assets might impact your Tableau workspace.

.. currentmodule:: dagster_tableau

***********
Tableau API
***********

Here, we provide interfaces to manage Tableau projects using the Tableau API.

Assets (Tableau API)
====================

.. autoclass:: TableauCloudWorkspace

.. autoclass:: TableauServerWorkspace

.. autoclass:: DagsterTableauTranslator


2 changes: 2 additions & 0 deletions docs/tox.ini
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,7 @@ deps =
-e ../python_modules/libraries/dagster-openai
-e ../python_modules/libraries/dagster-looker
-e ../python_modules/libraries/dagster-sigma
-e ../python_modules/libraries/dagster-tableau
-e ../python_modules/libraries/dagster-powerbi

commands =
Expand Down Expand Up @@ -82,6 +83,7 @@ deps =
-e ../python_modules/libraries/dagster-openai
-e ../python_modules/libraries/dagster-looker
-e ../python_modules/libraries/dagster-sigma
-e ../python_modules/libraries/dagster-tableau
-e ../python_modules/libraries/dagster-powerbi

commands =
Expand Down
Original file line number Diff line number Diff line change
@@ -1,34 +1,28 @@
from dagster_tableau import TableauCloudWorkspace

from dagster import (
Definitions,
EnvVar,
RunFailureSensorContext,
asset,
run_failure_sensor,
)
import dagster as dg

# Connect to Tableau Cloud using the connected app credentials
workspace = TableauCloudWorkspace(
connected_app_client_id=EnvVar("TABLEAU_CONNECTED_APP_CLIENT_ID"),
connected_app_secret_id=EnvVar("TABLEAU_CONNECTED_APP_SECRET_ID"),
connected_app_secret_value=EnvVar("TABLEAU_CONNECTED_APP_SECRET_VALUE"),
username=EnvVar("TABLEAU_USERNAME"),
site_name=EnvVar("TABLEAU_SITE_NAME"),
pod_name=EnvVar("TABLEAU_POD_NAME"),
connected_app_client_id=dg.EnvVar("TABLEAU_CONNECTED_APP_CLIENT_ID"),
connected_app_secret_id=dg.EnvVar("TABLEAU_CONNECTED_APP_SECRET_ID"),
connected_app_secret_value=dg.EnvVar("TABLEAU_CONNECTED_APP_SECRET_VALUE"),
username=dg.EnvVar("TABLEAU_USERNAME"),
site_name=dg.EnvVar("TABLEAU_SITE_NAME"),
pod_name=dg.EnvVar("TABLEAU_POD_NAME"),
)


@asset(
@dg.asset(
# Define which Tableau data source this upstream asset corresponds to
metadata={"dagster/tableau_data_source_id": "f5660c7-2b05-4ff0-90ce-3199226956c6"}
)
def upstream_asset(): ...


@run_failure_sensor
@dg.run_failure_sensor
def tableau_run_failure_sensor(
context: RunFailureSensorContext, tableau: TableauCloudWorkspace
context: dg.RunFailureSensorContext, tableau: TableauCloudWorkspace
):
asset_keys = context.dagster_run.asset_selection or set()
for asset_key in asset_keys:
Expand All @@ -45,10 +39,10 @@ def tableau_run_failure_sensor(
# We use Definitions.merge to combine the definitions from the Tableau workspace
# and the Dagster definitions into a single set of definitions to load
tableau_defs = workspace.build_defs()
upstream_defs = Definitions(
upstream_defs = dg.Definitions(
assets=[upstream_asset],
sensors=[tableau_run_failure_sensor],
resources={"tableau": workspace},
)

defs = Definitions.merge(tableau_defs, upstream_defs)
defs = dg.Definitions.merge(tableau_defs, upstream_defs)
Loading

2 comments on commit 9811d73

@github-actions
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deploy preview for dagster-docs ready!

✅ Preview
https://dagster-docs-mvoa0skf0-elementl.vercel.app
https://master.dagster.dagster-docs.io

Built with commit 9811d73.
This pull request is being automatically deployed with vercel-action

@github-actions
Copy link

@github-actions github-actions bot commented on 9811d73 Oct 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deploy preview for dagster-docs-beta ready!

✅ Preview
https://dagster-docs-beta-3ja5lsdxr-elementl.vercel.app

Built with commit 9811d73.
This pull request is being automatically deployed with vercel-action

Please sign in to comment.