Skip to content

Commit

Permalink
fix: only select models in graph.nodes
Browse files Browse the repository at this point in the history
  • Loading branch information
gsaiz committed Jul 15, 2023
1 parent 2012ee8 commit 0a521e8
Show file tree
Hide file tree
Showing 4 changed files with 12 additions and 6 deletions.
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,13 +19,15 @@ This macro generates a series of terminal commands (appended w) bash script whic
- Add optional arguments to include database and schema properties in `sources.yml` generated from `generate_source` ([#123](https://github.com/dbt-labs/dbt-codegen/issues/123))

## Fixes
- Fix only selecting nodes that are models when using the `generate_model_yaml` macro with `upstream_descriptions=True`
- Fix handling of nested `STRUCT` fields in BigQuery ([#98](https://github.com/dbt-labs/dbt-codegen/issues/98), [#105](https://github.com/dbt-labs/dbt-codegen/pull/105))

## Quality of life
- Addition of the [base_model_creation](bash_scripts/base_model_creation.sh) bash script which allows users to input multiple tables as a list and generate a terminal command that will combine **all** [create_base_models](macros/create_base_models.sql) commands. This way, you can generate base models for all your sources at once.
- Instructions for contributing ([#99](https://github.com/dbt-labs/dbt-codegen/issues/99), [#104](https://github.com/dbt-labs/dbt-codegen/pull/104))

## Contributors:
- [@gsaiz](https://github.com/gsaiz) ([#132](https://github.com/dbt-labs/dbt-codegen/pull/132))
- [@fivetran-joemarkiewicz](https://github.com/fivetran-joemarkiewicz) (#83)
- [@GSokol](https://github.com/GSokol) (#76)

Expand Down
4 changes: 2 additions & 2 deletions integration_tests/tests/test_generate_source_all_args.sql
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,8 @@ version: 2
sources:
- name: {{ raw_schema | trim | lower }}
description: ""
database: analytics
schema: codegen_integration_tests_snowflake_raw_data
database: {{ target.database }}
schema: {{ target.schema }}_raw_data
tables:
- name: data__a_relation
description: ""
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ version: 2

sources:
- name: {{ raw_schema | trim | lower }}
database: analytics
database: {{ target.database }}
tables:
- name: data__a_relation
- name: data__b_relation
Expand Down
10 changes: 7 additions & 3 deletions macros/helpers/helpers.sql
Original file line number Diff line number Diff line change
@@ -1,14 +1,18 @@
{# retrieve models directly upstream from a given model #}
{% macro get_model_dependencies(model_name) %}
{% for node in graph.nodes.values() | selectattr('name', "equalto", model_name) %}
{% for node in graph.nodes.values()
| selectattr('resource_type', 'equalto', 'model')
| selectattr('name', 'equalto', model_name) %}
{{ return(node.depends_on.nodes) }}
{% endfor %}
{% endmacro %}


{# add to an input dictionary entries containing all the column descriptions of a given model #}
{% macro add_model_column_descriptions_to_dict(model_name,dict_with_descriptions={}) %}
{% for node in graph.nodes.values() | selectattr('name', "equalto", model_name) %}
{% for node in graph.nodes.values()
| selectattr('resource_type', 'equalto', 'model')
| selectattr('name', 'equalto', model_name) %}
{% for col_name, col_values in node.columns.items() %}
{% do dict_with_descriptions.update( {col_name: col_values.description} ) %}
{% endfor %}
Expand All @@ -32,7 +36,7 @@
{# filter by directory or prefix arguments, if provided #}
{% macro get_models(directory=None, prefix=None) %}
{% set model_names=[] %}
{% set models = graph.nodes.values() | selectattr('resource_type', "equalto", 'model') %}
{% set models = graph.nodes.values() | selectattr('resource_type', 'equalto', 'model') %}
{% if directory and prefix %}
{% for model in models %}
{% set model_path = "/".join(model.path.split("/")[:-1]) %}
Expand Down

0 comments on commit 0a521e8

Please sign in to comment.