You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I have searched the existing issues, and I could not find an existing issue for this feature
I am requesting a straightforward extension of existing dbt functionality, rather than a Big Idea better suited to a discussion
Describe the feature
We have a use case for dynamic assignment of database names for models which select a database name based on the currently authenticated user to allow for usage of developer-specific data sets during development. While we have been able to implement a custom generate_database_name macro for this functionality on models, we cannot apply this same logic on sources as dbt does not support using the generate_X_name macros for dynamic assignment of either database or schema properties for sources.
For context, we currently use logic similar to below to assign database names within sources properties files.
While this functionally works, we have to copy and paste this code for every single source property file in our project to reuse this logic for each source. This exact same logic is contained within our generate_database_name macro so it would be useful to be able to reuse that and not violate DRY for every new source.
I therefore suggest one of the following approaches to bring source configuration more in-line with model configuration to allow for this functionality with reference to the above example on how this would improve it.
Have sources properties use the generate_X_name macros automatically. e.g.
sources:
- name: source_xdatabase: "reporting"# This would then use the `generate_database_name` macro to use the same logic as above
Allow usage of generate_X_name macros within source properties files to allow us to unify handling of these properties for both sources and models.
If this is not possible due to design choices, bring parity of source properties with other resource types to allow for definitions at the project file level to pass down to all sources within a resource path as is possible with models. This will at least allow us to define the logic for sources once at the project file level.
Manually copying and pasting database jinja templates across all source properties which violates DRY.
Who will this benefit?
Anyone who uses dynamic database or schema assignment for sources in a similar way to models. Specifically this will benefit users who need developer specific data environment handling for sources.
Are you interested in contributing this feature?
Yes
Anything else?
I am aware there are stale discussions for all of the suggestions above as described here and here.
These discussions are stale and closed so I am consolidating these here in an effort to unify discussion on this issue.
The text was updated successfully, but these errors were encountered:
Is this your first time submitting a feature request?
Describe the feature
We have a use case for dynamic assignment of database names for models which select a database name based on the currently authenticated user to allow for usage of developer-specific data sets during development. While we have been able to implement a custom
generate_database_name
macro for this functionality on models, we cannot apply this same logic on sources as dbt does not support using thegenerate_X_name
macros for dynamic assignment of eitherdatabase
orschema
properties for sources.For context, we currently use logic similar to below to assign database names within sources properties files.
While this functionally works, we have to copy and paste this code for every single source property file in our project to reuse this logic for each source. This exact same logic is contained within our
generate_database_name
macro so it would be useful to be able to reuse that and not violate DRY for every new source.I therefore suggest one of the following approaches to bring source configuration more in-line with model configuration to allow for this functionality with reference to the above example on how this would improve it.
generate_X_name
macros automatically. e.g.generate_X_name
macros within source properties files to allow us to unify handling of these properties for both sources and models.Describe alternatives you've considered
Manually copying and pasting database jinja templates across all source properties which violates DRY.
Who will this benefit?
Anyone who uses dynamic database or schema assignment for sources in a similar way to models. Specifically this will benefit users who need developer specific data environment handling for sources.
Are you interested in contributing this feature?
Yes
Anything else?
I am aware there are stale discussions for all of the suggestions above as described here and here.
These discussions are stale and closed so I am consolidating these here in an effort to unify discussion on this issue.
The text was updated successfully, but these errors were encountered: