Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Core] 1.9 upgrade guide #6184

Merged
merged 37 commits into from
Oct 3, 2024
Merged

[Core] 1.9 upgrade guide #6184

merged 37 commits into from
Oct 3, 2024

Conversation

runleonarun
Copy link
Collaborator

@runleonarun runleonarun commented Oct 1, 2024

Closes #6159

What are you changing in this pull request and why?

First draft of upgrade guide. This is the most current content!

Checklist

  • I have reviewed the Content style guide so my content adheres to these guidelines.
  • The topic I'm writing about is for specific dbt version(s) and I have versioned it according to the version a whole page and/or version a block of content guidelines.
  • I have added checklist item(s) to this list for anything anything that needs to happen before this PR is merged, such as "needs technical review" or "change base branch."

@runleonarun runleonarun requested a review from a team as a code owner October 1, 2024 01:01
Copy link

vercel bot commented Oct 1, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Updated (UTC)
docs-getdbt-com ✅ Ready (Inspect) Visit Preview Oct 3, 2024 6:04pm

@github-actions github-actions bot added content Improvements or additions to content size: medium This change will take up to a week to address Docs team Authored by the Docs team @dbt Labs labels Oct 1, 2024
@runleonarun runleonarun changed the title adding upgrade guide [Core] 1.9 upgrade guide Oct 1, 2024
Copy link
Collaborator

@jtcohen6 jtcohen6 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @runleonarun - we're doing it! 🎉


While this works for many use-cases, there’s a clear limitation with this approach: *Some datasets are just too big to fit into one query.*

Starting in Core 1.9, you can use the new microbatch strategy to optimize your largest datasets -- **process your event data in discrete periods with their own SQL queries, rather than all at once.** The benefits include:
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Starting in Core 1.9, you can use the new microbatch strategy to optimize your largest datasets -- **process your event data in discrete periods with their own SQL queries, rather than all at once.** The benefits include:
Starting in Core 1.9, you can use the new [microbatch strategy](/docs/build/incremental-microbatch) to optimize your largest datasets -- **process your event data in discrete periods with their own SQL queries, rather than all at once.** The benefits include:

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

blocked on #6194

Copy link
Collaborator

@graciegoheen graciegoheen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couple small comments - but otherwise this is ready to go

@runleonarun runleonarun enabled auto-merge (squash) October 3, 2024 17:59
@runleonarun runleonarun merged commit fc5d053 into current Oct 3, 2024
6 checks passed
@runleonarun runleonarun deleted the upgrade-1.9-guide branch October 3, 2024 18:04
@@ -71,7 +71,6 @@ Read more about [Snapshots meta fields](/docs/build/snapshots#snapshot-meta-fiel
We’ve made improvements to `state:modified` behaviors to help reduce the risk of false positives and negatives. Read more about [the `state:modified` behavior flags](#managing-changes-to-legacy-behaviors) that leverage these improvements:

- Added environment-aware enhancements for environments where the logic purposefully differs (for example, materializing as a table in `prod` but a `view` in dev).
- Models that use `var` or `env_var` in their definition are included in `state:modified` when their values change.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added to #6222

@@ -80,7 +79,6 @@ dbt Core v1.9 has a handful of new flags for [managing changes to legacy behavi
You can read more about each of these behavior changes in the following links:

- (Introduced, disabled by default) [`state_modified_compare_more_unrendered_values`](/reference/global-configs/behavior-changes#behavior-change-flags). Set to `True` to start persisting unrendered_database and unrendered_schema configs during source parsing, and do comparison on unrendered values during `state:modified` checks.
- (Introduced, disabled by default) [`state_modified_compare_vars`](/reference/global-configs/behavior-changes#behavior-change-flags). Set to `True` if a model uses a `var` or `env_var` in its definition. dbt will be able to identify its lineage to include the model in `state:modified` because the var or env_var value has changed.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added to #6222

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
content Improvements or additions to content Docs team Authored by the Docs team @dbt Labs size: medium This change will take up to a week to address
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Core] Add content to v1.9 upgrade guide for beta
7 participants