Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix for dbt compile overwriting source files #10887

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
+31 −1
Open

Fix for dbt compile overwriting source files #10887

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
e98e3a7
Fix: Stop overwriting absolute paths
Myles1 Oct 18, 2024
2b18d70
changie new
Myles1 Oct 18, 2024
dc370bf
remove extra file
Myles1 Oct 23, 2024
9ae8aae
add regression test for overwriting absolute paths
Myles1 Oct 23, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .changes/unreleased/Fixes-20241018-150256.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
kind: Fixes
body: Prevent dbt compile from overwriting files that are passed in as absolute paths
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wasn't able to get an error with dbt compile, but I was with dbt seed and dbt build.

Suggested change
body: Prevent dbt compile from overwriting files that are passed in as absolute paths
body: Prevent `dbt seed` from overwriting files when `seed-path` contains an absolute path

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Happy to update it if needed. Lmk after giving this a shot: #10886 (comment)

time: 2024-10-18T15:02:56.289808-07:00
custom:
Author: Myles1
Issue: "10886"
2 changes: 1 addition & 1 deletion core/dbt/contracts/graph/nodes.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -251,7 +251,7 @@ def get_target_write_path(
# This is called for both the "compiled" subdirectory of "target" and the "run" subdirectory
if os.path.basename(self.path) == os.path.basename(self.original_file_path):
# One-to-one relationship of nodes to files.
path = self.original_file_path
path = self.path
else:
# Many-to-one relationship of nodes to files.
path = os.path.join(self.original_file_path, self.path)
Expand Down
24 changes: 24 additions & 0 deletions tests/functional/adapter/simple_seed/test_seed.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -386,3 +386,27 @@ def test_empty_seeds(self, project):

class TestEmptySeed(BaseTestEmptySeed):
pass


class TestAbsoluteSeedPaths:
"""
This is a regression test for issue #10886.
We're ensuring that dbt will not overwrite files that are passed in as asbolute paths in dbt_project.yml
"""

@pytest.fixture(scope="class")
def project_config_update(self, project_root):
# Assign an absolute path to seed-paths
return {
"seed-paths": [str(Path(project_root) / "seeds")],
}

@pytest.fixture(scope="class")
def seeds(self):
return {"my_seed.csv": seed__with_dots_csv}

def test_absolute_seeds_paths(self, project):
results = run_dbt(["seed"])
assert len(results) == 1
# Should not fail due to file being overwritten
results = run_dbt(["seed"])