Fix for dbt compile overwriting source files #10887
Conversation
|
Thanks for your pull request, and welcome to our community! We require contributors to sign our Contributor License Agreement and we don't seem to have your signature on file. Check out this article for more information on why we have a CLA. In order for us to review and merge your code, please submit the Individual Contributor License Agreement form attached above above. If you have questions about the CLA, or if you believe you've received this message in error, please reach out through a comment on this PR. CLA has not been signed by users: @Myles1 |
|
CLA should be signed now |
There was a problem hiding this comment.
Thanks for opening this PR @Myles1 !
Could you add a functional test for this case?
One place it could go initially is tests/functional/adapter/simple_seed/test_seed.py (when this is reviewed by one of our software engineers, they might request to remove it to a different location).
I'm guessing the test could look something like this (which I didn't try out at all, so probably doesn't work as-is):
import Path
from tests.functional.adapter.simple_seed.seeds import (
seed__with_dots_csv,
)
class BaseTestAbsoluteSeedPaths:
@pytest.fixture(scope="class")
def project_config_update(self, project_root):
# Assign an absolute path to seed-paths
return {
"seed-paths": [Path(project_root, "seeds")],
}
@pytest.fixture(scope="class")
def seeds(self):
return {"my_seed.csv": seed__with_dots_csv}
def test_absolute_seeds_paths(self, project):
results = run_dbt(["seed"])
assert len(results) == 1
# Should not fail due to file being overwritten
results = run_dbt(["seed"])
index.html
Outdated
There was a problem hiding this comment.
Could you restore the original index.html (assuming this was an accidental change)?
| @@ -0,0 +1,6 @@ | |||
| kind: Fixes | |||
| body: Prevent dbt compile from overwriting files that are passed in as absolute paths | |||
There was a problem hiding this comment.
I wasn't able to get an error with dbt compile, but I was with dbt seed and dbt build.
| body: Prevent dbt compile from overwriting files that are passed in as absolute paths | |
| body: Prevent `dbt seed` from overwriting files when `seed-path` contains an absolute path |
There was a problem hiding this comment.
Happy to update it if needed. Lmk after giving this a shot: #10886 (comment)
|
@cla-bot check |
|
The cla-bot has been summoned, and re-checked this pull request! |
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #10887 +/- ##
==========================================
- Coverage 89.18% 79.31% -9.88%
==========================================
Files 183 183
Lines 23443 23443
==========================================
- Hits 20908 18593 -2315
- Misses 2535 4850 +2315
Flags with carried forward coverage won't be shown. Click here to find out more.
|
Sure thing. Here's a (very) slightly modified version of your test that seems to do the trick: 9ae8aae |
Resolves #10886
Problem
Running
dbt compilecan overwrite source files with the newly compiled sqlSolution
Update path building to avoid issues with absolute paths.
The issue appears on this line:
dbt-core/core/dbt/contracts/graph/nodes.py
Line 267 in a0674db
Consider the case where
pathis an absolute path.For example, running this will give the absolute path as the return value, instead of the expected relative path that will be used as the output target file.
Following the change in this PR, we can expect this as the result:
Checklist