Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Debug #907

Closed
wants to merge 87 commits into from
Closed

Debug #907

wants to merge 87 commits into from

Conversation

merelcht
Copy link
Member

Description

Development notes

Checklist

  • Opened this PR as a 'Draft Pull Request' if it is work-in-progress
  • Updated the documentation to reflect the code changes
  • Added a description of this change in the relevant RELEASE.md file
  • Added tests to cover my changes
  • Received approvals from at least half of the TSC (required for adding a new, non-experimental dataset)

tdhooghe and others added 30 commits October 21, 2024 17:22
@tdhooghe
check and convert pd df to snowflake df
2305c3d 
Signed-off-by: tdhooghe <[email protected]>
@tdhooghe
add tests
07ef67b 
Signed-off-by: tdhooghe <[email protected]>
@tdhooghe
revert import to previous state
04928ba 
Signed-off-by: tdhooghe <[email protected]>
@tdhooghe
relax snowpark version
77551d6 
Signed-off-by: tdhooghe <[email protected]>
@tdhooghe
revert pyproject.toml
fbb3374 
Signed-off-by: tdhooghe <[email protected]>
@tdhooghe
bump snowpark version
fb621ae 
Signed-off-by: tdhooghe <[email protected]>
@tdhooghe
relax python version for snowpark
cb320bd 
Signed-off-by: tdhooghe <[email protected]>
@tdhooghe
downgrade snowpark to 1.22
628def6 
Signed-off-by: tdhooghe <[email protected]>
@tdhooghe
upgrade cloudpickle to 2.2.1
136fb09 
Signed-off-by: tdhooghe <[email protected]>
@deepyaman @tdhooghe
feat(datasets): create separate ibis.FileDataset (#842)
fe3bfbe 
* feat(datasets): create separate `ibis.FileDataset`

Signed-off-by: Deepyaman Datta <[email protected]>

* chore(datasets): deprecate `TableDataset` file I/O

Signed-off-by: Deepyaman Datta <[email protected]>

* feat(datasets): implement `FileDataset` versioning

Signed-off-by: Deepyaman Datta <[email protected]>

* chore(datasets): try `os.path.exists`, for Windows

Signed-off-by: Deepyaman Datta <[email protected]>

* revert(datasets): use pathlib, ignore Windows test

Refs: b7ff0c7

Signed-off-by: Deepyaman Datta <[email protected]>

* docs(datasets): add `ibis.FileDataset` to contents

Signed-off-by: Deepyaman Datta <[email protected]>

* chore(datasets): add docstring for `hashable` func

Signed-off-by: Deepyaman Datta <[email protected]>

* chore(datasets): add docstring for `hashable` func

Signed-off-by: Deepyaman Datta <[email protected]>

* feat(datasets)!: expose `load` and `save` publicly

Signed-off-by: Deepyaman Datta <[email protected]>

* chore(datasets): remove second filepath assignment

Signed-off-by: Deepyaman Datta <[email protected]>

---------

Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: tdhooghe <[email protected]>
@ankatiyar @tdhooghe
ci(docker): Update error code in e2e test (#888)
fce2974 
Update error code in e2e test

Signed-off-by: Ankita Katiyar <[email protected]>
Signed-off-by: tdhooghe <[email protected]>
@tdhooghe
update docstring
60b4a86 
Signed-off-by: tdhooghe <[email protected]>
@tdhooghe
test ci/cd without python version 3.12
3e578c6 
Signed-off-by: tdhooghe <[email protected]>
@tdhooghe
fix linting error
a161191 
Signed-off-by: tdhooghe <[email protected]>
@tdhooghe
add max python version for compatibility
e4509bc 
Signed-off-by: tdhooghe <[email protected]>
@tdhooghe
fix error
c98aef8 
Signed-off-by: tdhooghe <[email protected]>
@merelcht @deepyaman @tdhooghe
chore: Update PR template with checkbox for core dataset contribution (…
e4f1c3c 
…#891)

* Update PR template with checkbox for core dataset contribution

Signed-off-by: Merel Theisen <[email protected]>

* Update .github/PULL_REQUEST_TEMPLATE.md

Co-authored-by: Deepyaman Datta <[email protected]>
Signed-off-by: Merel Theisen <[email protected]>

* Fix lint

Signed-off-by: Merel Theisen <[email protected]>

---------

Signed-off-by: Merel Theisen <[email protected]>
Signed-off-by: Merel Theisen <[email protected]>
Co-authored-by: Deepyaman Datta <[email protected]>
Signed-off-by: tdhooghe <[email protected]>
@ankatiyar @tdhooghe
ci(datasets): Fix mypy errors (#893)
bcec735 
Signed-off-by: tdhooghe <[email protected]>
@deepyaman @tdhooghe
fix(datasets): default to DuckDB in in-memory mode (#897)
c243c6e 
* fix(datasets): default to DuckDB in in-memory mode

Signed-off-by: Deepyaman Datta <[email protected]>

* test(datasets): use `object()` sentinel as default

Signed-off-by: Deepyaman Datta <[email protected]>

* docs(datasets): add default database to RELEASE.md

Signed-off-by: Deepyaman Datta <[email protected]>

---------

Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: tdhooghe <[email protected]>
@deepyaman @tdhooghe
build(datasets): make Kedro-Datasets 5.1.0 release (#887)
388f455 
Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: tdhooghe <[email protected]>
@merelcht @tdhooghe
ci: Add GH action to check for TSC votes on core dataset changes (#896)
3c85b1f 
* Add GH action to check for TSC votes on core dataset changes
* Ignore TSC vote action in gatekeeper
* Trigger TSC vote action only on changes in core dataset

---------

Signed-off-by: Merel Theisen <[email protected]>
Signed-off-by: tdhooghe <[email protected]>
@tdhooghe
update testing framework and docstrings
101696c 
Signed-off-by: tdhooghe <[email protected]>
@tdhooghe
Update kedro-datasets/kedro_datasets/snowflake/snowpark_dataset.py
c1eeb77 
Signed-off-by: Thomas <[email protected]>
Signed-off-by: tdhooghe <[email protected]>
@tdhooghe
validate and check table name for nones
b9bf73e 
Signed-off-by: tdhooghe <[email protected]>
@tdhooghe
remove breakpoint
2c804c9 
Signed-off-by: tdhooghe <[email protected]>
@tdhooghe
change doc type hinting for tests to pass
7b7811d 
Signed-off-by: tdhooghe <[email protected]>
@deepyaman @tdhooghe
feat(datasets): create separate ibis.FileDataset (#842)
23b056a 
* feat(datasets): create separate `ibis.FileDataset`

Signed-off-by: Deepyaman Datta <[email protected]>

* chore(datasets): deprecate `TableDataset` file I/O

Signed-off-by: Deepyaman Datta <[email protected]>

* feat(datasets): implement `FileDataset` versioning

Signed-off-by: Deepyaman Datta <[email protected]>

* chore(datasets): try `os.path.exists`, for Windows

Signed-off-by: Deepyaman Datta <[email protected]>

* revert(datasets): use pathlib, ignore Windows test

Refs: b7ff0c7

Signed-off-by: Deepyaman Datta <[email protected]>

* docs(datasets): add `ibis.FileDataset` to contents

Signed-off-by: Deepyaman Datta <[email protected]>

* chore(datasets): add docstring for `hashable` func

Signed-off-by: Deepyaman Datta <[email protected]>

* chore(datasets): add docstring for `hashable` func

Signed-off-by: Deepyaman Datta <[email protected]>

* feat(datasets)!: expose `load` and `save` publicly

Signed-off-by: Deepyaman Datta <[email protected]>

* chore(datasets): remove second filepath assignment

Signed-off-by: Deepyaman Datta <[email protected]>

---------

Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: tdhooghe <[email protected]>
@ankatiyar @tdhooghe
ci(datasets): Fix mypy errors (#893)
27e58cd 
Signed-off-by: tdhooghe <[email protected]>
@tdhooghe
more strict type hinting and casting for linting
178f0f4 
Signed-off-by: tdhooghe <[email protected]>
@tdhooghe
add back python 3.12 to see if it is compatible
0133025 
Signed-off-by: tdhooghe <[email protected]>
tdhooghe and others added 28 commits October 22, 2024 16:14
@tdhooghe
improve docstring and remove ignore from dataset-doctest
0886454 
Signed-off-by: tdhooghe <[email protected]>
@tdhooghe
Merge branch 'main' into feature/save-pd-to-snowflaketable
25c28c4 
Signed-off-by: Thomas <[email protected]>
@tdhooghe
add back ignore snowflake for doctesting and remove unexpected additi…
624baa7 
…ons to pyproject.toml

Signed-off-by: tdhooghe <[email protected]>
@tdhooghe
only skip for python 3.12 or higher
80f7765 
Signed-off-by: tdhooghe <[email protected]>
@tdhooghe
remove separate pytest skips as we have a module wide one
ba0a030 
Signed-off-by: tdhooghe <[email protected]>
@tdhooghe
remove test from omit in pytproject.toml
36264c2 
Signed-off-by: tdhooghe <[email protected]>
@tdhooghe
100 test coverage
fe43b5c 
Signed-off-by: tdhooghe <[email protected]>
@tdhooghe
bump matplotlib to prevent installing test requirements from failing
10829ed 
Signed-off-by: tdhooghe <[email protected]>
@tdhooghe
revert matplotlib version bump
435159b 
Signed-off-by: tdhooghe <[email protected]>
@tdhooghe
exclude lines from coverage
ba6975c 
Signed-off-by: tdhooghe <[email protected]>
@tdhooghe
only test python 3.12
52ce490 
Signed-off-by: tdhooghe <[email protected]>
@tdhooghe
omit snowflake testing from python 3.12
71a3801 
Signed-off-by: tdhooghe <[email protected]>
@tdhooghe
add python version check
b89a552 
Signed-off-by: tdhooghe <[email protected]>
@tdhooghe
fix bug in ignoreopts and add back python versions 3.10 and 3.11 in m…
d967ff5 
…atrix

Signed-off-by: tdhooghe <[email protected]>
@tdhooghe
fix bug in ignoreopts and add back python versions 3.10 and 3.11 in m…
233b4f0 
…atrix

Signed-off-by: tdhooghe <[email protected]>
@tdhooghe
revert requires-python to 3.10
7349fac 
Signed-off-by: tdhooghe <[email protected]>
@tdhooghe
ignore entire folder
e64551e 
Signed-off-by: tdhooghe <[email protected]>
@tdhooghe
add back omit for snowflake as python 3.12 is not supported, hence ci…
ceacc8c 
…cd fails on coverage

Signed-off-by: tdhooghe <[email protected]>
@tdhooghe
revert makefile to original state
e0735ad 
Signed-off-by: tdhooghe <[email protected]>
@tdhooghe
remove commented out setters
d17b8a6 
Signed-off-by: tdhooghe <[email protected]>
@tdhooghe
remove commented out fixtures
c5c9e4d 
Signed-off-by: tdhooghe <[email protected]>
@tdhooghe
Merge branch 'main' into feature/save-pd-to-snowflaketable
fbc2545
@merelcht
Skip instead of xfail
f4383c7 
Signed-off-by: Merel Theisen <[email protected]>
@merelcht
clean up
6f85afd 
Signed-off-by: Merel Theisen <[email protected]>
@merelcht
Try skip import
97cb8af 
Signed-off-by: Merel Theisen <[email protected]>
@merelcht
Try only import skip
8e8fa8a 
Signed-off-by: Merel Theisen <[email protected]>
@merelcht
Fix lint
f9263d1 
Signed-off-by: Merel Theisen <[email protected]>
@merelcht
lint
c8e74ef 
Signed-off-by: Merel Theisen <[email protected]>
@merelcht merelcht closed this Oct 24, 2024
@merelcht merelcht deleted the debug branch October 24, 2024 15:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@merelcht @tdhooghe @deepyaman @ankatiyar