Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove PartitionedDataSet and IncrementalDataSet from kedro.io #2713

Closed
Tracked by #3043
jmholzer opened this issue Jun 21, 2023 · 3 comments · Fixed by #3187
Closed
Tracked by #3043

Remove PartitionedDataSet and IncrementalDataSet from kedro.io #2713

jmholzer opened this issue Jun 21, 2023 · 3 comments · Fixed by #3187
Assignees

Comments

@jmholzer
Copy link
Contributor

jmholzer commented Jun 21, 2023

Description

In tech design 21/06/23 we decided to migrate these datasets to kedro-datasets. This is a breaking change and needs to go into thedev branch for 0.19.

Check if any requirements can be removed from the core kedro package. E.g. pyarrow for testing.

See comment under #2456 for context.

@merelcht
Copy link
Member

Double check that Kedro isn't using these datasets in the Framework.

@noklam
Copy link
Contributor

noklam commented Sep 13, 2023

The confirm method only exist for these two datasets.

https://github.com/kedro-org/kedro/blob/955bc935d709ee85ecbb4c36e987162c2ae4f0ca/kedro/io/data_catalog.py#L747-L752C16

for name in node.confirms:
catalog.confirm(name)
return node

@merelcht merelcht moved this to To Do in Kedro Framework Sep 21, 2023
@SajidAlamQB SajidAlamQB moved this from To Do to In Progress in Kedro Framework Oct 17, 2023
@SajidAlamQB SajidAlamQB moved this from In Progress to In Review in Kedro Framework Oct 17, 2023
@noklam noklam linked a pull request Oct 17, 2023 that will close this issue
7 tasks
@SajidAlamQB SajidAlamQB moved this from In Review to Done in Kedro Framework Oct 24, 2023
@merelcht merelcht reopened this Oct 25, 2023
@github-project-automation github-project-automation bot moved this from Done to In Progress in Kedro Framework Oct 25, 2023
@merelcht
Copy link
Member

Completed in #3187

@github-project-automation github-project-automation bot moved this from In Progress to Done in Kedro Framework Oct 25, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

4 participants