Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update KFP super pipeline for code #388

Open
Bytes-Explorer opened this issue Jul 8, 2024 · 5 comments
Open

Update KFP super pipeline for code #388

Bytes-Explorer opened this issue Jul 8, 2024 · 5 comments
Assignees

Comments

@Bytes-Explorer
Copy link
Collaborator

Bytes-Explorer commented Jul 8, 2024

Update the KFP super-pipeline for code modules with the new code2parquet transform and license transforms.

@Bytes-Explorer Bytes-Explorer changed the title KFP super pipeline for code Update KFP super pipeline for code Jul 8, 2024
@roytman
Copy link
Member

roytman commented Jul 8, 2024

@Bytes-Explorer , do you have a specific place where the license transformer should be put in the super pipeline?

@roytman
Copy link
Member

roytman commented Jul 8, 2024

the code super pipeline uses the code2parquettransformer

@Bytes-Explorer
Copy link
Collaborator Author

It can go after malware. Can the super pipeline be automatically generated based on the order of transforms given as input? For eg, I may want to call the filter operation after every operation, if I have a lot of data.

@roytman
Copy link
Member

roytman commented Jul 9, 2024

there are two PRs with license-related transforms, and both are not merged.
#332
#257

@Bytes-Explorer
Copy link
Collaborator Author

Both of these are separate transforms and both need to be added. Yes there is a dependency here. @Param-S @shivdeep-singh-ibm

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants