-
Notifications
You must be signed in to change notification settings - Fork 128
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Crawler transform #797
base: dev
Are you sure you want to change the base?
Crawler transform #797
Commits on Nov 8, 2024
-
first implementation of web2parquet for crawling/downloading from see…
…dURLs Signed-off-by: Maroun Touma <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 41bed68 - Browse repository at this point
Copy the full SHA 41bed68View commit details
Commits on Nov 11, 2024
-
Signed-off-by: Maroun Touma <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for cf516b5 - Browse repository at this point
Copy the full SHA cf516b5View commit details
Commits on Nov 13, 2024
-
complete full implementation and testing with python runtime
Signed-off-by: Maroun Touma <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for acc35cd - Browse repository at this point
Copy the full SHA acc35cdView commit details -
identified current requirements for web2parquet module
Signed-off-by: Maroun Touma <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 3e05f30 - Browse repository at this point
Copy the full SHA 3e05f30View commit details -
Configuration menu - View commit details
-
Copy full SHA for 5710653 - Browse repository at this point
Copy the full SHA 5710653View commit details -
Configuration menu - View commit details
-
Copy full SHA for 80e4ebe - Browse repository at this point
Copy the full SHA 80e4ebeView commit details -
Configuration menu - View commit details
-
Copy full SHA for cf20268 - Browse repository at this point
Copy the full SHA cf20268View commit details
Commits on Nov 14, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 4dcebb6 - Browse repository at this point
Copy the full SHA 4dcebb6View commit details -
Configuration menu - View commit details
-
Copy full SHA for 137d92c - Browse repository at this point
Copy the full SHA 137d92cView commit details -
Configuration menu - View commit details
-
Copy full SHA for d2404f4 - Browse repository at this point
Copy the full SHA d2404f4View commit details -
generate cicd workflow for new transform
Signed-off-by: Maroun Touma <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 1e810d0 - Browse repository at this point
Copy the full SHA 1e810d0View commit details -
build image only if a Dockerfile is defined
Signed-off-by: Maroun Touma <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for fcbcc0a - Browse repository at this point
Copy the full SHA fcbcc0aView commit details -
Ignore page content as long as we get the right count
Signed-off-by: Maroun Touma <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for b5031c9 - Browse repository at this point
Copy the full SHA b5031c9View commit details
Commits on Nov 15, 2024
-
Signed-off-by: Maroun Touma <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 9ad3d18 - Browse repository at this point
Copy the full SHA 9ad3d18View commit details -
Signed-off-by: Maroun Touma <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for c9c9779 - Browse repository at this point
Copy the full SHA c9c9779View commit details -
Signed-off-by: Maroun Touma <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for b77bbe9 - Browse repository at this point
Copy the full SHA b77bbe9View commit details