Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Disable ci/cd spark image build when transform does not implement spark #604

Merged
merged 69 commits into from
Sep 20, 2024
Merged

Disable ci/cd spark image build when transform does not implement spark #604

merged 69 commits into from
Sep 20, 2024

Conversation

daw3rd
Copy link
Member

@daw3rd daw3rd commented Sep 20, 2024 •
edited
Loading

Not sure why the full history of commits its listed here since this branch was merged with dev below.
But, this changes the transforms test workflow to not build the spark image when not needed.

Why are these changes needed?

Speed up ci/cd builds for non-spark transforms

Related issue number (if any).

#515

@daw3rd
disable test workflow when none code files change
3fe8d2c 
Signed-off-by: David Wood <[email protected]>
@daw3rd
one more path-ignore in test.yml
86951f8 
Signed-off-by: David Wood <[email protected]>
@daw3rd
one more fix for path-ignore in test.yml
d368dc7 
Signed-off-by: David Wood <[email protected]>
@daw3rd
test universal transform separately
6d7c186 
Signed-off-by: David Wood <[email protected]>
@daw3rd
rename test universal workflow
35f9f31 
Signed-off-by: David Wood <[email protected]>
@daw3rd
add comments to noop src to trigger new universal test workflow
37cd7ef 
Signed-off-by: David Wood <[email protected]>
@daw3rd
fix paths in test universal workflow
a7fa50b 
Signed-off-by: David Wood <[email protected]>
@daw3rd
addj back ignore paths in test universal workflow
300d06a 
Signed-off-by: David Wood <[email protected]>
@daw3rd
another noop comment
0154179 
Signed-off-by: David Wood <[email protected]>
@daw3rd
move ignored paths to paths in univesal test workflow
5dd78e6 
Signed-off-by: David Wood <[email protected]>
@daw3rd
test-universal workflow name changes
d91da0d 
Signed-off-by: David Wood <[email protected]>
@daw3rd
noop comments
1be3292 
Signed-off-by: David Wood <[email protected]>
@daw3rd
noop readme change'
54d48b6 
Signed-off-by: David Wood <[email protected]>
@daw3rd
change test universal not paths
288ffdf 
Signed-off-by: David Wood <[email protected]>
@daw3rd
disable all but new noop and doc_id test workflows
668b70d 
Signed-off-by: David Wood <[email protected]>
@daw3rd
code change in noop
aa24ee7 
Signed-off-by: David Wood <[email protected]>
@daw3rd
remake test transforms
1d72cf5 
Signed-off-by: David Wood <[email protected]>
@daw3rd
Merge branch 'dev' into cicd-opt
142da80
@daw3rd
add individual test transform workflows
6bee51b 
Signed-off-by: David Wood <[email protected]>
@daw3rd
noop README change
db18c74 
Signed-off-by: David Wood <[email protected]>
@daw3rd
better ignore of .md on test transform workflows
42e18fc 
Signed-off-by: David Wood <[email protected]>
@daw3rd
noop readme change
ca09d75 
Signed-off-by: David Wood <[email protected]>
@daw3rd
noop test transform worklow 1 ignore
f16780b 
Signed-off-by: David Wood <[email protected]>
@daw3rd
noop readme
189bdd7 
Signed-off-by: David Wood <[email protected]>
@daw3rd
split out the tests into test-kfp/lib/misc and remove test.yml, add r…
54b3c69 
…eadme

Signed-off-by: David Wood <[email protected]>
@daw3rd
test-kfp only on kfp/**
d530ac9 
Signed-off-by: David Wood <[email protected]>
@daw3rd
noop code change to trigger build
781113f 
Signed-off-by: David Wood <[email protected]>
@daw3rd
comments in workflows
f09fbee 
Signed-off-by: David Wood <[email protected]>
@daw3rd
updated workflow readme
193f439 
Signed-off-by: David Wood <[email protected]>
@daw3rd
only run build-library workflow on data-processing-lib changes
3ed0fbd 
Signed-off-by: David Wood <[email protected]>
@daw3rd
noop src change
f419059 
Signed-off-by: David Wood <[email protected]>
@daw3rd
check for makefile in test transform workflow
a887bf9 
Signed-off-by: David Wood <[email protected]>
@daw3rd
automatically determine transforms in transforms directory for which …
f9399c1 
…to generate test workflows

Signed-off-by: David Wood <[email protected]>
@daw3rd
worklow readme, transform existence verification, disable build-libra…
e064e00 
…ry, and tools tests

Signed-off-by: David Wood <[email protected]>
@daw3rd
workflow readme details on kfp and misc tests
3a81d21 
Signed-off-by: David Wood <[email protected]>
@daw3rd
backing out change to dpk lib code
5d957ed 
Signed-off-by: David Wood <[email protected]>
@daw3rd
restore filter code
b02b52c 
Signed-off-by: David Wood <[email protected]>
@daw3rd
restore noop code
7cb4390 
Signed-off-by: David Wood <[email protected]>
@daw3rd
workflow readme
b780cf5 
Signed-off-by: David Wood <[email protected]>
@daw3rd
really restore noop code
9a5ccdc 
Signed-off-by: David Wood <[email protected]>
@daw3rd
check for makefile in transform test-src testing
93c56c2 
Signed-off-by: David Wood <[email protected]>
@daw3rd
don't include lib test dependencies in transform test workflows
8f8592d 
Signed-off-by: David Wood <[email protected]>
@daw3rd
noop code change
40ce888 
Signed-off-by: David Wood <[email protected]>
@daw3rd
disable noop, don't include lib test-data in transform dependencies
cb0d1f9 
Signed-off-by: David Wood <[email protected]>
@daw3rd
use job.id.if on Makefile to enable transform test job
d6c5784 
Signed-off-by: David Wood <[email protected]>
@daw3rd
use job.id.if on Makefile to enable transform test job
4f6b2f2 
Signed-off-by: David Wood <[email protected]>
@daw3rd
restore noop Makefile
c0f4935 
Signed-off-by: David Wood <[email protected]>
@daw3rd
exclude kfp_ray from transfor test workflow and change noop code
0a09daa 
Signed-off-by: David Wood <[email protected]>
@daw3rd
remove if: from test workflows
1bfb33d 
Signed-off-by: David Wood <[email protected]>
@daw3rd
backout noop code change
a80c0fd 
Signed-off-by: David Wood <[email protected]>
@daw3rd
backout noop code change
ee08033 
Signed-off-by: David Wood <[email protected]>
@daw3rd
only build spark image for transform image tests that need it
30a1524 
Signed-off-by: David Wood <[email protected]>
@daw3rd
header_cleanser code change
6d6a233 
Signed-off-by: David Wood <[email protected]>
@daw3rd
Merge branch 'dev' into cicd-opt
b062ae0
@daw3rd
only build spark image for transform image tests that need it
f495f82 
Signed-off-by: David Wood <[email protected]>
shivdeep-singh-ibm
shivdeep-singh-ibm approved these changes Sep 20, 2024
View reviewed changes
Copy link
Collaborator

@shivdeep-singh-ibm shivdeep-singh-ibm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@daw3rd daw3rd merged commit eba80df into dev Sep 20, 2024
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@shivdeep-singh-ibm shivdeep-singh-ibm shivdeep-singh-ibm approved these changes

@revit13 revit13 Awaiting requested review from revit13

@blublinsky blublinsky Awaiting requested review from blublinsky

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@daw3rd @shivdeep-singh-ibm