Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Moving from flyteplugins - Implement Spark pod template tolerations #4130

Closed

Conversation

eapolinario
Copy link
Contributor

TL;DR

Populate driver and executor tolerations from optional pod templates. This is an initial step narrowly focused on pod tolerations, to be followed by a more generic solution that handles all pod template functionality.

Type

  • Bug Fix
  • Feature
  • Plugin

Are all requirements met?

  • Code completed
  • Smoke tested
  • Unit tests added
  • Code documentation added
  • Any pending items have an associated Issue

Complete description

This change builds on optional Spark pod template changes added in flyteorg/flyteidl#450. For now appends tolerations from the driver and executor pod templates, if any, to the defaults used to build the SparkApplicationSpec. A follow up PR will refactor the Spark plugin and pod_helper.go to reuse ToK8sPodSpec in order to consume other aspect of pod templates, as is done for other plugins.

Tracking Issue

#4105

@codecov
Copy link

codecov bot commented Oct 3, 2023

Codecov Report

All modified lines are covered by tests ✅

Comparison is base (d9586b0) 58.98% compared to head (b9f31a0) 59.50%.

❗ Current head b9f31a0 differs from pull request most recent head a086d07. Consider uploading reports for the commit a086d07 to get more accurate results

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #4130      +/-   ##
==========================================
+ Coverage   58.98%   59.50%   +0.52%     
==========================================
  Files         619      216     -403     
  Lines       52804    15938   -36866     
==========================================
- Hits        31146     9484   -21662     
+ Misses      19174     5479   -13695     
+ Partials     2484      975    -1509     
Flag Coverage Δ
unittests ?

Flags with carried forward coverage won't be shown. Click here to find out more.

see 600 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@andrewwdye andrewwdye force-pushed the monorepo-move/flyteplugins/spark-pod-templates branch from 1506281 to 2babaab Compare October 3, 2023 23:00
Signed-off-by: Andrew Dye <[email protected]>
@andrewwdye andrewwdye force-pushed the monorepo-move/flyteplugins/spark-pod-templates branch from 2babaab to a086d07 Compare October 3, 2023 23:01
@andrewwdye
Copy link
Contributor

Closing in favor of #4183

@andrewwdye andrewwdye closed this Oct 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants