How are composite solids executed?

[szalai1]

Hi,

if I have a composite solid, will that be executed as one single solid on k8s-celery or it's just a way to make complex solids to make more user-friendly?
What about k8s configurations, if I tag the composite solid and the inner ones with different k8s resource tags for example, which one will be passed to the actual execution?

[alangenfeld]

will that be executed as one single solid on k8s-celery or it's just a way to make complex solids to make more user-friendly?

Just a way to make complex DAG structures more user friendly. When the "pipeline" graph of solids and composite solids is compiled in to an ExecutionPlan all of the composition melts away and you end up with a flat graph of ExecutionSteps that correspond with the Solids which may have been wrapped in composites.

What about k8s configurations, if I tag the composite solid and the inner ones with different k8s resource tags for example, which one will be passed to the actual execution?

I believe it should be what ever ended up on the solid invocation. Tags on the composite solid only effect dagit display at this time. Note that you can set additional tags or override them at the solid invocation site

@solid(tags={'foo': '1', 'bar': '1'})
def example(_):
  pass

@composite_solid
def compose():
  example.tag({'foo': '2'})()

@pipeline
def pipe():
  compose()

The resulting solid invocation compose.example will have tags foo=2 and bar=1

[szalai1]

Thank you very much for the explanation, it makes sense.