You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently we don't support these, since we have a custom delta Parquet writer (i.e. plan_to_object_store) as a way to mitigate memory performance of writes.
This could be achieved in two ways:
Ditch our custom writing logic and use delta-rs one, which has support for partitioning columns at the expense of getting sub-optimal memory profiles.
Implement partitioning into our own writing logic.
The text was updated successfully, but these errors were encountered:
Currently we don't support these, since we have a custom delta Parquet writer (i.e.
plan_to_object_store
) as a way to mitigate memory performance of writes.This could be achieved in two ways:
The text was updated successfully, but these errors were encountered: