Improve cudf::io::datasource::create(). #17115
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
tpn
force-pushed
the
17110-fea-improve-cudfiodatasourcecreate
branch
from
44c3e36
to
a060422
Compare
Introduce new `datasource_kind` and `datasource_params` data types, and
update the cudf::io::datasource::create() signature to allow
parameterized datasource creation.
Additionally, implement new datasources:
- host_source: base class that does simple host-based pread() calls
- odirect_source: derivation of above that uses O_DIRECT
- kvikio_source: simple Kvikio-based class (that does not fall back
to mmap)
tpn
force-pushed
the
17110-fea-improve-cudfiodatasourcecreate
branch
from
a060422
to
fe981b1
Compare
tpn
changed the title
tpn
commented
Oct 21, 2024
| // parameter in the `kvikio_datasource_params`. | ||
| new_params.use_compat_mode = true; | ||
| } else if (kind == datasource_kind::KVIKIO_GDS) { | ||
| // GDS is unique in that we are expected to throw a cudf::runtime_error |
There was a problem hiding this comment.
Nit: s/runtime_error/logic_error/
tpn
commented
Oct 21, 2024
|
|
||
| /** | ||
| * @brief The threshold at which the data source will switch from using | ||
| * host-based reads to device-based (i.e. GPUDirect) reads, if GPUDirect is |
There was a problem hiding this comment.
Nit: I've used GDS everywhere else--change these GPUDirect occurrences to GDS too, for consistency.
tpn
commented
Oct 21, 2024
| @@ -92,15 +299,23 @@ class datasource { | |||
| * this case, `max_size_estimate` can include padding after the byte range, to include additional | |||
| * data that may be needed for processing. | |||
| * | |||
| * @throws cudf::logic_error if the minimum size estimate is greater than the maximum size estimate | |||
There was a problem hiding this comment.
Ah, dodgy rebase! This should be removed.
tpn
commented
Oct 21, 2024
| // Copy the user-provided parameters into our local variable. | ||
| new_params = *odirect_params; | ||
| } else { | ||
| throw cudf::logic_error("Invalid parameters for O_DIRECT-based datasource."); |
tpn
commented
Oct 21, 2024
| // Copy the user-provided parameters into our local variable. | ||
| new_params = *kvikio_params; | ||
| } else { | ||
| throw cudf::logic_error("Invalid parameters for KVIKIO-based datasource."); |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR introduces new functionality to
cudf::io::datasourcethat allows for greater control over backend datasource creation. Specifically, the static factorycreate()method has been expanded to takedatasource_kindanddatasource_paramsthat can be used to parameterize the datasource creation.Also introduced are three new datasources:
- host_source: base class that does simple host-based pread() calls
- odirect_source: derivation of above that uses O_DIRECT
- kvikio_source: simple Kvikio-based class (that does not fall back to mmap)
To NVIDIA cudf folks: this undoubtedly warrants some design discussion as I'm introducing new ways of doing things that might not align with in-flight or planned tasks for the datasource component. Happy to jump on a call and discuss. One thing that stands out to me is that the
KVIKIOvsKVIKIO_COMPATvsKVIKIO_GDSkinds feels a bit hacky and/or like a leaky abstraction, especially when you factor in alternate configuration like cufile or env vars.The idea behind the PR overall is that callers can have much more control over the exact kind of datasource they want created, including fast-fail if they're expecting to create a GDS-accelerated source but can't at runtime, for whatever reason.
The datasource_kind::ODIRECT is also extremely useful for eliminating the variance associated with the page cache when doing back to back runs of large data sets (where the presence or absence of data in the cache will have a huge impact on runtime).
Checklist