what's the exact plan for torchdata now?

[keunwoochoi]

hi, as a user of torchdata, i'm very happy to see the resurrection of the project.

i have a question about the development plan. from the README, i see:

torchdata repo to be an iterative enhancement of torch.utils.data.DataLoader

this is somewhat surprising. although the current Datapipes seem to have various issues underneath the shell, so far, Datapipes ARE torchdata. the current API reference:

API Reference:

Stateful DataLoader
Iterable-style DataPipes
Map-style DataPipes
Utility Functions
DataLoader2
ReadingService

and this is it; i.e., until ver 0.7, torchdata == the datapipes and other necessary utilities (dataloader2 and reading service).

and that's why it is surprising for me, that while the development of torchdata has re-started, it is being done in a way it discards everything it had.

so, can i ask for a bit more details about what the new direction (enhancement of torch.utils.data.DataLoader)? or am i missing something here?

thanks.

[keunwoochoi]

moving my question to #1196