ORC-262: [C++] Support async io prefetch for orc c++ lib

[taiyang-li]

What changes were proposed in this pull request?

Support async io prefetch for orc c++ lib. Close https://issues.apache.org/jira/browse/ORC-262

Changes:

• Added new interface InputStream::readAsync(default unimplemented). It reads io asynchronously within the specified range.
• Added IO Cache implementation ReadRangeCache to cache async io results. This borrows from a similar design of Parquet Reader in https://github.com/apache/arrow
• Added interface Reader::preBuffer to trigger io prefetch. In the specific implementation of ReaderImpl::preBuffer, the io ranges will be calculated according to the selected stripe and columns, and then these ranges will be merged and sorted, and ReadRangeCache::cache will be called to trigger the asynchronous io in the background, waiting for the use of the upper layer
• Added the interface Reader::releaseBuffer, which is used to release all cached io ranges before an offset

Why are the changes needed?

Async io prefetch could hide io latency during reading orc files, which improves performance of scan operators in ClickHouse.

How was this patch tested?

Was this patch authored or co-authored using generative AI tooling?

[ffacs]

Reader::preBuffer prefetch stripes as a unit which might be too large. For those users who don't want to prefetch entire file one-shot, they have to know the structure of the file. Do you think it is a good idea to make prefetch transparent to users and let the orc reader prefetch data(eg. 1MB for each column at a time) when it's proper.
What's more, we could make enable async IO a option and expose a cache interface for users so they can implement their eviction policy.

[taiyang-li]

Reader::preBuffer prefetch stripes as a unit which might be too large. For those users who don't want to prefetch entire file one-shot, they have to know the structure of the file. Do you think it is a good idea to make prefetch transparent to users and let the orc reader prefetch data(eg. 1MB for each column at a time) when it's proper. What's more, we could make enable async IO a option and expose a cache interface for users so they can implement their eviction policy.

It is totally decided by users to choose whether to prefetch the whole orc file or single/multiple columns in single stripe or single column in single/multiple stripes. Reader::preBuffer already supported all those options.

It is better letting user invoke Reader::preBuffer explicitly because only user knows which stripe/columns to read. Thus they could find the best change to prefetch to hide io latency sufficiently. e.g. the orc prefetch implementation in ClickHouse relying on current PR: ClickHouse/ClickHouse#70534 (speed up 1.47x). Besides, the parquet reader in apache arrow also has similar design.

[taiyang-li]

@ffacs @wgtmac any more comments ? Thanks!

[wgtmac]

Sorry that I'm a little bit overwhelmed these days. Will take a look when I get the chance.

BTW, @luffy-zh is implementing exposing RowIndex positions: #2005. Perhaps there is an opportunity to further prefetch io together with predicate pushdown.

[taiyang-li]

@wgtmac That's a great work. We could do more improvements on IO latency hiding after it is merged.

[wgtmac]

I have just finished the initial review. Thanks @taiyang-li! Please see my inline comments. My main concern is the usability that it requires user to call preBuffer instead of automatically prefetching required data.