Towards a unified IO buffering approach in Ruby

[noteflakes]

attn @ioquatix

Ruby includes multiple APIs and mechanism for buffering data read or written to IOs (or fds):

• The IO class includes a buffering mechanism for both reading and writing.
• The StringIO API which mimics that of IO on top of a string, can effectively be used as a buffer for reading.
• The IO::Buffer API implements a subset of the IO API of top of its internal storage.
• The Net::BufferedIO API implements its own API, used by gems such as net-http
• The OpenSSL::Buffering::Buffer API mimics the IO API, in the openssl gem.

It is telling that there are so many similar buffering mechanisms and APIs used across core Ruby and its bundled gems. There's a clear overlap of functionality across these different APIs and mechanisms. The big question is: can we come up with a single API that will be able to handle all those different uses, including the internal buffering layer used by the IO class.

The Proposal

What I propose is to basically take the IO internal buffering mechanism, and turn it inside-out: instead of having the read and write buffers as an integral part of the IO implementation, we separate them out into separate objects. Each buffer object holds a reference to the IO instance, and implements the entire IO read/write API (e.g. getc, gets, each_line etc). The buffer has a position and invoking any of its read/write methods moves the position accordingly, just like a regular file. For read methods, whenever the buffer is exhausted, it is refilled from the associated IO. For write methods, whenever the buffer position crosses a certain threshold (which might be configurable), the buffer content is flushed to the associated IO.

Eeach IO instance has default read and write buffers, to which it delegates all of its read/write methods calls (except for sysread/pread). Here's a little sketch to show how the implementation can look:

class IO
  def default_read_buffer
    @default_read_buffer ||= Buffer.new(self)
  end
end

class Buffer
  def initialize(io)
    @io = io
    @buffer = +''
    @pos = 0
  end

  def getc
    if @pos >= @buffer.bytesize
      return nil if !fill_buffer
    end

    c = @buffer[@pos]
    @pos += 1
    c
  end

  def fill_buffer
    # the actual logic for filling the buffer would be more complicated than this:
    @pos = 0
    @io.read(8192, @buffer)
  end
end

Coalescing the Different Buffering APIs

Most if not all of the buffering APIs enumerated above can be coalesced into this single class. In fact, we can use the existing IO::Buffer and StringIO classes as the base for such functionality. The IO::Buffer class already provides the functionality for storing bytes, and the StringIO class already implements the different read/write methods on top of what is essentially a buffer of bytes.

Finally, any other APIs that need some specialized way of reading or writing data, could be reimplemented as subclasses of this unified buffer class.

Transport-agnostic IO

Such a unified buffering approach will also enable the creation of transport-agnostic protocol implementations. Here's an example: the h1p gem (of which I'm the author) implements a transport-agnostic way of parsing and sending an HTTP/1 request/response. It does so by detecting the read/write methods used by the associated IO instance.

In fact, since the different Socket classes implement the same API as IO, they can just delegate the entire read/write API to the default buffers, just like the IO class, except that the buffer will be filled by calling recv instead of read.

Advantages

There are multiple advantages to doing this:

• Separating the buffering from the IO class will greatly simplify the io.c implementation.
• Reduces the need for reimplementing the same IO API over different transports (sockets, SSL sockets, strings etc).
• Transport-agnostic also means much easier testing for anything IO-related.
• Easier to implement different protocols, as it provides a buffering mechanism coupled with the underlying IO.

[ioquatix]

I think you are looking in the right direction. My long term goal is to use IO::Buffer and rewrite IO in pure Ruby. I think there is a reasonable alignment between your ideas and what I want to do, long term. I think IO is too complex so I want to propose introducing IO::Handle or IO::Descriptor but this isn’t something I’ve discussed with anyone else yet.

[noteflakes]

Thanks @ioquatix for your thoughts.

Such a unified approach will also simplify a lot of what Polyphony is doing in terms of IO. Right now Polyphony reimplements the entire IO read/write API across the different classes (IO, TCPSocket, SSLSocket etc). With such a unified buffering API Polyphony would just need to override Buffer#fill_buffer.

Another consideration: it would be beneficial to also provide a set of C APIs for accessing the buffer. This would allow native extensions to use the buffer directly without converting the data into Ruby strings. It would greatly simplify implementing protocols such as HTTP/1, HTTP/2, Redis etc.

I'm currently working on taking the StringIO implementation and modifying it to read data from an associated IO instance using #sysread whenever it reaches the end of the buffer. I'll let you know when I have something that works :-).

[noteflakes]

I think IO is too complex so I want to propose introducing IO::Handle or IO::Descriptor but this isn’t something I’ve discussed with anyone else yet.

FWIW In Polyphony, all backend APIs support working with fds instead of IOs, you just pass the fd as an Integer.

[ioquatix]

Yes, it was my original design too, but I got told it wouldn't work with JRuby/TruffleRuby.