Add support for server-sent events

[hadley]

https://developer.mozilla.org/en-US/docs/Web/API/Server-sent%5Fevents/Using%5Fserver-sent%5Fevents#event_stream_format

Maybe req_perform_sse()? And then have some way to register functions for each event?

cutpoint <- function(bytes) {
  nl <- which(bytes == charToRaw("\n"))
  nl[nl %in% (setdiff(nl, 1) - 1)]
}

parser <- function(bytes) {
  text <- rawToChar(bytes)
  events <- strsplit(text, "\n\n")[[1]]
  lines <- strsplit(text, "\n")
  ...
}

req |> 
  req_perform_stream(round = cutpoint, callback = parser)

[hadley]

Done in #521

[lawremi]

Here is an alternative implementation that I was preparing to submit as a PR, but @jcheng5 beat me by a few weeks :)

It follows the simple approach outlined by @hadley above and mimics the Javascript EventSource API, with handlers for specific events. Just wanted to stick it here in case it is useful. The new connection-based streaming in #521 is more generally useful, of course.

req_perform_sse <- function(req, onmessage_callback = NULL,
                            event_callbacks = list(),
                            timeout_sec = Inf, buffer_kb = 64)
{
    callback <- function(bytes) {
        text <- rawToChar(bytes)
        msgs <- strsplit(text, "\n\n", fixed = TRUE)[[1L]]
        m <- gregexec("(event|data|id|retry): ?(.*)", msgs, perl = TRUE)
        for (mat in regmatches(msgs, m)) {
            event <- split(mat[3L,], mat[2L,])
            event$data <- paste(event$data, collapse = "\n")
            if (is.null(event$event))
                callback <- onmessage_callback
            else callback <- event_callbacks[[event$event]]
            if (!is.null(callback) && !isTRUE(callback(event)))
                return(FALSE)
        }
        TRUE
    }
    round <- function(bytes) {
        nl <- bytes == charToRaw("\n")
        which(diff(nl) == 0L & nl[-1L])
    }
    httr2::req_perform_stream(req, callback, timeout_sec, buffer_kb, round)
}

[lawremi]

I looked a bit into satisfying the draft spec, but the retry logic is a bit complicated. For example, I think req_retry() would need to gain the ability to modify the header on retries, in order to add the Last-Event-ID header.

[jcheng5]

I looked at it and thought for retry, it made more sense to implement the EventSource object exactly as it is in JS: a separate abstraction that sits one level above requests. I didn’t do it due to time pressure but also I am slightly skeptical that there is much adoption of Last-Event-ID in real world services (would love to find out I’m wrong though).

[lawremi]

Yea, it's still an evolving spec, so it makes sense to hold off until there is a use case.

[jcheng5]

Oh, I don’t think it’s evolving; the HTML whatwg spec is “living” so it’s never officially stable. The EventSource part of it is very old, there’s a w3c spec from back in the day.

If you wanted to write a full EventSource client just for completeness I think that would make sense and also be pretty fun. But I think we already have everything we would need or want for the LLM API cases in particular, as surely none of them implement retry?