Skip to content

Commit

Permalink
ResponseLazy: Work in buffers, not bytes
Browse files Browse the repository at this point in the history
When reading a response lazily, we want to work with it efficiently,
as buffers of bytes coming out of the OS TCP stack. While we could
`Iterator::collect::<Vec<u8>>()` the bytes back together from the
current interface, this incurs an extra copy. (And unless optimizations
can miraculously burn through all our iteration logic, we'll get an
inefficient byte-by-byte copy at that!)

Instead, have ResponseLazy just implement Read.
Users who want individual bytes can get them with std::io::Bytes;
that's what it's for.

While we're at it, don't lie about content-length headers that weren't
actually there for a chunked transfer.
  • Loading branch information
mrkline committed Jan 18, 2024
1 parent 38481c8 commit 1a7bf12
Show file tree
Hide file tree
Showing 5 changed files with 107 additions and 170 deletions.
2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "minreq"
version = "2.11.1-alpha"
version = "3.0.0-alpha"
authors = ["Jens Pitkanen <[email protected]>"]
description = "Simple, minimal-dependency HTTP client"
documentation = "https://docs.rs/minreq"
Expand Down
8 changes: 8 additions & 0 deletions examples/download.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
/// This example shows how a [`Response`] is a [`std::io::Read`],
/// so it can be easily copied to [`std::io::write`], such as stdout
/// or a file.
fn main() -> Result<(), minreq::Error> {
let mut response = minreq::get("http://example.com").send_lazy()?;
std::io::copy(&mut response, &mut std::io::stdout().lock())?;
Ok(())
}
13 changes: 3 additions & 10 deletions examples/iterator.rs
Original file line number Diff line number Diff line change
@@ -1,24 +1,17 @@
/// This example demonstrates probably the most complicated part of
/// `minreq`. Useful when making loading bars, for example.
use std::io::Read;

fn main() -> Result<(), minreq::Error> {
let mut buffer = Vec::new();
for byte in minreq::get("http://example.com").send_lazy()? {
for byte in minreq::get("http://example.com").send_lazy()?.bytes() {
// The connection could have a problem at any point during the
// download, so each byte needs to be unwrapped.
let (byte, len) = byte?;
let byte = byte?;

// The `byte` is the current u8 of data we're iterating
// through.
print!("{}", byte as char);

// The `len` is the expected amount of incoming bytes
// including the current one: this will be the rest of the
// body if the server provided a Content-Length header, or
// just the size of the remaining chunk in chunked transfers.
buffer.reserve(len);
buffer.push(byte);

// Flush the printed text so each char appears on your
// terminal right away.
flush();
Expand Down
6 changes: 5 additions & 1 deletion src/request.rs
Original file line number Diff line number Diff line change
Expand Up @@ -240,7 +240,11 @@ impl Request {
self
}

/// Sends this request to the host.
/// Sends this request to the host and collect the *whole* response
///
/// **WARNING:** This does what it says on the tin — so long as the
/// server keeps sending bytes, they will be appended, in-memory,
/// to the repsonse. Consider reading from a [`ResponseLazy`] instead.
///
/// # Errors
///
Expand Down
Loading

0 comments on commit 1a7bf12

Please sign in to comment.