Skip to content

Latest commit

 

History

History
367 lines (299 loc) · 42.4 KB

CHANGELOG.md

File metadata and controls

367 lines (299 loc) · 42.4 KB
Raw

v0.2.0 (2022-12-21)

First steps for iroh as a library

Many folks have asked for iroh as a library embeddable in other rust projects, and with this release we land the the first version of iroh-embed. Check the example for an initial guide.

We still have a lot of work to do on iroh-api to expose a clean API to consume as a library, but if you're the kind of project that's intersted in embedded IPFS, and willing to roll up your sleeves on a moving API, Iroh is ready for you to use today.

Gateway Subdomain & ETH domain support

Our gateway spec compatibility continues to grow. Both of these features were outside contributions by @ppodolsky, for which we're super grateful. Be sure to check out their work on summa!

QUIC Support

The majority of traffic on public IPFS networks runs over QUIC, which we now support within iroh thanks to a massive push by the rust-libp2p team. Huge thanks to the libp2p team!

Bug Fixes

  • actually resolve and check paths (#623) (8b6844f)
  • ci: Call cargo with beta toolchain from env var (#622) (b62030d)
  • ci: Set the protoc-arch matrix for weekly job (#619) (22d68df), closes #618
  • ci: Use beta toolchain for the clippy run (#620) (03624b5)
  • iroh-one: Wire up mem addresses to each other (#555) (ee9677d)
  • iroh: do not use self version dep (6f6fee0)
  • iroh: exclude tests and fixtures from publishing (6ebee5f)
  • only use name if name is set (#605) (4dda2d0)
  • unixfs seeking (#606) (51e3ddf)
  • update quic-rpc to get rid of debug output (bf4128b)
  • use http when creating hyper URL (#591) (e9e932d)
  • version handling in tests & git_version (59c5dc7)

Features

v0.1.3 (2022-11-28)

Bug fix release, for issues discovered while publishing to crates.io.

Bug Fixes

  • iroh: do not use self version dep (6f6fee0)
  • iroh: exclude tests and fixtures from publishing (6ebee5f)
  • version handling in tests & git_version (59c5dc7)

v0.1.2 (2022-11-28)

This relase brings about a couple of highlights, the first of which is the first relase of all iroh-* crates to crates.io. The second one being that we now support the experimental quic transport in libp2p.

Bug Fixes

  • iroh-one: Wire up mem addresses to each other (#555) (ee9677d)

Features

v0.1.1 (2022-11-21)

Bitswap client mode

Iroh can now run in "client mode", which can fetch content from bitswap, but not provide content. This is useful if you're running on a lower-powered device or have limited bandwidth. To use client mode change your p2p.config.toml:

bitswap_server = false

Use indexer nodes to find content providers

Iroh can use network indexers like cid.contact to find providers of a given CID. To use the cid.contact indexer adjust your gateway.config.toml to add an indexer_endpoint URL:

indexer_endpoint = https://cid.contact/cid/

Docker support

Iroh now ships with docker images! Spin iroh up locally with:

$ cd iroh/docker
$ docker-compose up

then run iroh status from another terminal if you already have iroh installed. Now you're backing iroh with docker! docker-compose down to stop. The docker compose file also includes helpful guidance on which ports to expose from each iroh service.

We're shipping multi-arch distroless builds for amd64 & arm64 architectures.

Fixed a critical bug adding files

We fixed a known bug in iroh v0.1.0 that would break iroh add if files referenced the same CID on the way in. It's now fixed, and now we can happily do roundtrip add/get of the linux kernel, which is always fun 😊.

Benchmarks

Benchmarks for this release coming soon. We'll update this readme once they're up.

Production Readiness

Is this production ready? Maybe. Numerous projects are now using iroh in the wild. You'll need to judge for yourself if iroh meets your needs. Please reach out on our discussion forums if you have questions. We're going to keep the status at "Maybe" until we release a v1.0. v1.0 is currently slated for Q4 2023.

Bug Fixes

  • iroh-p2p: implement full local lookup (#537) (0c388b9)
  • breadcrumb links in gateway directory template (#513) (f429baa)
  • cli: Respect the --cfg flag (#506) (e7bcac6)
  • convert v0 -> v1 before base32 conversion (#527) (ab3969c)
  • Correct dependencies and build checks (#484) (7e9cdc3)
  • docker-compose RPC ports should only listen on loopback (#524) (8471e49), closes #520
  • iroh start service set construction (#522) (070be80)
  • iroh-store: put_many bug (#507) (88f1a49)
  • Process -> Service in status command, add after help text (#493) (804255c)
  • store: Make store operations atomic (#480) (ac8ec35)
  • Switch to tempfile for temporary test directories (#509) (89959cb)
  • verify content fetched from racing http gateways (e281a4a)

Features

v0.1.0 - 2022-10-28

We’re on the board 🎉! This first release of iroh brings a new implementation of IPFS to the world.

Key things to highlight about the first release of iroh:

  • Exchange Data with Kubo. Iroh can interoperate with kubo nodes on the IPFS network, pushing & fetching data.
  • Service Oriented architecture. Don't want p2p? Turn it off! Iroh will still work without it.
  • Single CLI to control all services.
  • Built for efficiency. Iroh's memory footprint & CPU usage are either on par or better than the best interoperable IPFS implementations out there.
  • Runs well on a laptop. We have a custom installer script that will configure iroh for your laptop. See our install docs

v0.1.0 ships as 4 binaries for Linux & macOS with zero external dependencies:

  • iroh - command line client
  • iroh-store - data storage daemon
  • iroh-p2p - peer 2 peer networking
  • iroh-gateway - IPFS-to-HTTP bridge

You'll need to download all of them & put them on your $PATH to work with iroh. Our install docs have more info.

Please do give iroh a try, we'd love to hear your feedback. Thanks!

Benchmarks

We're runnning benchmarks on a relatively stock AWS box for ease-of-replication. We don't have benchmarks for IPFS network retrieval this round.

Adding Single Files

Add a single file with iroh add --no-wrap --offline:

file size real (s) user(s) sys(s) cpu % cpu/total %
100K 0.0102 0.0162 0.0317 469.6078431 7.337622549
1M 0.0167 0.004 0.0519 334.7305389 5.230164671
10M 0.01345 0.0101 0.0418 385.87 6.03
100M 0.01345 0.0101 0.0418 385.87 6.03
1G 0.01345 0.0101 0.0418 385.87 6.03

Fetching Cached Gateway Content

Repeatedly request the same CID of different file sizes to measure cached content throughput via HTTP request

File size cpu/total (%) rps throughput/s avg latency
100K 90.48072917 40,064.27 3.85GB 5.08ms
1M 87.66614583 6,926.95 6.77GB 29.08ms
10M 87.34010417 711.55 6.98GB 278.86ms
100M 88.00885417 66.52 6.82GB 2.85s
1G 87.84947917 6.65 6.85GB 0.00s

Production Readiness

Is this production ready? No. We need your help to kick tires & find bugs!.

The v0.1.0 moniker should testify to just how new this software is. Please don't deploy it anywhere mission critical until we've had at least a month or two to address bugs & write tests.

Things you should know about this release:

There are two problems we couldn't properly address

  • There is no delete command. Data added to iroh will stay there until a future release where we add content removal. See issue #432 for details. The workaround for now is to blow away the storage database.
  • We have one particularly nasty known bug that can cause iroh to not persist data for highly nested directories upon add we plan to address ASAP in a patch release. See issue #423.

Bug Fixes

  • check and watch should display the same service names (#101) (ef99fe5)
  • add --no-metrics support to p2p (#65) (a782fcb)
  • add macos builds to ci (#33) (a0e79fb)
  • add missing metrics and update to fixed libp2p (0f4f467)
  • allow Raw as UnixfsType (#327) (275d88b)
  • allow concurrent rpc calls & de-duplicate provider requests (#60) (a74f8dd)
  • auto-raise FD limits where possible (#215) (9e429bb)
  • avoid 0 divisions (#76) (8f491d5)
  • bitswap sessions shutdown (6de85c9)
  • bitswap: fetch provider loop (93517a3)
  • ci release aarch64 linux (#184) (4cc66aa)
  • CI, add macOS & windows builds, cargo audit (5dbd907)
  • correctly check headers parse result (#227) (82a1b82)
  • correctly map memory channels (0742fb2), closes #165
  • default off metrics & tracing (#174) (bbc2ec1)
  • don't error on empty providers in the wrong place (414a8f2)
  • drop openssl requirement (#72) (b72ce04)
  • error when someone tries to start an unknown service (#431) (9a57fb7)
  • extend metrics (#32) (34c53da)
  • fix builds (#173) (932161c)
  • Fix test that for some reason assumes sort order in a vec (d57b12a)
  • gateway readme (#31) (e94a7e0)
  • gateway upgrade (#36) (0bb4fe9)
  • gateway: do not panic on missing location info (c518ce7)
  • gateway: populate directory listings (c7217f1)
  • gha disk space hacks (#387) (a9d9c25)
  • hamt directory listings (5187773), closes #192
  • histogram metric assignment (#119) (b75bbbd)
  • Improve error handling (#379) (7a945ed)
  • Improve error management and reporting when using unix domain sockets (#228) (8d13e4f)
  • iroh-car: limit to 4MiB not 4GiB (34eedb8)
  • iroh-gateway: add FileType::Raw option (#332) (ff4ef02)
  • iroh-gateway: increase timeouts (c3d56c6)
  • iroh-one: use default store path when no --store-path flag present (#317) (643fd43), closes #255 #309
  • limit max memory allocations per node (9817b5a)
  • loger only on p2p (b24c5fe)
  • macos releases (#34) (2fc0109)
  • make trace_id visible only if in use (#359) (c29bf85)
  • metrics rework & update (#69) (6c1d003)
  • metrics toggle on store (2b12158)
  • move bitswap msg counters out of encode/decode (#374) (85d80b9)
  • p2p subcommand output cleanup (#427) (0ca029c)
  • p2p: cleanup sessions when they are canceled (#419) (efa9e82)
  • p2p: do not panic on shutdown of bs req (b05b94a)
  • p2p: enable rsa key keys again (afeafb4)
  • p2p: ensure default grpc connections work (eec218d)
  • p2p: Increase the size of the kad mem store (#320) (00f5660)
  • p2p: record Kad, Identify and Ping metrics (af8eaa9)
  • p2p: remove debug logs (#420) (23e1b3a)
  • parameters and connection handling (96514d3)
  • params (#128) (1f47d7c)
  • pass links to store when adding data (#299) (9476d9c)
  • prom metrics no longer ping on --no-metrics (#44) (7f9c81a)
  • properly saves filenames for hamt dirs (#422) (1fe8e69)
  • properly setup the default config for iroh-one (#256) (a1e4940)
  • protoc from release bins (#42) (28cc3ab)
  • require passing providers to fetch_bitswap (771ee00), closes #73
  • resolver: handle trailing slashes in the path with more sense (#414) (9ef95e8)
  • resolver: ipld, double-fetching, and trailing slash fixes (#408) (f861102)
  • resolver: stop sessions when requests are done (9c46cb0)
  • rpc creation no longer blocks the gateway (#111) (8f1d93c)
  • rpc-client: correct address format for clients (8cde5cf)
  • rpc-client: use multihash to query providers (2b11a14)
  • stop preventing any directory or file foo from existing in the project (#334) (3ef7ccc)
  • switch back to rust-libp2p master (1336e9e)
  • tracing on request level (#85) (562e6d8)
  • undo label change (#219) (3433340)
  • Update ipld in iroh-car to limit prost duplicates (#307) (ee43bd7)
  • use rustls (#64) (0c37cef)
  • use the uds-gateway feature only to control the http uds endpoint (#252) (9bcbb59)
  • windows symlink support (#386) (563aac0)

Features

  • iroh-ctl add & iroh-ctl get (#164) (979a36f)
  • iroh-ctl status -w watches the health of each iroh process (#84) (22b2bb5)
  • abstract over internal rpc mechanism & add support for UDS & Mem RPC (6e859c2)
  • add car importer (aabfb9a)
  • add iroh-car (#27) (26a1b86), closes #9
  • add placeholder iroh-bitswap and iroh-p2p crates (9c4ab10)
  • add [email protected] (b08ac2b)
  • add trace id to responses (#28) (9d9c89e)
  • Allow to pass custom resolvers (#220) (9d319c5)
  • bad bits implementation (#172) (bda0173)
  • bitswap: add dial timeouts (06e0104)
  • bitswap: add query errors and start cleanups (72ae431)
  • bitswap: allow tracking queries based on ids (21f75d5)
  • bitswap: custom protocol handler for bitswap (ff2ebc3)
  • bitswap: refactor & improve general strategy (0747310)
  • CLI get command improvements (#331) (b18b6c8), closes #269
  • CLI tests for iroh add, and API changes (#343) (fdf2170)
  • connect the dots (4b15df5)
  • connection pooling (#404) (b499751)
  • ctl ci (#175) (b7ebf02)
  • deps: update libp2p to 0.47 (9dadadc)
  • distributed tracing (#46) (3e555bc)
  • don't run 'dht_nice_tick()' when kadmelia is disabled (#242) (050ad48)
  • Enable the CompressionLayer middleware (#236) (9151fc1)
  • fetch providers via bitswap (b16be5a)
  • gateway upgrade (#87) (4732a76)
  • gateway: initial directory listing support (#328) (3316a53)
  • get config from files, environment vars, & command line flags (#112) (652b7ff)
  • implement basic iroh-store (a1b9586)
  • implement iroh-ctl find-provs (#182) (ec06238)
  • import flatfs-store and rocks-store crates (4ac82a8)
  • importer: add hash verification (96eda12)
  • improve add experience (#401) (4f86388)
  • improve CLI texts and flags (#342) (9e0a03e)
  • improve provider fetching (38ce246)
  • iroh-api rename, iroh get basics, rpc client construction (d24edc1)
  • iroh-api: implement lookup & connect (#372) (a93dfb4)
  • iroh-ctl: implement available commands (#123) (c6bdca3)
  • iroh-one (#212) (11fe705)
  • iroh-resolver: create a balanced tree dag (3285d1f)
  • iroh-resolver: implement unixfs dir listing (263b3c2)
  • iroh-store: add minimal main implementation (b1c49a1)
  • Map id to cid instead of multihash (#336) (092bdf7), closes #335
  • offline adding (#415) (7725508)
  • p2p api service calls with a p2p-specific error, better feedback (#402) (b565231)
  • p2p: add gossipsub to p2p node & gossipsub rpc methods (#132) (f18d726)
  • p2p: improve default configurations for dht and bs (791181a)
  • p2p: integrate autonat protocol (5c5c1b0)
  • p2p: integrate relay and dcutr (6555090)
  • p2p: make bitswap toglable (648bcbf)
  • p2p: make gossipsub configurable (071fd1b)
  • p2p: persist identity (8f57278)
  • p2p: refresh kad buckets & rebootstrap regularly (339ed13)
  • p2p: respond to bitswap requests (965e21c)
  • progress bar for iroh add command (#368) (bff3560)
  • provide a mechanism to check if an iroh program is already running. (#293) (bddbae9)
  • provide only root, and only at the end of add (#406) (96c6148)
  • race a http gateway with p2p in iroh-one's content loader (#338) (40cb232)
  • range request support (#330) (074f0c6)
  • recursive directories & directory link limits (#213) (be4e931)
  • relases for p2p, store & some refactoring (#45) (d101e5a)
  • rename the iroh-ctl crate to the iroh crate (#321) (8b02977)
  • resolver: add support for raw leaves in unixfs (1d6d57d)
  • resolver: expose metadata about the resolution result (a229875)
  • resolver: expose way to get directory content (9b59326)
  • resolver: handle symlinks properly (5cc84ea)
  • resolver: implement dsnlink resolution (5fa893d)
  • resolver: stream content (3d419b5)
  • resolver: support raw codec (25d5b28)
  • resolver: unixfs: resolve simple multi chunk files (fa94b6d)
  • rpc client that can handle out-going commands and incoming requests (#26) (f2fd5cb)
  • rpc: add rpc client and server for store process (#41) (854b605)
  • streaming providers (a810e07)
  • switch to [email protected] (f1dd5f7)
  • Unixfs Improvements (42dee06)
  • unixfs: add filesize and blocksizes for UnixfsNode::File nodes (#254) (6ba6b57)
  • unixfs: add symlink support (#337) (3157c15)
  • update libp2p to latest (39b25e8)
  • update rust-libp2p (c595c4d)
  • update to [email protected] (#391) (8a906ee)
  • upload gateway builds to s3 (linux/amd64) (#29) (1918a83)
  • use the store to cache received data (9dac53a)

Performance Improvements

  • add parallelism to hashing (#411) (12db7ed)
  • impl put_many and reuse column families (#412) (4cab5c7)
  • resolver: avoid some allocations (9f2d718)
  • use content_bytes instead of content_reader (9dd0c22)
  • Use the libipld built in way to scrape references (93546b1)

Reverts