Skip to content

Commit

Permalink
Merge pull request #411 from ipfs/publish-routing
Browse files Browse the repository at this point in the history
  • Loading branch information
hacdias authored May 12, 2023
2 parents b3ab660 + 9470fd4 commit b07b1bc
Show file tree
Hide file tree
Showing 6 changed files with 204 additions and 178 deletions.
176 changes: 2 additions & 174 deletions routing/ROUTING_V1_HTTP.md
Original file line number Diff line number Diff line change
@@ -1,175 +1,3 @@
# Routing v1 HTTP API
# Routing V1 HTTP API

![reliable](https://img.shields.io/badge/status-reliable-green.svg?style=flat-square) Delegated Content Routing HTTP API

**Author(s)**:

- Gus Eggert

**Maintainer(s)**:

* * *

**Abstract**

"Delegated content routing" is a mechanism for IPFS implementations to use for offloading content routing to another process/server. This spec describes an HTTP API for delegated content routing.

## API Specification

The Delegated Content Routing Routing HTTP API uses the `application/json` content type by default.

As such, human-readable encodings of types are preferred. This spec may be updated in the future with a compact `application/cbor` encoding, in which case compact encodings of the various types would be used.

## Common Data Types

- CIDs are always string-encoded using a [multibase](https://github.com/multiformats/multibase)-encoded [CIDv1](https://github.com/multiformats/cid#cidv1).
- Multiaddrs are string-encoded according to the [human-readable multiaddr specification](https://github.com/multiformats/multiaddr#specification)
- Peer IDs are string-encoded according [PeerID string representation specification](https://github.com/libp2p/specs/blob/master/peer-ids/peer-ids.md#string-representation)
- Multibase bytes are string-encoded according to [the Multibase spec](https://github.com/multiformats/multibase), and *should* use base64.
- Timestamps are Unix millisecond epoch timestamps

Until required for business logic, servers should treat these types as opaque strings, and should preserve unknown JSON fields.

### Versioning

This API uses a standard version prefix in the path, such as `/v1/...`. If a backwards-incompatible change must be made, then the version number should be increased.

### Provider Records

A provider record contains information about a content provider, including the transfer protocol and any protocol-specific information useful for fetching the content from the provider.

The information required to write a record to a router (*"write" provider records*) may be different than the information contained when reading provider records (*"read" provider records*).

For example, indexers may require a signature in `bitswap` write records for authentication of the peer contained in the record, but the read records may not include this authentication information.

Both read and write provider records have a minimal required schema as follows:

```json
{
"Protocol": "<transfer_protocol_name>",
"Schema": "<transfer_protocol_schema>",
...
}
```

Where:

- `Protocol` is the multicodec name of the transfer protocol or an opaque string (for experimenting with novel protocols without a multicodec)
- `Schema` denotes the schema to use for encoding/decoding the record
- This is separate from the `Protocol` to allow this HTTP API to evolve independently of the transfer protocol
- Implementations should switch on this when parsing records, not on `Protocol`
- `...` denotes opaque JSON, which may contain information specific to the transfer protocol

Specifications for some transfer protocols are provided in the "Transfer Protocols" section.

## API

### `GET /routing/v1/providers/{CID}`

#### Response codes

- `200` (OK): the response body contains 0 or more records
- `404` (Not Found): must be returned if no matching records are found
- `422` (Unprocessable Entity): request does not conform to schema or semantic constraints

#### Response Body

```json
{
"Providers": [
{
"Protocol": "<protocol_name>",
"Schema": "<schema>",
...
}
]
}
```

Response limit: 100 providers

Each object in the `Providers` list is a *read provider record*.

## Pagination

This API does not support pagination, but optional pagination can be added in a backwards-compatible spec update.

## Streaming

This API does not currently support streaming, however it can be added in the future through a backwards-compatible update by using a content type other than `application/json`.

## Error Codes

- `501` (Not Implemented): must be returned if a method/path is not supported
- `429` (Too Many Requests): may be returned along with optional [Retry-After](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Retry-After) header to indicate to the caller that it is issuing requests too quickly
- `400` (Bad Request): must be returned if an unknown path is requested

## CORS and Web Browsers

Browser interoperability requires implementations to support
[CORS](https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS).

JavaScript client running on a third-party Origin must be able to send HTTP
request to the endpoints defined in this specification, and read the received
values. This means HTTP server implementing this API must (1) support
[CORS preflight requests](https://developer.mozilla.org/en-US/docs/Glossary/Preflight_request)
sent as HTTP OPTIONS, and (2) always respond with headers that remove CORS
limits, allowing every site to query the API for results:

```plaintext
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: GET, OPTIONS
```

## Known Transfer Protocols

This section contains a non-exhaustive list of known transfer protocols (by name) that may be supported by clients and servers.

### Bitswap

Multicodec name: `transport-bitswap`
Schema: `bitswap`
Specification: [ipfs/specs/BITSWAP.md](https://github.com/ipfs/specs/blob/main/BITSWAP.md)

#### Bitswap Read Provider Records

```json
{
"Protocol": "transport-bitswap",
"Schema": "bitswap",
"ID": "12D3K...",
"Addrs": ["/ip4/..."]
}
```

- `ID`: the [Peer ID](https://github.com/libp2p/specs/blob/master/peer-ids/peer-ids.md) to contact
- `Addrs`: a list of known multiaddrs for the peer
- This list may be incomplete or incorrect and should only be treated as *hints* to improve performance by avoiding extra peer lookups

The server should respect a passed `transport` query parameter by filtering against the `Addrs` list.

### Filecoin Graphsync

Multicodec name: `transport-graphsync-filecoinv1`
Schema: `graphsync-filecoinv1`
Specification: [ipfs/go-graphsync/blob/main/docs/architecture.md](https://github.com/ipfs/go-graphsync/blob/main/docs/architecture.md)

#### Filecoin Graphsync Read Provider Records

```json
{
"Protocol": "transport-graphsync-filecoinv1",
"Schema": "graphsync-filecoinv1",
"ID": "12D3K...",
"Addrs": ["/ip4/..."],
"PieceCID": "<cid>",
"VerifiedDeal": true,
"FastRetrieval": true
}
```

- `ID`: the peer ID of the provider
- `Addrs`: a list of known multiaddrs for the provider
- `PieceCID`: the CID of the [piece](https://spec.filecoin.io/systems/filecoin_files/piece/#section-systems.filecoin_files.piece) within which the data is stored
- `VerifiedDeal`: whether the deal corresponding to the data is verified
- `FastRetrieval`: whether the provider claims there is an unsealed copy of the data available for fast retrieval
Moved to https://specs.ipfs.tech/routing/http-routing-v1/
2 changes: 1 addition & 1 deletion src/architecture/principles.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ over arbitrary transports using a CID*. As Juan Benet once put it,
Conversely, any system that exposes interactions with resources based on CIDs is
an IPFS implementation. There are
[many contexts in which CIDs can be used for addressing](https://docs.ipfs.tech/how-to/address-ipfs-on-web/)
and [content routing delegation](https://github.com/ipfs/specs/blob/main/routing/ROUTING_V1_HTTP.md)
and content routing delegation (:cite[http-routing-v1])
can support a wealth of interaction options by resolving CIDs.

## Robustness
Expand Down
8 changes: 8 additions & 0 deletions src/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,14 @@ <h3><a href="/ipns/">InterPlanetary Naming System</a></h3>
</p>
{% include 'list.html', posts: collections.ipns %}
</section>
<section>
<h3><a href="/routing/">Routing</a></h3>
<p>
Content routing is the way to determine where to find a given CID on the network;
specifically, which network peers provide specific CIDs.
</p>
{% include 'list.html', posts: collections.routing %}
</section>
<section>
<h3><a href="/ipips/">InterPlanetary Improvement Proposals</a></h3>
<p>
Expand Down
6 changes: 3 additions & 3 deletions src/ipips/ipip-0337.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ and supporting large content providers is a key strategy for driving down IPFS c
These providers must handle high volumes of traffic and support many users, so leveraging industry-standard tools and services
such as HTTP load balancers, CDNs, reverse proxies, etc. is a requirement.
To maximize compatibility with standard tools, IPFS needs an HTTP API specification that uses standard HTTP idioms and payload encoding.
The [Reframe spec](https://github.com/ipfs/specs/blob/main/reframe/REFRAME_PROTOCOL.md) for delegated content routing is an experimental attempt at this,
The Reframe spec for delegated content routing is an experimental attempt at this,
but it has resulted in a very unidiomatic HTTP API which is difficult to implement and is incompatible with many existing tools.
The cost of a proper redesign, implementation, and maintenance of Reframe and its implementation is too high relative to the urgency of having a delegated content routing HTTP API.

Expand All @@ -36,13 +36,13 @@ We expect this API to be extended beyond "content routing" in the future, so add

## Detailed design

See the [Delegated Content Routing HTTP API spec](../routing/DELEGATED_CONTENT_ROUTING_HTTP.md) included with this IPIP.
See the Delegated Content Routing HTTP API spec (:cite[http-routing-v1]) included with this IPIP.

## Design rationale

To understand the design rationale, it is important to consider the concrete Reframe limitations that we know about:

- Reframe [method types](../reframe/REFRAME_KNOWN_METHODS.md) using the HTTP transport are encoded inside IPLD-encoded messages
- Reframe method types using the HTTP transport are encoded inside IPLD-encoded messages
- This prevents URL-based pattern matching on methods, which makes it hard and expensive to do basic HTTP scaling and optimizations:
- Configuring different caching strategies for different methods
- Configuring reverse proxies on a per-method basis
Expand Down
Loading

0 comments on commit b07b1bc

Please sign in to comment.