Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat!: add ContentType, expanded Accept parsing #6

Merged
merged 2 commits into from
Sep 8, 2023
Merged

feat!: add ContentType, expanded Accept parsing #6

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
7777929
feat!: add ContentType, expanded Accept parsing
rvagg Sep 7, 2023
d5b02bc
feat: switch ContentType from options pattern to fluent
rvagg Sep 7, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
86 changes: 79 additions & 7 deletions http/constants.go
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,34 +1,106 @@
package trustlesshttp

import "fmt"
import (
"strconv"
"strings"
)

type ContentTypeOrder string

const (
MimeTypeCar = "application/vnd.ipld.car" // The only accepted MIME type
MimeTypeCarVersion = "1" // We only accept version 1 of the MIME type
FormatParameterCar = "car" // The only valid format parameter value
FilenameExtCar = ".car" // The only valid filename extension
DefaultIncludeDupes = true // The default value for an unspecified "dups" parameter. See https://github.com/ipfs/specs/pull/412.
ResponseCacheControlHeader = "public, max-age=29030400, immutable" // Magic cache control values
DefaultIncludeDupes = true // The default value for an unspecified "dups" parameter.
DefaultOrder = ContentTypeOrderDfs // The default value for an unspecified "order" parameter.

ContentTypeOrderDfs ContentTypeOrder = "dfs"
ContentTypeOrderUnk ContentTypeOrder = "unk"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this a useful value? we don't seem to do anything with an unknown ordering, and if the ordering is anything other than an understood 'dfs' we presumably would want to stream it through unchanged rather than try to change it to "unk"? since we'd be comparing against known values, like 'dfs', what do we get by defining and parsing an 'unk' value here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's defined by the spec https://specs.ipfs.tech/http-gateways/trustless-gateway/#car-order-content-type-parameter

I'm not particularly a fan of this existing but but the idea of it being in here is that a consumer of this library can decide what to do with it. Currently Lassie will ignore it and assume dfs and error on you for feeding bad data. Frisbii will always give you dfs regardless of what you ask for (on the assumption that dfs is a subset of unk so this is acceptable).

This isn't new btw, it was in Lassie like this; no behaviour should be changing with this, it just gets surfaced as a parameter you can inspect now if you care.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it does seem like premature complexity / over engineering

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ipfs/specs#412

It was nearly dfs, rnd, weak!

Our approach atm is to 🙈; although it's at least exposed here if you actually wanted to do something with it. I'm going to assume that nobody will that wants to interoperate with our current tools (Lassie primarily).

)

var (
ResponseChunkDelimeter = []byte("0\r\n") // An http/1.1 chunk delimeter, used for specifying an early end to the response
baseContentType = fmt.Sprintf("%s; version=%s; order=dfs", MimeTypeCar, MimeTypeCarVersion)
)

// ContentType represents a Content-Type descriptor for use with the response
// Content-Type header or the request Accept header specifically for
// Trustless Gateway requests and responses.
type ContentType struct {
MimeType string
Order ContentTypeOrder
Duplicates bool
Quality float32
}

func (ct ContentType) String() string {
sb := strings.Builder{}
sb.WriteString(ct.MimeType)
sb.WriteString(";version=")
sb.WriteString(MimeTypeCarVersion)
sb.WriteString(";order=")
sb.WriteString(string(ct.Order))
if ct.Duplicates {
sb.WriteString(";dups=y")
} else {
sb.WriteString(";dups=n")
}
if ct.Quality < 1 && ct.Quality >= 0.00 {
sb.WriteString(";q=")
// write quality with max 3 decimal places
sb.WriteString(strconv.FormatFloat(float64(ct.Quality), 'g', 3, 32))
}
return sb.String()
}

// WithOrder returns a new ContentType with the specified order.
func (ct ContentType) WithOrder(order ContentTypeOrder) ContentType {
ct.Order = order
return ct
}

// WithDuplicates returns a new ContentType with the specified duplicates.
func (ct ContentType) WithDuplicates(duplicates bool) ContentType {
ct.Duplicates = duplicates
return ct
}

// WithMime returns a new ContentType with the specified mime type.
func (ct ContentType) WithMimeType(mime string) ContentType {
ct.MimeType = mime
return ct
}

// WithQuality returns a new ContentType with the specified quality.
func (ct ContentType) WithQuality(quality float32) ContentType {
ct.Quality = quality
return ct
}

func DefaultContentType() ContentType {
return ContentType{
MimeType: MimeTypeCar,
Order: DefaultOrder,
Duplicates: DefaultIncludeDupes,
Quality: 1,
}
}

// ResponseContentTypeHeader returns the value for the Content-Type header for a
// Trustless Gateway response which will vary depending on whether duplicates
// are included or not. Otherwise, the header is the same for all responses.
//
// Deprecated: Use DefaultContentType().WithDuplicates(duplicates).String() instead.
func ResponseContentTypeHeader(duplicates bool) string {
if duplicates {
return baseContentType + "; dups=y"
}
return baseContentType + "; dups=n"
return DefaultContentType().WithDuplicates(duplicates).String()
}

// RequestAcceptHeader returns the value for the Accept header for a Trustless
// Gateway request which will vary depending on whether duplicates are included
// or not. Otherwise, the header is the same for all requests.
//
// Deprecated: Use DefaultContentType().WithDuplicates(duplicates).String() instead.
func RequestAcceptHeader(duplicates bool) string {
return ResponseContentTypeHeader(duplicates)
}
17 changes: 13 additions & 4 deletions http/constants_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,17 @@ import (
)

func TestContentType(t *testing.T) {
require.Equal(t, "application/vnd.ipld.car; version=1; order=dfs; dups=y", trustlesshttp.ResponseContentTypeHeader(true))
require.Equal(t, "application/vnd.ipld.car; version=1; order=dfs; dups=y", trustlesshttp.RequestAcceptHeader(true))
require.Equal(t, "application/vnd.ipld.car; version=1; order=dfs; dups=n", trustlesshttp.ResponseContentTypeHeader(false))
require.Equal(t, "application/vnd.ipld.car; version=1; order=dfs; dups=n", trustlesshttp.RequestAcceptHeader(false))
req := require.New(t)

req.Equal("application/vnd.ipld.car;version=1;order=dfs;dups=y", trustlesshttp.ResponseContentTypeHeader(true))
req.Equal("application/vnd.ipld.car;version=1;order=dfs;dups=y", trustlesshttp.RequestAcceptHeader(true))
req.Equal("application/vnd.ipld.car;version=1;order=dfs;dups=n", trustlesshttp.ResponseContentTypeHeader(false))
req.Equal("application/vnd.ipld.car;version=1;order=dfs;dups=n", trustlesshttp.RequestAcceptHeader(false))

req.Equal("application/vnd.ipld.car;version=1;order=dfs;dups=y", trustlesshttp.DefaultContentType().String())
req.Equal("application/vnd.ipld.car;version=1;order=dfs;dups=y;q=0.8", trustlesshttp.DefaultContentType().WithQuality(0.8).String())
req.Equal("application/vnd.ipld.car;version=1;order=dfs;dups=y;q=0.333", trustlesshttp.DefaultContentType().WithQuality(1.0/3.0).String())
req.Equal("application/vnd.ipld.car;version=1;order=dfs;dups=y", trustlesshttp.DefaultContentType().WithQuality(-1.0).String())
req.Equal("application/vnd.ipld.car;version=1;order=dfs;dups=n", trustlesshttp.DefaultContentType().WithDuplicates(false).String())
req.Equal("application/vnd.ipld.car;version=1;order=unk;dups=n", trustlesshttp.DefaultContentType().WithDuplicates(false).WithOrder(trustlesshttp.ContentTypeOrderUnk).String())
}
160 changes: 89 additions & 71 deletions http/parse.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@
"fmt"
"net/http"
"path/filepath"
"sort"
"strconv"
"strings"

"github.com/ipfs/go-cid"
Expand Down Expand Up @@ -73,31 +75,32 @@
//
// IPFS Trustless Gateway only allows the "car" format query parameter
// https://specs.ipfs.tech/http-gateways/path-gateway/#format-request-query-parameter
func CheckFormat(req *http.Request) (bool, error) {
includeDupes := DefaultIncludeDupes
func CheckFormat(req *http.Request) (ContentType, error) {
// check if format is "car"
format := req.URL.Query().Get("format")
var validFormat bool
if format != "" {
if format != FormatParameterCar {
return ContentType{}, fmt.Errorf("invalid format parameter; unsupported: %q", format)
}
validFormat = true
}

accept := req.Header.Get("Accept")
if accept != "" {
// check if Accept header includes application/vnd.ipld.car
var validAccept bool
validAccept, includeDupes = ParseAccept(accept)
if !validAccept {
return false, fmt.Errorf("invalid Accept header; unsupported: %q", accept)
accepts := ParseAccept(accept)
if len(accepts) == 0 {
return ContentType{}, fmt.Errorf("invalid Accept header; unsupported: %q", accept)
}
}
// check if format is "car"
format := req.URL.Query().Get("format")
if format != "" && format != FormatParameterCar {
return false, fmt.Errorf("invalid format parameter; unsupported: %q", format)
return accepts[0], nil // pick the top one we can support
}

// if neither are provided return
// one of them has to be given with a CAR type since we only return CAR data
if accept == "" && format == "" {
return false, fmt.Errorf("neither a valid Accept header nor format parameter were provided")
if validFormat {
return DefaultContentType(), nil // default is acceptable in this case (no accept but format=car)
}

return includeDupes, nil
return ContentType{}, fmt.Errorf("neither a valid Accept header nor format parameter were provided")
}

// ParseAccept validates a request Accept header and returns whether or not
Expand All @@ -106,72 +109,87 @@
// This will operate the same as ParseContentType except that it is less strict
// with the format specifier, allowing for "application/*" and "*/*" as well as
// the standard "application/vnd.ipld.car".
func ParseAccept(acceptHeader string) (validAccept bool, includeDupes bool) {
return parseContentType(acceptHeader, false)
func ParseAccept(acceptHeader string) []ContentType {
acceptTypes := strings.Split(acceptHeader, ",")
accepts := make([]ContentType, 0, len(acceptTypes))
for _, acceptType := range acceptTypes {
accept, valid := parseContentType(acceptType, false)
if valid {
accepts = append(accepts, accept)
}
}
// sort accepts by ContentType#Quality
sort.SliceStable(accepts, func(i, j int) bool {
return accepts[i].Quality > accepts[j].Quality
})
return accepts
}

// ParseContentType validates a response Content-Type header and returns whether
// or not duplicate blocks are expected in the response.
// ParseContentType validates a response Content-Type header and returns
// a ContentType descriptor form and a boolean to indicate whether or not
// the header value was valid or not.
//
// This will operate the same as ParseAccept except that it strictly only
// allows the "application/vnd.ipld.car" Content-Type.
func ParseContentType(contentTypeHeader string) (validContentType bool, includeDupes bool) {
// This will operate similar to ParseAccept except that it strictly only
// allows the "application/vnd.ipld.car" Content-Type (and it won't accept
// comma separated list of content types).
func ParseContentType(contentTypeHeader string) (ContentType, bool) {
return parseContentType(contentTypeHeader, true)
}

func parseContentType(header string, strictType bool) (validAccept bool, includeDupes bool) {
acceptTypes := strings.Split(header, ",")
validAccept = false
includeDupes = DefaultIncludeDupes
for _, acceptType := range acceptTypes {
typeParts := strings.Split(acceptType, ";")
if typeParts[0] == MimeTypeCar || (!strictType && (typeParts[0] == "*/*" || typeParts[0] == "application/*")) {
validAccept = true
if typeParts[0] == MimeTypeCar {
// parse additional car attributes outlined in IPIP-412: https://github.com/ipfs/specs/pull/412
for _, nextPart := range typeParts[1:] {
pair := strings.Split(nextPart, "=")
if len(pair) == 2 {
attr := strings.TrimSpace(pair[0])
value := strings.TrimSpace(pair[1])
switch attr {
case "dups":
switch value {
case "y":
includeDupes = true
case "n":
includeDupes = false
default:
// don't accept unexpected values
validAccept = false
}
case "version":
switch value {
case MimeTypeCarVersion:
default:
validAccept = false
}
case "order":
switch value {
case "dfs":
case "unk":
default:
// we only do dfs, which also satisfies unk, future extensions are not yet supported
validAccept = false
}
default:
// ignore others
}
func parseContentType(header string, strictType bool) (ContentType, bool) {
typeParts := strings.Split(header, ";")
mime := strings.TrimSpace(typeParts[0])
if mime == MimeTypeCar || (!strictType && (mime == "*/*" || mime == "application/*")) {
contentType := DefaultContentType().WithMimeType(mime)
// parse additional car attributes outlined in IPIP-412
// https://specs.ipfs.tech/http-gateways/trustless-gateway/
for _, nextPart := range typeParts[1:] {
pair := strings.Split(nextPart, "=")
if len(pair) == 2 {
attr := strings.TrimSpace(pair[0])
value := strings.TrimSpace(pair[1])
switch attr {
case "dups":
switch value {
case "y":
contentType.Duplicates = true
case "n":
contentType.Duplicates = false
default:
// don't accept unexpected values
return ContentType{}, false
}
case "version":
switch value {
case MimeTypeCarVersion:
default:
return ContentType{}, false
}
case "order":
switch value {
case "dfs":
contentType.Order = ContentTypeOrderDfs
case "unk":
contentType.Order = ContentTypeOrderUnk
default:
// we only do dfs, which also satisfies unk, future extensions are not yet supported
return ContentType{}, false
}
case "q":
// parse quality
quality, err := strconv.ParseFloat(value, 32)
if err != nil || quality < 0 || quality > 1 {
return ContentType{}, false
}
contentType.Quality = float32(quality)
default:

Check warning on line 185 in http/parse.go

View check run for this annotation

Codecov / codecov/patch

http/parse.go#L185 

Added line #L185 was not covered by tests
// ignore others
}
}
// only break if further validation didn't fail
if validAccept {
break
}
}
return contentType, true
}
return
return ContentType{}, false
}

var (
Expand Down
Loading