Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update resolution scheme #42 #47

Closed
wants to merge 6 commits into from
Closed

Conversation

gribneau
Copy link
Contributor

Based on reviews of #43, I've reduced the scope of the PR to index files rather than hardcoded /did.json and removed references to JSON-LD. Full file paths are still included, but can be removed and left in history for future consideration.

using the Koblitz Curve, and storing the <code>did.json</code> file under the
well-known URL to represent the entire domain, or under the specified path if
many DIDs will be resolved in this domain.
creating the DID document JSON, and storing the file as the index of the
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

re "storing the file as the index" -- we don't define what we mean by "index" anywhere.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am referring to the "directory index'. It has been commonplace for webservers to return index.html or some other configured. Nginx uses index and Apache uses DirectoryIndex to set a file or files to return when a user requests a directory path.

I can add a description.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. Adding a description would be helpful (so that people don't think we mean index.html)

Copy link
Contributor Author

@gribneau gribneau Nov 24, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

index.html has been a common default as long as I can remember, but it is possible to set other files as the index.

Does this do it?
b7ffd8e

If an optional path is specified rather the bare domain, the
<code>did.json</code> will be available under the specified path:
If an optional path is specified in addition to the bare domain, the
file will be available as the index of the specified path:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above -- it's unclear what "index of the specified path" means.

index.html Outdated Show resolved Hide resolved
index.html Outdated
did:web:example.com%3A3000
did:web:example.com%3A3000:bob%3Awork
did:web:example.com:christine:111.json
did:web:example.com:christine:222.didjson
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm, I feel like this one should be removed... since application/did+json is application/json....

The current method only supports JSON, these extensions should now appear in the identifier.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we want to add this capability, we should do so in a separate PR....

did:web:w3c-ccg.github.io
-> https://w3c-ccg.github.io/.well-known/did.json
-> https://w3c-ccg.github.io/.well-known/did/
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I interpret this as really => https://w3c-ccg.github.io/.well-known/did/index.json after some internal server redirect... can you confirm?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Index files - the files returned when a directory is requested - are configured on the server.

Index.html has been a default for ages, but any file name may be specified in most webserver configurations.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand how we're supposed to interpret this change.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The resolver translates the method specific identifier to a URL and makes an HTTPS request to retrieve the DID document. This changes the HTTPS URL to a directory rather than a full file path in order to meet the requirements in IANA Considerations to support multiple representations with appropriate filename extensions and content types (which are mapped to filename extensions in webservers and many clients).

->  - https://w3c-ccg.github.io/.well-known/did.json
->  + https://w3c-ccg.github.io/.well-known/did/

The first url should always return a content type of application/json based on the .json filename extension, while the second can return the contents of a file named appropriately for a different DID representation, together with the associated content type HTTP response header.

If we return a file with a .json filename extension, and a content type header value associated with some alternate DID representation, we potentially break a long standing best practice. One can argue that .json is consistent with application/did+json and application/did+ld+json, but as soon as we see application/did+cbor or some other alternative, we'll no longer have an argument that we're in compliance.

index.html Outdated
</p>
<pre class="example nohighlight" title="DID with path and filename">
did:web:w3c-ccg.github.io:user:alice:111.didjson
-> https://w3c-ccg.github.io/user/alice/111.didjson
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please do no add any new file extensions in this PR, we are trying to change 1 thing, and its did.json -> index.json... as far as I can tell... its not that we can't do these changes later, but lets not co-mingle them.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are not specifying index.json - that would be configured server side.

It might not be index.json, and there is no requirement that it is.

This flexibility will be necessary to support multiple representations without breaking the association between content types and filename extensions.

index.html Outdated
@@ -310,20 +309,26 @@ <h3>
<ol>
<li>
Replace ":" with "/" in the method specific identifier to obtain the fully
qualified domain name and optional path.
qualified domain name, optional path, and optional filename.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggest removing the filename part from the current PR, and opening a new one once we have fixed the index issue.

@@ -353,11 +367,17 @@ <h3>
Update
</h3>
<p>
To update the DID document, the <code>did.json</code> has to be updated.
To update the DID document, the associated file or files have to be updated.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

single file currently, no support for multiple representations, JSON only... for now at least.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I disagree.

Index files are configured server-side, and we should leave that capability in place.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where did this whole index file thing come from? What's the use case there?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Index files have always been with us. Wikipedia has a historic overview.

The use case here is to comply with the core spec on various representations without breaking other long-standing best practices on filename extension and content type header agreement.

In the event that multiple representations of the same DID are present in a
directory, updates of all representations MUST be made when any representation is
updated. Key material and service endpoints MUST be consistent across representations
of a DID. Other metadata SHOULD be consistent across representations.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's remove this possibility, JSON only for DID Web.... no JSON-LD and no CBOR.

They are interop destroying, and create massive security issues, we don't need them, at least right now.

Copy link
Collaborator

@dmitrizagidulin dmitrizagidulin Nov 29, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@OR13 - wait, in what way are JSON-LD did:webs interop destroying?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@OR13 - n/m, I think I'm +1 with just JSON for DID Web.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They are interop destroying, and create massive security issues, we don't need them, at least right now.

Citation? Strong opinions require strong evidence.

</p>
<pre class="nohighlight">
https://example.com/u/bob/did.json
https://example.com/u/bob/
Copy link
Collaborator

@OR13 OR13 Nov 29, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://example.com/u/bob/ -> https://example.com/u/bob/index.json

Adding 1 more arrow here might help better explain the potential upgrade path for other representations....

IMO, DID Web SHOULD NOT support representations other than JSON, because it makes interop harder and introduces dangerous security issues which are "not worth it".

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Likewise strongly opposed to anything other than JSON here

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The proposed change is to resolve the DID to a directory URL, leaving file selection up to the webserver.

The index file could very well be index.json, but it could also be arbitrary.json if so configured.

The resolvers would request https://example.com/u/bob/.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

because it makes interop harder and introduces dangerous security issues which are "not worth it".

Citation? Evidence?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@msporny

IMO, DID Web SHOULD NOT support representations other than JSON, because it makes interop harder and introduces dangerous security issues which are "not worth it".

DID Core defines 2 JSON Formats... application/did+json which only requires an id.... and application/did+ld+json which requires an id and @context.... if DID web has to support both, in different files, with potentially different key material, service endpoints, etc... and the JSON version wants to just let JSON be JSON and add fields not defined in the context, and the JSON-LD version wants to add terms only defined in a w3id.org hosted context... and a developer needs to understand both representations... thats all a lot harder than just having a single json file on a web server that has an id property that is a DID.

JSON-LD should be possible, but it should not be required for DID Web... and I don't think we get any value from having 2 JSON files that are potentially different... other than pain.

The simpler we can make DID web, the better.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@OR13 @msporny

The simpler we can make DID web, the better.

I absolutely agree that simplicity in the specification is important.

Is there a middle ground here in which the specification makes a variety of representations possible, but does not require any particular representation?

Resolving to directory paths and leaving file selection up to server configuration and client preference assertion is intended to put the choice of DID representation in the hands of each implementation.

There may be a number of alternative representations in the future, and my preference is a specification that need not be revised each time a new one is added. Static filenames ( did.json ) have been a feature of this specification from the early days when it was intended only as a single anchor for a given domain. The added flexibility of various representations via directory paths feels like an appropriate extension following the support of multiple DIDs on a single domain.

I don't think we get any value from having 2 JSON files that are potentially different...

My intent with this PR is not to require multiple representations, but rather to allow for variety within the representations used by any given implementation. The language indicating that multiple representations must be kept consistent is not intended to imply that multiple representations must exist - only to note they should be consistent if they do exist.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like a lot of the changes in this PR, but I don't see us getting any value from normatively defining support for multiple JSON representations at this time...

If I could go back and fix DID Core I would, but since its frozen, I think its best that DID Web support the JSON representation that is a super set and only that JSON representation... which is application/did+json.. but since that representation will not be understood by any web server... we should allow application/json to be returned for any did web document... google and others also do this... messing with accept headers should probably not be added until there is a really good reason, and so far, I am not seeing one.

We have a path to allowing multiple representations, but we should not take that path, because it introduces lots of security issues and is not worth the complexity or attack surface.

Copy link
Contributor

@msporny msporny Nov 30, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

JSON-LD should be possible, but it should not be required for DID Web... and I don't think we get any value from having 2 JSON files that are potentially different... other than pain.

Then we have a really deep technical problem here (no machine-verifiable extensibility mechanism), with no technical solution being proposed. Digital Bazaar will (eventually) formally object to any such change to did:web -- I don't think you guys are thinking this through.

An analogy would be "We made this car simpler by removing the emergency brake and the complex power steering system." -- you did make it simpler, you also made it more dangerous.

The only other alternative suggestion I can think of is, "yeah, do application/did+json ONLY and make the use of "@context" a MUST". Those that want to process it can do so and those that don't... don't. At least in that case, we can tell which properties were meant to be used and which ones are superfluous/unknown. If we don't do that, you end up having to publish a JSON Schema file for did:web -- and at that point, you're just re-creating JSON-LD (but with something that's a one-off for this DID Method). Wrt. interop... we have no idea without either of those two solutions, and that is a huge security vulnerability that Digital Bazaar is not willing to expose themselves to.

The more I think about this, the more I think not allowing proper content negotiation is a terrible idea. I'm starting to sour on this PR given the direction that @OR13 and @mprorock seem to be proposing... and I have no idea what portions of it that @gribneau is trying to move towards. We'll need to discuss this.

Copy link
Contributor

@msporny msporny Nov 30, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Upon re-reading @gribneau's comments, I tend to agree with him. @OR13 and @mprorock -- you guys seem to be trying to make this PR into something it's not. If you want to push your proposal, do it via a separate PR. Both of you seem to be arguing against content negotiation, which is going against Web Architecture (both at W3C and IETF).

Copy link
Contributor Author

@gribneau gribneau Nov 30, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Streamlined.

See #47 (comment)

Copy link
Contributor

@mprorock mprorock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe @OR13 captured all the changes I would like to see in, but I am reviewing for any additional CRs

Copy link
Contributor

@msporny msporny left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please make the following changes to this PR:

  1. Remove rants from the abstract, focus on the positive aspects of did:web.
  2. Clarify whether or not JSON-LD is illegal for did:web, and if so, how machine verifiability is practically achievable?
  3. Move the 'index' feature into its own PR
  4. Be specific about the server configuration information that is associated with MUST statements.
  5. Ideally, move different concepts/upgrades into different PRs.

index.html Outdated Show resolved Hide resolved
index.html Outdated Show resolved Hide resolved
Comment on lines -220 to -242
<section>
<h2>
JSON-LD context definition
</h2>
<p>
The definition of the Web DID JSON-LD context is:
</p>

<pre title="Web DID method JSON-LD context">
{
"@context":
{
"ethereumAddress": "https://github.com/uport-project/ethr-did-resolver#ethereumAddress",
"Secp256k1VerificationKey2018": "https://github.com/uport-project/ethr-did-resolver#Secp256k1VerificationKey2018",
"Secp256k1SignatureAuthentication2018": "https://github.com/uport-project/ethr-did-resolver#Secp256k1VerificationKey2018",
}
}
</pre>
<p class="issue" title="Update context">
Bring the context definition in line with latest DID Core.
</p>
</section>

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How do we express new cryptographic keys in a machine readable way? For example, when post-quantum public keys become a thing, how do we enforce any sort of machine verifiability here? How do I say I want to express JWK-based public keys and not LDI-based public keys?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this PR making it so that it's illegal for a did:web-based DID Document to contain JSON-LD?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR does not, at present, rule out JSON-LD. There is ongoing discussion on the topic, but this PR is currently representation-agnostic.

did:web:w3c-ccg.github.io
-> https://w3c-ccg.github.io/.well-known/did.json
-> https://w3c-ccg.github.io/.well-known/did/
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand how we're supposed to interpret this change.

index.html Outdated Show resolved Hide resolved
index.html Show resolved Hide resolved
Comment on lines +434 to +436
more than one. If multiple representations are supported, the server MUST prefer
the representation in the HTTP <code>Accept:</code> request header when selecting
an index file to return.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is confusing since the selection of a representation via HTTP is typically a negotiation between the client and the server... the client can provide preferences and the server typically picks among the ones it supports and prefers to respond with what the client would prefer (not what the server prefers).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The HTTP request headers are set by the client. The language in the PR obligates implementations to respect the client preference if possible.

Content negotiation is supported in some servers and will require custom configuration or scripting in others. Apache has a mod_negotiation that can be configured, but Nginx will require unusual configuration or scripting to meet the requirement.

index.html Outdated Show resolved Hide resolved
index.html Outdated Show resolved Hide resolved
</p>
<pre class="nohighlight">
https://example.com/u/bob/did.json
https://example.com/u/bob/
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

because it makes interop harder and introduces dangerous security issues which are "not worth it".

Citation? Evidence?

@gribneau
Copy link
Contributor Author

Resolved

  1. Remove rants from the abstract, focus on the positive aspects of did:web.

Done. 93a94dc

  1. Be specific about the server configuration information that is associated with MUST statements.

The inappropriate normative language has been removed.

Done. e308e22

  1. Ideally, move different concepts/upgrades into different PRs.

The full filepath behavior described here is an easy win, but we can pick it up later.

Removed. f18ed93

Outstanding

It seems that we have two outstanding issues to resolve.

  1. Clarify whether or not JSON-LD is illegal for did:web, and if so, how machine verifiability is practically achievable?

This PR is currently agnostic as to DID representation - it simply makes it possible to support a variety. JSON is used as an example, but the core specification is referenced for an authoritative list of possible representations, which included JSON and JSON-LD the last time I checked.

My personal preference is to preserve the flexibility, leaving DID representation as an implementation choice. It has been suggested that we limit to JSON and derivative representations and only handle media types that IANA has recognized, which means that we would presently support only the .json filename extension and application/json for all representations in the JSON family.

Within this issue, there is a conversation about whether @context should be required, and we will likely need to discuss that.

  1. Move the 'index' feature into its own PR

Looking forward, hardcoded filenames ( did.json ) will eventually break the agreement between filename extensions and media types when DID representations outside the JSON family are supported. Index files enable us to return the contents of a DID document with variable media types, so this provides a path forward.

IANA has not yet approved the DID-specific media types mentioned in IANA Considerations, so we can limit the present view to .json and application/json while noting the path forward in the information section.

It is reasonable on the same basis to defer content negotiation (handling the Accept) header, and also move those references into information with an indication that content negotiation should be standardized when the media types are available.

@OR13 @msporny @dmitrizagidulin @mprorock

@dmitrizagidulin
Copy link
Collaborator

dmitrizagidulin commented Dec 1, 2021

@gribneau - thank you for your continued work on this PR; this is great stuff.

Ok, so, the remaining issues in the PR (which unfortunately we can't really separate, since they do have to do with the resolution scheme) have to do with:

  1. Index files (both for the top-level domain, and for each individual directory)
  2. Whether or not we should allow multiple representations for DID Documents, for the did:web method.
  3. (related to above) Whether or not we should allow content negotiation.

After having separate conversations with a couple of you, I'd like to see if we can agree on some guiding principles for this DID method, and would like to make a proposal that would hopefully satisfy everyone's concerns

Guiding Principles

  1. We want did:web DIDs to be able to be hosted on "dumb" web servers (which cannot do dynamic processing of DID Documents, to convert between representations etc). An example of this would be Github Pages, or the innumerable amount of hosted websites where a webmaster just has a CPanel (FTP-like) access to hosted files.
  2. We do NOT want webmasters (people who can upload files to the top directory, or subdirectories of a site) to have to maintain multiple copies of a DID Document (one for each representation). As @OR13 points out, the security implications of needing to rotate a key, and making sure all copies/representations are in sync... should be avoided.
  3. We DO want JSON-LD @contexts to be possible/supported, so that those companies that use contexts can continue using them in their did docs.
  4. We recognize that did:web implementers will be adding new properties and types (one very common example -- new serviceEndpoint types), but may not have the skills or the ability to add those new properties to proper @contexts. So, we do NOT want to require that all did:web documents be able to be processed by JSON-LD processors. (However, see the previous items -- we DO want to still allow those who do want to do it.)
  5. We DO want to be able to host multiple DIDs per domain (which means, at very least, different DID Docs in subdirectories).

Implications

Can we agree on these guidelines?

On the remote chance that we can -- the guidelines only leave us a couple of options (and hey, constraints are good). For example, principle 1 (we want people who only have access to "dumb" web servers to be able to host these DIDs) means that we cannot do content negotiation. And yes, we know that the Web Architecture would prefer it if we could. But most web servers will not be able to convert JSON DID Documents to YAML or whatever.
Meaning, we not only have to deal with the fact that IANA has not registered application/ld+did+json, but that even if it did, it doesn't matter, many webservers just can't do conneg at all.

Furthermore, if we want to support dumb web servers (and we do), it means no automatic server-side handling of index files, either. Many webmasters just don't have access to the Nginx config for their servers, so it's not guaranteed that they can set up default index file handling.
But we still want nice concise DIDs like did:web:example.com, right? So how can we have that without server-side index file handling?

Well, we can have those nice things if we specify the client-side mapping. So that it's the client resolver (not the web server via redirect), when encountering did:web:example.com knows to automatically convert it to https://example.com/ and append did.json (or whatever index file convention we agree on). And so the resolver will always request https://example.com/did.json, when faced with that DID.

So early on in this PR's comments, I thought that we might be able to still have multiple representations, using this client-side index file mapping. Meaning, if a resolver wants a JSON-LD DID Doc, it'll append index.jsonld, and if it wants a JSON one, it'll append index.json, and so on.
But as principle 2 (we do NOT want multiple copies of a DID Doc to keep in sync) points out, that's asking for potential trouble.

So where does that leave us? If we can't have content negotiation, and we can't have automatic server-side redirects to index files based on Accept headers, and we can't emulate content negotiation via just having multiple copies of a DID Doc with different extensions?

It means we need to agree on a superset representation, and agree on a client-side index file mapping.

Could that superset representation be JSON-LD? I'm a huge fan of JSON-LD, but I recognize that many implementers (or just regular application developers doing prototypes using did:web DIDs) will want to add new properties (such as new service endpoint types, etc), but won't know how to add those properties to their own @contexts.
So I don't think it's realistic (meaning: people will object) that we require did:web docs to be fully JSON-LD processable, and that all terms in them be also present in valid contexts.
Also, you only need to be strict about contexts when you're signing stuff (like in VCs), and we agreed we're not going to be signing the did:web docs themselves.

But! We still want the option to use JSON-LD. For many companies, it's our competitive advantage / secret sauce.
So, making JSON-LD or @contexts illegal for did:web is out of the question.

So where does this all leave us?

Proposal

I'd like to propose the following, which I think would follow those guiding principles, address everyone's concerns, and makes for fairly concise spec and easy implementation.

  1. We require that all did:web DIDs be served as JSON resources -- to be files with .json extensions, so that even if a web server doesn't know how to provide the correct content-type, the browser can at least derive application/json just from the extension. This means - no content negotiation, and no multiple copies for different representations.
  2. We specify a client-side mapping -- when a resolver encounters a did:web DID, it will first convert it to a URL (performing percent-decoding as specified in this PR), and then always know to append did.json to it. And that's the file it will request.
  3. We do not bother with did:web DIDs specifying individual files. So, no https://example.com/alice-did.json. The file part of the path is always derived automatically.
  4. We do not require @contexts to be in those JSON files on the web server. But we also don't do anything ridiculous like make contexts illegal.
  5. We take advantage of the fact that DID Core guarantees us that any conforming DID Document has reserved properties (id, service endpoints, verificationMethod, etc) that also make it compliant with a DID Core @context (https://www.w3.org/ns/did/v1). And so we specify that all did:web docs have an implicit default context of https://www.w3.org/ns/did/v1. Which means that if a resolver does want to process it as JSON-LD, it can always append the did-core context (if it's not in the did doc already).

What do people think?

@gribneau
Copy link
Contributor Author

gribneau commented Dec 1, 2021

Thanks @dmitrizagidulin.

I am comfortable with the statement of issues and the Guiding Principles.

The Implications that webservers cannot do content negotiation, and won't be able to specify index files seem overbroad to me. Apache and Nginx can be made to work without redirects, including at least basic content negotiation as described in the core spec, and Apache's DirectoryIndex can appear in a .htaccess file within a directory to set the index, even with FTP management or a dashboard like cPanel. @OR13 has made Github static pages work. Directory index functionality, at a minimum, is ubiquitous.

In short, I think the failure cases will be less common than anticipated. Given that what we're dealing with here is a failure case, I think there is one upgrade we can make to the Proposal:

  1. We specify a client-side mapping -- when a resolver encounters a did:web DID, it will first convert it to a URL (performing percent-decoding as specified in this PR), and then always know to append did.json to it. And that's the file it will request.

If we first attempt to make the request as currently described in the PR, but in cases of failure append the did.json and run the request again as an additional fallback, then we will have covered both the common success case and the infrequent failures while preserving our path forward in compliance with the core specification.

Does that work for everyone?

@OR13
Copy link
Collaborator

OR13 commented Dec 1, 2021

@gribneau thanks for pushing on this... :)

I would prefer not to need to make 2 network requests to resolve a DID, ever.

These edge cases are complexity we should factor out imo.

In a world where there is only 1 did content type, accept header can be ignored, and a file with a .json extension can be produced from the did to url conversion.

in a world where there is more than 1 did content type, accept header MUST be understood, and a file with a MATCHING file extension can be produced from the DID and Accept parameter conversion.

I don't see asking web servers to handle this for us as helping simplify the standard.

@dmitrizagidulin
Copy link
Collaborator

+1, I agree, I'd prefer not to make 2 network requests, if at all possible.
What would supporting directory index functionality actually buy us, anyway?

@gribneau
Copy link
Contributor Author

gribneau commented Dec 1, 2021

Further Consideration

I would prefer not to need to make 2 network requests to resolve a DID, ever.

+1
I thought about this last night, and it seems to me that a second request as a failback is a poor replacement for simply supporting full filenames in the spec itself.

These edge cases are complexity we should factor out imo.

If we revert f18ed93 then implementations that suffer from difficulty with index files or content negotiation still have a path enabling a simple resolution to a single specified DID representation by specifying the full filename. I prefer that as a solution to second requests in cases of failure.

Representation selection is explicit when full filenames are used - the filename extension specifies the representation. This effectively factors out edge cases by removing any ambiguity.

In a world where there is only 1 did content type, accept header can be ignored...

Consider these cases:

did:web:example.com:alice.json
--> https://example.com/alice.json

did:web:example.com:alice.didjson
--> https://example.com/alice.didjson

did:web:example.com:alice.didjsonld
--> https://example.com/alice.didjsonld

did:web:example.com:alice.didcbor
--> https://example.com/alice.didcbor

Each of those maps to exactly one file, and the filename extension controls the content-type header that the server returns. Using this scheme, there is no need for content negotiation or index file selection - the webserver simply returns the file, which is as basic and simple as it gets on the server side.

Dmitry's upgrade on full filenames solved this problem before it was called out. Very nice.

Benefits

What would supporting directory index functionality actually buy us, anyway?

The core spec envisions multiple representations for a DID, and there are different filename extensions and media types associated with each. We've been resolving to a file named did.json since the earliest versions of this method when it was serving a single DID for a domain to act as an anchor for that entire domain.

Now that we are able to serve multiple DIDs on a given domain, the method can support significantly more use cases. This PR takes the next step to support more of the core specification by making it possible to return various representations. The use of index files removes the filename (and filename extension) from the HTTPS URL to enable webservers to return a DID document with the appropriate media type in the content-type header, in compliance with very long standing best practices.

The use of directory urls, index files, and accept header analysis is a very direct mapping between the existing HTTP world and the IANA considerations section of the core spec.

To summarize the above, directory index functionality and content negotiation gets us a considerably more complete method for resolving DIDs with simple webservers.

The Bigger Picture

That's nice, as far as it goes, but there is a higher order benefit to be had here.

There is tremendous overlap between the core spec and existing HTTP content negotiation practices. This will all be very familiar territory for systems administrators and software developers, and the similarity should be comforting to those tasked with implementing both client and server side applications to handle this method.

Given that we are presently facing formal objections on the core spec, I think we should move forward with this PR, inclusive of full filename resolution to avoid edge cases, to provide one more concrete example that addresses at least some of the stated concerns.

@dmitrizagidulin
Copy link
Collaborator

dmitrizagidulin commented Dec 1, 2021

@gribneau - You're right that allowing did:web DIDs to link to individual files would allow multiple representations without the need for server-side index file handling or content negotiation.

However, it greatly increases the risk of going against guiding principle 2 from above -- I strongly believe (as I think does @OR13) that we must not allow multiple file versions of the same DID document.

So, I would support linking to individual files only if we also add normative language in the spec that only one such representation file must exist, for any given DID. (Meaning, if you have did:web:example.com:did.yaml, there must not exist any other file for that did with any other extension.)

While I can live with that, I would personally prefer we simplify the did:web method to having just one representation. (Greatly simplifies the developer ask.)

@tplooker
Copy link

tplooker commented Dec 7, 2021

While I can live with that, I would personally prefer we simplify the did:web method to having just one representation. (Greatly simplifies the developer ask.)

+1, I may still be mis-understanding the ask being made in your latest proposal here @gribneau as I am just getting up to speed on this thread.

Supporting multiple representations of the same DID document and further more having these resolved through different identifiers (DIDs) (e.g did:web:example.com:did.yaml vs did:web:example.com:did.json) feels like problematic coupling (identifier, data model and data representation used) and would be liable to create security issues through the opportunity for consistency problems across the different representations.

So, I would support linking to individual files only if we also add normative language in the spec that only one such representation file must exist, for any given DID. (Meaning, if you have did:web:example.com:did.yaml, there must not exist any other file for that did with any other extension.)

At this stage Im a -1 to any proposal that involves requiring the file extension to be leaked into the resulting did web identifier, as I said before though I may still be mis-understanding this ask?

Copy link

@tplooker tplooker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just putting in a review here recording my current position on this PR as described in my prior comments.

@gribneau
Copy link
Contributor Author

gribneau commented Dec 13, 2021

The consensus seems to be that we should not support multiple representations or content negotiation within did:web as laid out in did:core. Upon reflection, I think it would be counterproductive to move forward with this PR if we aren't actually going to follow through to support standard Web Architecture upon which those sections of did:core are clearly based.

If there will be a more complete HTTP method for DID resolution, that method should

  • resolve to directory URLs,
  • handle Accept headers,
  • and return the appropriate representation together with the appropriate media type in the content-type header.

This is rather common functionality in the HTTP world and is described in the IANA Considerations and DID Resolution Options sections of the core spec. Resolving to directory URLs in did:web is unnecessary to return a single representation and would introduce confusion by hijacking the HTTP content negotiation capabilities without supporting content negotiation.

Absent objections in the next few days, I'll simply revert and close.

@OR13
Copy link
Collaborator

OR13 commented Dec 13, 2021

@gribneau there appear to still be changes in here that might be accepted in isolation... but you might find it easier to open new PRs with smaller change sets to get them in.

Thanks for helping drive these issues forward.

I am in favor of closing this PR, and opening smaller ones focused on 1 item at a time, so long as they do not:

  1. add any new representations
  2. remove any existing resolution rules for did web

I do think we should continue to debate removing the .well-known resolution rule, but its probably better to stick to issues for now.

@OR13
Copy link
Collaborator

OR13 commented Dec 13, 2021

I opened #49 to discuss .well-known resolution rule simplification, lets discuss there.

@kdenhartog
Copy link
Contributor

Yeah splitting this PR into a few different ones would be good. For example the abstract change could be addressed independently

@gribneau
Copy link
Contributor Author

Yeah splitting this PR into a few different ones would be good. For example the abstract change could be addressed independently

I agree. There are many cosmetic improvements that should be made.

@msporny
Copy link
Contributor

msporny commented Dec 14, 2021

I opened #52 to discuss general simplification for DID URL resolution -- we're making this way harder than it needs to be.

@msporny
Copy link
Contributor

msporny commented Dec 14, 2021

At this stage Im a -1 to any proposal that involves requiring the file extension to be leaked into the resulting did web identifier, as I said before though I may still be mis-understanding this ask?

What if a did:web identifier file extension was a part of the identifier? URLs are meant to be opaque. Once you go did.json, you are stuck with that as your identifier... you /can't/ serve anything else... we have people objecting to conneg in this same thread... one of those things is going to have to give.

@gribneau
Copy link
Contributor Author

At this stage Im a -1 to any proposal that involves requiring the file extension to be leaked into the resulting did web identifier, as I said before though I may still be mis-understanding this ask?

What if a did:web identifier file extension was a part of the identifier? URLs are meant to be opaque. Once you go did.json, you are stuck with that as your identifier... you /can't/ serve anything else... we have people objecting to conneg in this same thread... one of those things is going to have to give.

Before removing the full filename option, this PR offered a path to both content negotiation and full file paths, leaving the choice in the hands of the implementation.

It is now at an impasse and has now been superseded by multiple more granular PRs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants