make extensions follow the same pattern as normal attrs (#505)

* issue 498 - require extensions to have ce- version

Signed-off-by: Doug Davis <[email protected]>

* fix mqtt

Signed-off-by: Doug Davis <[email protected]>

* add pointer for mark

Signed-off-by: Doug Davis <[email protected]>

amqp-format.md

@@ -76,15 +76,17 @@ representation before surfacing the value to the API. An AMQP implementation
 SHOULD convert from/to the native runtime or language type system to the AMQP
 type system directly without translating through strings whenever possible.
 
-Extension specifications MAY define diverging mapping rules for the values of
-attributes they define.
+Extension specifications MAY define a
+[secondary mapping rule](./amqp-transport-binding.md#313-metadata-headers)
+for the values of attributes they define, but MUST also include the previously
+defined primary mapping.
 
 For instance, the attribute value may be a data structure defined in a standard
 outside of CloudEvents, with a formal AMQP mapping, and there might be risk of
 translation errors or information loss when the original format is not
 preserved.
 
-An extension specification that defines a diverging mapping rule for AMQP, and
+An extension specification that defines a secondary mapping rule for AMQP, and
 any revision of such a specification, MUST also define explicit mapping rules
 for all other event formats that are part of the CloudEvents core at the time of
 the submission or revision.

amqp-transport-binding.md

@@ -160,13 +160,14 @@ All [CloudEvents][ce] attributes with exception of `datacontenttype` and `data`
 MUST be individually mapped to and from the AMQP
 [application-properties][app-properties] section, with exceptions noted below.
 
-CloudEvents extensions that define their own attributes MAY define a diverging
+CloudEvents extensions that define their own attributes MAY define a secondary
 mapping to AMQP properties for those attributes, also in different message
 sections, especially if specific attributes or their names need to align with
 AMQP features or with other specifications that have explicit AMQP header
-bindings.
+bindings. However, they MUST also include the previously defined primary
+mapping.
 
-An extension specification that defines a diverging mapping rule for AMQP, and
+An extension specification that defines a secondary mapping rule for AMQP, and
 any revision of such a specification, MUST also define explicit mapping rules
 for all other transport bindings that are part of the CloudEvents core at the
 time of the submission or revision.

avro-format.md

@@ -56,8 +56,9 @@ The CloudEvents type system MUST be mapped to Avro types as follows.
 | URI-reference | [string][avro-primitives] following [RFC 3986 §4.1][rfc3986-section41] |
 | Timestamp     | [string][avro-primitives] following [RFC 3339][rfc3339] (ISO 8601)     |
 
-Extension specifications MAY define diverging mapping rules for the values of
-attributes they define.
+Extension specifications MAY define secondary mapping rules for the values of
+attributes they define, but MUST also include the previously defined primary
+mapping.
 
 ### 2.3 OPTIONAL Attributes
 

documented-extensions.md

@@ -35,7 +35,7 @@ Extensions always follow a common placement strategy for in-memory formats (e.g.
 representations. Transport bindings (e.g. [HTTP](http-transport-binding.md),
 [MQTT](mqtt-transport-binding.md), [AMPQ](amqp-transport-binding.md),
 [NATS](nats-transport-binding.md)) provide default placement for extensions, but
-an extension MAY require special representation when transported (e.g. tracing
+an extension MAY require special secondary representation when transported (e.g. tracing
 standards that require specific headers). Extension authors SHOULD only require
 special representation in transport bindings where extensions integrate with
 pre-existing specs; extensions with custom transport bindings are much more

extensions/distributed-tracing.md

@@ -43,3 +43,13 @@ CURL -X POST example/webhook.json \
 -H 'content-type: application/cloudevents+json' \
 -d '@sample-event.json'
 ```
+
+## Conflicts
+
+Since this extension defines secondary, special, seialization that differs
+from other CloudEvents attributes, it is possible that the values of these two
+could differ by the time the event is received at a destination. In those
+cases, the serialization that followed the "general CloudEvents serialization
+rules" MUST be used as the CloudEvents attribute. The other, secodary,
+mapping MAY be picked-up and offered to the receiving application as
+"additional" metadata.

http-transport-binding.md

@@ -70,8 +70,9 @@ _binary_, _structured_ and _batched_. Every compliant implementation SHOULD
 support the _structured_ and _binary_ modes.
 
 In the _binary_ content mode, the value of the event `data` is placed into the
-HTTP request or response body as-is, with the `datacontenttype` attribute value
-declaring its media type; all other event attributes are mapped to HTTP headers.
+HTTP request, or response, body as-is, with the `datacontenttype` attribute
+value declaring its media type in the HTTP `Content-Type` header; all other
+event attributes are mapped to HTTP headers.
 
 In the _structured_ content mode, event metadata attributes and event data are
 placed into the HTTP request or response body using an
@@ -102,15 +103,9 @@ identically to [HTTP over TLS]([RFC2818][RFC2818]).
 This specification does not further define any of the core [CloudEvents][ce]
 event attributes.
 
-Two of the event attributes, `datacontenttype` and `data` are handled specially
-and mapped onto HTTP constructs, all other attributes are transferred as
-metadata without further interpretation (except that extensions MAY define
-additional header mappings).
-
 This mapping is intentionally robust against changes, including the addition and
 removal of event attributes, and also accommodates vendor extensions to the
-event metadata. Any mention of event attributes other than `datacontenttype` and
-`data` is exemplary.
+event metadata.
 
 ### 2.1. datacontenttype Attribute
 
@@ -161,34 +156,30 @@ efficient transfer and without transcoding effort.
 
 For the _binary_ mode, the HTTP `Content-Type` header value corresponds to
 (MUST be populated from or written to) the CloudEvents `datacontenttype`
-attribute.
+attribute. Note that a `ce-datacontenttype` HTTP header MUST NOT also be
+present in the message.
 
 #### 3.1.2. Event Data Encoding
 
 The [`data`](#22-data) byte-sequence is used as the HTTP message body.
 
 #### 3.1.3. Metadata Headers
 
-All [CloudEvents][ce] attributes with the exception of `datacontenttype` MUST be
-individually mapped to and from distinct HTTP message headers, with exceptions
-noted below.
+All other [CloudEvents][ce] attributes, including extensions, MUST be
+individually mapped to and from distinct HTTP message header.
 
-CloudEvents extensions that define their own attributes MAY define a diverging
+CloudEvents extensions that define their own attributes MAY define a secondary
 mapping to HTTP headers for those attributes, especially if specific attributes
 need to align with HTTP features or with other specifications that have explicit
-HTTP header bindings.
-
-An extension specification that defines a diverging mapping rule for HTTP, and
-any revision of such a specification, MUST also define explicit mapping rules
-for all other transport bindings that are part of the CloudEvents core at the
-time of the submission or revision.
+HTTP header bindings. Note that these attributes MUST also still appear in the
+HTTP message as HTTP headers with the `ce-` prefix as noted in
+[HTTP Header Names](#3131-http-header-names).
 
 ##### 3.1.3.1. HTTP Header Names
 
-Except for attributes
-[explicitly handled in this specification](#2-use-of-cloudevents-attributes),
-the naming convention for the HTTP header mapping of well-known CloudEvents
-attributes is that each attribute name MUST be prefixed with "ce-".
+Except where noted, all CloudEvents context attributes, including extensions,
+MUST be mapped to HTTP headers with the same name as the attribute name but
+prefixed with `ce-`.
 
 Examples:
 
@@ -213,7 +204,7 @@ String values MUST be percent-encoded as described in [RFC3986, section
 2.4][rfc3986-section-2-4] before applying the header encoding rules described in
 [RFC7230, section 3.2.6][rfc7230-section-3-2s6].
 
-When decoding an HTTP message into a Cloud Event, these rules MUST be applied in
+When decoding an HTTP message into a CloudEvent, these rules MUST be applied in
 reverse -- [RFC7230, section 3.2.6][rfc7230-section-3-2-6] decoding to an ASCII
 string, and then a **single round** of percent-decoding as described in
 [RFC3986, section 2.4][rfc3986-section-2-4] to produce a valid UTF-8 String.

json-format.md

@@ -70,15 +70,16 @@ with exceptions noted below.
 | URI-reference | [string][json-string] following [RFC 3986][rfc3986]            |
 | Timestamp     | [string][json-string] following [RFC 3339][rfc3339] (ISO 8601) |
 
-Extension specifications MAY define diverging mapping rules for the values of
-attributes they define.
+Extension specifications MAY define secondary mapping rules for the values of
+attributes they define, but MUST also include the previously defined primary
+mapping.
 
 For instance, the attribute value might be a data structure defined in a
 standard outside of CloudEvents, with a formal JSON mapping, and there might be
 risk of translation errors or information loss when the original format is not
 preserved.
 
-An extension specification that defines a diverging mapping rule for JSON, and
+An extension specification that defines a secondary mapping rule for JSON, and
 any revision of such a specification, MUST also define explicit mapping rules
 for all other event formats that are part of the CloudEvents core at the time of
 the submission or revision.

mqtt-transport-binding.md

@@ -65,9 +65,9 @@ placed into the MQTT PUBLISH message payload section using an
 
 In the _binary_ content mode, the value of the event `data` is placed
 into the MQTT PUBLISH message's payload section as-is, with the
-`datacontenttype` attribute value declaring its media type; all other event
-attributes are mapped to the MQTT PUBLISH message's [properties
-section][5-publish-properties].
+`datacontenttype` attribute value declaring its media type in the MQTT
+PUBLISH message's [`Content Type`][5-content-type] property; all other
+event attributes are mapped to User Property fields.
 
 ### 1.4. Event Formats
 
@@ -88,14 +88,9 @@ mandate specific existing features to be used.
 This specification does not further define any of the [CloudEvents][ce] event
 attributes.
 
-Two of the event attributes, `datacontenttype` and `data` are handled specially
-and mapped onto MQTT constructs, all other attributes are transferred as
-metadata without further interpretation.
-
 This mapping is intentionally robust against changes, including the addition and
 removal of event attributes, and also accommodates vendor extensions to the
-event metadata. Any mention of event attributes other than `datacontenttype` and
-`data` is exemplary.
+event metadata.
 
 ### 2.1. datacontenttype Attribute
 
@@ -154,19 +149,15 @@ payload of the MQTT PUBLISH message.
 
 #### 3.1.3. Metadata Headers
 
-All [CloudEvents][ce] attributes with exception of `datacontenttype` and `data`
-MUST be individually mapped to and from the User Property fields in the MQTT
-PUBLISH message, with exceptions noted below.
+All other [CloudEvents][ce] context attributes, including extensions, MUST be
+individually mapped to and from the User Property fields in the MQTT
+PUBLISH message.
 
-CloudEvents extensions that define their own attributes MAY define a diverging
+CloudEvents extensions that define their own attributes MAY define a secondary
 mapping to MQTT user properties or features for those attributes, especially if
 specific attributes need to align with MQTT features, or with other
-specifications that have explicit MQTT header bindings.
-
-An extension specification that defines a diverging mapping rule for MQTT, and
-any revision of such a specification, MUST also define explicit mapping rules
-for all other transport bindings that are part of the CloudEvents core at the
-time of the submission or revision.
+specifications that have explicit MQTT header bindings. However, they MUST
+also include the previously defined primary mapping.
 
 ##### 3.1.3.1 User Property Names
 
@@ -203,6 +194,7 @@ type: com.example.someevent
 time: 2018-04-05T03:56:24Z
 id: 1234-1234-1234
 source: /mycontext/subcontext
+datacontenttype: application/json; charset=utf-8
        .... further attributes ...
 
 ------------------ payload -------------------
@@ -262,6 +254,10 @@ Content Type: application/cloudevents+json; charset=utf-8
 {
     "specversion" : "0.4-wip",
     "type" : "com.example.someevent",
+	"time" : 2018-04-05T03:56;24Z,
+	"id" : 1234-1234-1234,
+	"source" : "/mycontext/subcontext",
+	"datacontenttype" : "application/json; charset=utf-8",
 
     ... further attributes omitted ...
 
@@ -287,6 +283,10 @@ Topic Name: mytopic
 {
     "specversion" : "0.4-wip",
     "type" : "com.example.someevent",
+	"time" : 2018-04-05T03:56;24Z,
+	"id" : 1234-1234-1234,
+	"source" : "/mycontext/subcontext",
+	"datacontenttype" : "application/json; charset=utf-8",
 
     ... further attributes omitted ...
 

spec.md

@@ -406,8 +406,10 @@ See
 [CloudEvent Attributes Extensions](primer.md#cloudevent-attribute-extensions)
 for additional information concerning the use and definition of extensions.
 
-This specification places no restriction on the type or semantics of the
-extension attributes. Each definition of an extensions SHOULD fully define all
+This specification places no restriction on the semantics of the
+extension attributes, but they MUST use one of the types defined within the
+[type system](#type-system).
+Each definition of an extensions SHOULD fully define all
 aspects of the attribute - e.g. its name, semantic meaning and possible values
 or even to indicate that it places no restrictions on its values. New extension
 definitions SHOULD use a name that is descriptive enough to reduce the chances
@@ -419,13 +421,31 @@ extensions that might be of interest.
 Each specification that defines how to serialize a CloudEvent will define how
 extension attributes will appear.
 
+Extension attribtue MUST be serialized using the same general pattern as all
+CloudEvents context attributes. For example, in binary HTTP, that means they
+MUST appear as HTTP headers with the `ce-` prefix. The specification of an
+attribute MAY define a secondary serialization where the data is duplicated
+in some other location within the message.
+
+In cases where a secondary serialization is defined, the extension
+specification MUST also state what a receiver of the CloudEvent is to do
+if the data differs between those two serialization locations. Additionally,
+senders need to be prepared for intermediaries, and receivers, to not
+know about their extension and therefore the specialized serialization version
+version will most likely not be processed as a CloudEvent extension attribute.
+
+Many transports support the ability for senders to include additonal metadata,
+for example as HTTP headers. While a CloudEvents receiver is not mandated to
+process and pass them along, it is RECOMMENDED that they do so via some
+mechanism that makes it clear they are non-CloudEvents metadata.
+
 Here is an example that illustrates the need for additional attributes. In many
 IoT and enterprise use cases, an event could be used in a serverless application
 that performs actions across multiple types of events. To support such use
 cases, the event producer will need to add additional identity attributes to the
 "context attributes" which the event consumers can use to correlate this event
 with the other events. If such identity attributes happen to be part of the
-event "data", the event producer SHOULD also add the identity attributes to the
+event "data", the event producer would also add the identity attributes to the
 "context attributes" so that event consumers can easily access this information
 without needing to decode and examine the event data. Such identity attributes
 can also be used to help intermediate gateways determine how to route the