Swagger for WebSocket services

[sten]

Is there a way to use the Swagger Specification for WebSockets? It seems to be quite bound to HTTP.

[webron]

It's meant to be used for REST APIs so...

On Wed, May 14, 2014 at 6:48 PM, Sten Govaerts [email protected]:

Is there a way to use the Swagger Specification for WebSockets? It seems
to be quite bound to HTTP.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/55
.

[webron]

There are possible plans to expand the spec to provide support for it.
For now I'm closing the issue but marking at as a proposal for the next version.

[buremba]

+1

[ralfhandl]

+1: OData has a mechanism to "subscribe" to changes of requested resources, see http://docs.oasis-open.org/odata/odata/v4.0/errata02/os/complete/part1-protocol/odata-v4.0-errata02-os-part1-protocol-complete.html#_Toc406398235.

The ideal notification channel is WebSockets, and we'd like to be able to describe the service-specific shape of that callback/notification mechanism.

[travishaagen]

For barebone requirements, a Websocket has a HTTP endpoint (e.g., /websocket) and supports a few types of JSON messages,

• request / response
• push from client (no response expected)
• push from server (no response expected)

Sample request frame,
{ "cmd": "createUser", "cmdId": "12", ... }

Sample response frame,
{ "cmdId": "12", ..., "status": 200 }

[webron]

Parent: #586

[chrisdostert]

+1

[jroper]

While there may be some higher level semantics that people might want to specify on top of WebSockets, eg subscribing to notifications, I think use cases like that are not really that well understood from a standardisation perspective, and there would be little value in trying to standardise high level concepts like that now, as best practices and idioms are likely to change significantly over time.

I agree with @travishaagen, though I would remove the request/response requirement as it too is also a high level semantic that really has no relation to WebSockets itself.

At the most basic level, I would describe a WebSocket in the same way that a regular HTTP endpoint is described, with a request/response message schema, but unlike a regular HTTP endpoint, on a WebSocket there will be zero to many instances of the request/response messages sent in each direction. From a spec perspective, this could be indicated by simply adding a flag to say that the request and/or response messages are streamed. Further information could be added to indicate what streaming protocol should be used (websockets, SSE, newline separated JSON ala Twitter streaming endpoint style). Higher level semantics such as request/response within the stream, publish/subscribe, etc can in future be built on top of that, but I think starting with just being able to specify that a particular endpoint is a stream is a good start.

[jeromu]

+1

[jwilander]

+1

[alechenninger]

FWIW, HTTP2 is a multiplexing + streaming protocol, like WebSockets. Might make sense to tackle the uses cases for either protocol together.

[darrelmiller]

@alechenninger From an HTTP API perspective, HTTP/2 really makes no changes over HTTP/1.1. It is possible that we could describe pushed requests, but if an intermediary cache handles the pushed requests as intended, then there is no need to change the API description.

You are not the first person I have run into that has said HTTP/2 supports streaming, but I have not yet seen anything that changes HTTP semantics. You can do streaming in HTTP/2 in exactly the same way that you can do it in HTTP/1.1. Due to other changes it will likely be more efficient, but I don't see how it changes anything from an API description perspective.

There is the notion of control flow that should finally do away with chunk encoding and make life much easier for connectors to manage memory when dealing with large payloads. Allowing streams to pause will allow having multiple long polling requests going over a single connection.

All of these changes are awesomely fabulous, but I see nothing that changes how HTTP APIs should be described. It possibly will make websockets more efficient, but HTTP/2 isn't a full-duplex protocol. It is the same old client server protocol that we know and love.

If I am wrong about this, I would love somebody to point to me some specifications that shows me why I am wrong.

[alechenninger]

@darrelmiller As far as I know, HTTP/2 is a full duplex protocol. Disclaimer: I'm no expert.

For example see the gRPC wire format, which takes advantage of HTTP2 streaming semantics: http://www.grpc.io/docs/guides/wire.html

Also see the spec: https://http2.github.io/http2-spec/#StreamsLayer

[darrelmiller]

@alechenninger Maybe I'm misusing the terms, I need to investigate more.
"A "stream" is an independent, bidirectional sequence of frames exchanged between the client and server within an HTTP/2 connection"

The key here is that it is "within an HTTP/2 connection". Even, in a HTTP/1.1 connection, once a client has made the request, the server can return bytes down the wire when ever it chooses. That's how long polling works. What I don't think can ever happen is have the server initiate a HTTP/2 connection with a client.

What I may have misunderstood is that it may be possible for a server to send HTTP/2 frames that don't correspond to a request once a connection has been established. I have not seen this discussed in anything I have read so far, but that doesn't mean it is not possible.

[darrelmiller]

From here https://http2.github.io/http2-spec/#rfc.section.8.1

A server can send a complete response prior to the client sending an entire request if the response does not depend on any portion of the request that has not been sent and received.

This is a very interesting difference because this means that a long polling interaction can effectively do full-duplex communication of bytes, once the client has sent the initial request.

[jroper]

This is a very interesting difference because this means that a long polling interaction can effectively do full-duplex communication of bytes, once the client has sent the initial request.

That's not a difference, the HTTP/2 spec here is simply being explicit about something that HTTP/1.1 has always supported (and there are real world uses cases out in the wild that I've seen that exploit this feature of HTTP/1.1, as well as a number of client and server implementations that support it, including one that I used to be the lead developer of - Play Framework). In fact HTTP/1.1 even has a way of semantically expecting a complete response before sending a complete request, it's part of the Expect: 100-continue spec.

So I agree with your initial comment about there being no semantic difference between HTTP/1.1 and HTTP/2, and so nothing to change from API perspective. The streams feature of HTTP/2 only allows multiplexing of multiple concurrent exchanges down a single connection, semantically, nothing changes since in HTTP/1.1 you achieved exactly the same thing by making multiple connections, it changes nothing in terms of the full/half duplex nature of the exchanges that happen, and has no impact on the high level API semantics.

The doing away with chunked encoding by the way is a bit misleading, it's not so much that chunked encoding is done away with, its that now effectively everything is chunked into frames that have headers that specify the length of each frame (which is precisely what chunked encoding is).

[lyschoening]

@jroper are you saying HTTP/1.1 supported multiple bidirectional messages within a single HTTP request?

[jroper]

No, I'm saying it supports bidirectional streams in a single HTTP request. There's still only one message each way (from an HTTP semantics perspective), but that message can be a stream, and both ends can be streaming at the same time. The message can, as is the case with SSE, be an encoding of many smaller messages which are streamed over time, but the description of that is beyond the scope of HTTP.

Note it's one thing to say "HTTP supports this", it's another thing for it actually to be supported by clients and servers. Most clients (eg browsers) can't do it, and most traditional servers can't either. But there are some that can.

[MaksimOrlov]

+1

[RobDolinMS]

@sten Is this still an issue with the latest OAI v3 spec?

[BrunoZell]

Take a look at https://www.asyncapi.com/
It's basically the Swagger of message based systems.

[karezza]

Seems like it would be possible to add a tag such as [WebSocket] to a REST definition and then the swagger would show this attribute. In a way similar to supporting Authorize and getting a lock icon.

Even if this was the only websocket tag functionality, it would be helpful.

[derberg]

@BrunoZell thanks a lot for mentioning AsyncAPI

I wrote two articles about describing WebSockets with AsyncAPI that should explain the relation between the two and how to use AsyncAPI to describe WebSocket API. Have a look and let me know if you need anything more.

Articles:

• WebSocket, Shrek, and AsyncAPI - An Opinionated Intro
• Creating AsyncAPI for WebSocket API - Step by Step
• From API-First to Code Generation - A WebSocket Use Case

Video:

• AsyncAPI and WebSocket

So yeah, come and visit https://www.asyncapi.com/

[brunogirin]

For the sake of specifying a WS endpoint as part of a wider REST API, could the initial endpoint be defined as a HTTP GET with an alternate server and headers? For example:

paths:
  myws:
    get:
      servers:
        - url: ws://example.com
      parameters:
        - name: Connection
          in: header
          required: true
        - name: Upgrade
          in: header
          required: true
        - name: Sec-WebSocket-Key
          in: header
          required: true
        - name: Sec-WebSocket-Version
          in: header
          required: true
      responses:
        101:
          headers:
            Connection: 
              required: true
            Upgrade:
              required: true
            Sec-WebSocket-Accept:
              required: true

That's a bit long winded and it doesn't define any payloads but at least it should be enough to warn client and server libraries that "here be dragons". For a future version of the spec, what about making it possible to define endpoints that trigger a protocol switch via the Upgrade header and for such endpoints, specify an external spec document, such as an AsyncAPI document?

Use case for this: I work with services for which a REST API is the best solution for 90% of interactions and the remaining 10% is better served async, typically via web sockets. So ideally, I'd like my main document to be an OpenAPI spec that is able to hand over to other specialised specs for the endpoints that do things that are not standard REST.

[MarwanRefaat]

Any updates on this? +1 for sure!

[bali182]

I went through all the issues I found about the topic of sockets, and under each someone mentions that there needs to be a proposal for this to happen. How does one make a proposal? I see nothing in the readme about this.

In any case here's what I'm suggesting. The websocket API on the client side is very simple and low level, there are no topics or channels, the main 3 things that you can do with it is this:

1. Connect to a ws server - const webSocket = new WebSocket(url, protocols);
2. Listen to messages - webSocket.onmessage = ...
3. Publish messages - webSocket.send(message)

Since OpenAPI doesn't introduce higher level abstractions on top of HTTP either (other than auth, payloads), I think the websocket OpenAPI definition should stay true to this as well, and plainly define where to connect, what messages to expect and what messages are possible to send.

1. Where to connect?

I think the Paths Object is perfectly capable of describing this:

openapi: "3.0.2"
paths:
  /sample-socket:
    get:
      operationId: sampleSocket

This would obviously need a server with the wss protocol. We should probably also limit allowing websocket-enabled operations to get as with the standard browser API I don't think you can connect with any other verb, nor can you send a request body.

2. What messages can the client listen to (or the server send)

The already existing Content Object is perfectly capable of describing. I'd suggest adding an optional subscribe, listen or receive field on the Operation Object of Content Object type:

openapi: "3.0.2"
paths:
  /sample-socket:
    get:
      operationId: sampleSocket
      subscribe:
        application/json:
          schema:
            ... schema or ref to schemas

This leaves a lot to be desired, since I don't think it should be the responsibility of openapi to add higher level concepts to the existing web standard (like topics/subjects/channels), this leaves a bit to be desired in terms of what messages can you expect. However you can think of the message as a union type of all the different data you can receive. So you could do:

openapi: "3.0.2"
paths:
  /sample-socket:
    get:
      operationId: sampleSocket
      subscribe:
        application/json:
          schema:
            oneOf:
              - $ref: '#/components/schemas/UserCreatedMessage'
              - $ref: '#/components/schemas/UserDeletedMessage'
              - $ref: '#/components/schemas/PostLikedMessage'
              - ...

This would allow code/documentation generators to work in a very generic way, while still allowing the actual client and server code to do whatever higher level abstraction they want with the messages.

2. What messages can the client send (or the server listen to)

I'd do exactly the same thing with this as well, use a Content Object. I'd suggest adding an optional publish or send field to the Operation Object, that would be a Content Object, where you can do something like this:

openapi: "3.0.2"
paths:
  /sample-socket:
    get:
      operationId: sampleSocket
      publish:
        application/json:
          schema:
            oneOf:
              - $ref: '#/components/schemas/CreateUserPayload'
              - $ref: '#/components/schemas/DeleteUserPayload'
              - $ref: '#/components/schemas/LikePostPayload'
              - ...

Full example of my suggestion

openapi: "3.0.2"
paths:
  /sample-socket:
    get:
      operationId: sampleSocket
      subscribe:
        application/json:
          schema:
            oneOf:
              - $ref: '#/components/schemas/UserCreatedMessage'
              - $ref: '#/components/schemas/UserDeletedMessage'
              - $ref: '#/components/schemas/PostLikedMessage'
              - ...
      publish:
        application/json:
          schema:
            oneOf:
              - $ref: '#/components/schemas/CreateUserPayload'
              - $ref: '#/components/schemas/DeleteUserPayload'
              - $ref: '#/components/schemas/LikePostPayload'
              - ...

I'm not really sure if this makes sense. I don't think we can do much more with this. Let me know if this is a reasonable idea, and if so, how can I make a proposal.

[darrelmiller]

OpenAPI specification is scoped to the HTTP protocol. We recommend the use of AsyncAPI for messaging based interactions.

[stefan-niedermann]

OpenAPI specification is scoped to the HTTP protocol.

What a pitty that the well elaborated proposals have been rejected with just one sentence, not mentioning any reasons for the decision and ignoring more than hundred upvotes from OpenAPI users (who probably partially also represent companies with way more users in the end as in my case).

Sticking to REST only and not accepting the reality, that the number of pure REST only services will get lower in the time while the count of mixed services are increasing seems to be a bad decision in regards of providing a future proof and up to date tool.

We recommend the use of AsyncAPI for messaging based interactions.

Since AsyncAPI managed to stay compatible with OpenAPI (and common REST services), we will of course not "use […] AsyncAPI for messaging based interactions" but for all of our services - at least I can't see a reason to maintain and use two tools if one can cover all the needs.

I would have loved to see OpenAPI becoming this tool (and still do) because it is more well known with a (still) larger ecosystem. I appeal to explain the reasons behind your decision and rethink about the future of the OpenAPI project in the long term.

[Astiolo]

Might be worth noting that OpenAPI and AsyncAPI are both part of the Linux Foundation.

I echo the sentiments of @stefan-niedermann, except that AsyncAPI can't actually document HTTP APIs yet because it doesn't even support multiple methods for the one path (asyncapi/bindings#2).

Really, I don't understand why they aren't combined. Maybe AsyncAPI can eventually include OpenAPI as a subset of its functionality, but currently there just doesn't appear to be a nice, standardised way to document mixed APIs.

[webron]

This is one of the challenges of handling issues during open calls - sometimes the explanations we provide end up being a bit terse (even though they are normally a result of a long conversation). To clarify, the proposal have not been rejected yet (though it will be), but rather this issue was closed as something we're no longer looking to explore. @darrelmiller was indeed planning on providing a more elaborate explanation when closing the proposal PR, including thanking the people involved for spending the time on it, as that's genuinely appreciated.

We hope you're aware that our 'online' discussions are public and anyone can join them. The recording of the meeting is also available should anyone want to listen in after the fact.

As a team working on the OAS, we've made a decision to support HTTP-based APIs only, since we have to draw the line somewhere. It doesn't seem to be the right step for the spec, at the moment, to change that just for supporting websockets, especially given that AsyncAPI supports it already. As mentioned in our meeting, we feel there are plenty of HTTP API related topics we need to cover in the spec that should take priority over expanding it to support websockets.

We recognize that companies these days provide multiple API types to their users. This includes not just OpenAPI based APIs and AsyncAPI, but also gRPC, GraphQL and so on. In some of our SIGs (special interest workgroups), issues of cross collaboration between API types come up, but it's not something we've collectively solved by now.

[bali182]

I'm the author of the proposal, and I really respect the decision to keep the spec single responsibility and focus on HTTP only.

Initially, I was thinking the same as what many of you suggest: use AsyncAPI. The reason I made the proposal here, because I stumbled upon the roadmap of AsyncAPI, that I really disagree with (but I also respect it). To me it sounds like the approach is "support everything" - OpenAPI, GraphQL, Pub-sub communication, etc... included.

This in my opinion leads nowhere, as it's incredibly difficult to have a spec that idiomatically describes so many flavours of communication, and I'm already having a hard time grasping some of the concepts that are being brought up in AsyncAPI issues/proposals.

If AsyncAPI was an alternative, that I could confidently use today to build tooling for sockets, I'd never have made the proposal. But to me AsyncAPI is not there yet, and considering design goals I don't think I can effectively chime in on the AsyncAPI front.

What I tried to do here, is describe just the WebSocket protocol as part of OpenAPI without mixing in any frameworks, as it's very closely tied with HTTP. It would be super useful for me (and probably a lot of others too) to have a simple way of describing them.

[fmvilas]

I echo the sentiments of @stefan-niedermann, except that AsyncAPI can't actually document HTTP APIs yet because it doesn't even support multiple methods for the one path (asyncapi/bindings#2).

Right on time @Astiolo :) asyncapi/bindings#2 (comment)

This in my opinion leads nowhere, as it's incredibly difficult to have a spec that idiomatically describes so many flavours of communication, and I'm already having a hard time grasping some of the concepts that are being brought up in AsyncAPI issues/proposals.

This is also one of my fears, tbh @bali182. Our roadmap says to eventually support embedding or referencing subsets of other specs like OpenAPI. We definitely don't want to reinvent the wheel here (take this as an example). That said, I must say I haven't gotten this feedback before from the community. It would be awesome if you can elaborate on why you think so and how can we be better. Maybe just start a discussion even if you don't want to go too deep in the topic but at least that would serve as a place for others to chime in 🙏 The roadmap is not set in stone and is being continuously revisited. Let's collaborate on this.

[bali182]

@fmvilas Opened a discussion topic about this :) I hope it doesn't come through as harsh criticism, I simple disagree with the direction.

[SaadBazaz]

OpenAPI specification is scoped to the HTTP protocol. We recommend the use of AsyncAPI for messaging based interactions.

Should probably rename OpenAPI to HttpxAPI, in that case.

That way we can truly achieve scoped API specs.

[jroper]

gRPC has effectively solved the ability to specify an API for bidirectional streaming that works across languages/frameworks and works with HTTP. From my perspective, I don't think OpenAPI needs to support this anymore, and I lost interest in this issue as gRPC has matured and become widely support. So I think keeping OpenAPI purely focused on REST APIs is a perfectly reasonable decision, and people that want APIs that support streaming should choose the right tool for the job, which is not OpenAPI, and that's fine.

[lknite]

Agreed @jroper, thankyou for sharing. I think people love REST & swagger and naturally move to websockets for streaming & eventing to fill a need REST doesn't provide, and it seems natural that openapi might be able to similarly document the websocket calls.

Websockets are a fantastic technology, however with rabbitmq/activemq/kafka/etc... it can be seen that websockets are a low-level resource by comparison, and to implement something with multiple websockets to handle streaming/messaging/eventing you will "most likely" find yourself reinventing the wheel, and rewriting your code down the line.

For those who come across this issue looking to document websockets, you may want to seriously consider deploying your app via a container along with a rabbitmq or similar container to provide the streaming/messaging/eventing.

[olawalejuwonm]

Is there any resolution as regards this? Will AsyncAPI be combined to OpenAPI so we can have server side event documented with it?

[achievement008]

Hi all! We sometimes use WS in our projects, and often the interface has a rigid structure (what can be sent,
what can be obtained from it). For such a rigid structure, we wrote a new decorator (like drf_spectacular extend_schema) and plugin for the frontend, and now swagger can do this. Written quick-and-dirty, not for every situation at all, but in case it will be useful for someone.

Among the possibilities:
You can look at the messages and their structure
What can be sent, what can be received
Active socket connection
List of received and sent messages
Sending/receiving messages

Demo project link: https://github.com/advancedmonitoring/hba-demo-todo-app