From 416e24999863b0460e5703bb8940401249f3ee4b Mon Sep 17 00:00:00 2001 From: Cyril Ferlicot Date: Fri, 22 May 2015 00:13:12 +0200 Subject: [PATCH] updated Web Socket chapter --- WebSockets/WebSockets.pier | 99 +++++++++++++++++++++++++++----------- 1 file changed, 70 insertions(+), 29 deletions(-) diff --git a/WebSockets/WebSockets.pier b/WebSockets/WebSockets.pier index af6cfef..4f8ce37 100644 --- a/WebSockets/WebSockets.pier +++ b/WebSockets/WebSockets.pier @@ -6,33 +6,50 @@ Sven Van Caekenberghe September 2012 - Updated January 2013 (This is a draft) -The WebSocket protocol defines a full-duplex single socket connection over which messages can be sent between a client and a server. It simplifies much of the complexity around bi-directional web communication and connection management. WebSocket represents the next evolutionary step in web communication compared to Comet and Ajax. +The WebSocket protocol defines a full-duplex single socket connection over which messages can be sent between a client and a server. It simplifies much of the +complexity around bi-directional web communication and connection management. WebSocket represents the next evolutionary step in web communication compared to +Comet and Ajax. !!An Introduction to WebSockets -HTTP, one of the main technologies of the internet, defines a communication protocol between a client and a server where the initiative of the communication lies with the client and each interaction consists of a client request and a server response. When correctly implemented and used, HTTP is enormously scaleable and very flexible. +HTTP, one of the main technologies of the internet, defines a communication protocol between a client and a server where the initiative of the communication +lies with the client and each interaction consists of a client request and a server response. When correctly implemented and used, HTTP is enormously scaleable +and very flexible. -With the arrival of advanced Web applications mimicking regular desktop applications with rich user interfaces, as well as mobile Web applications, it became clear that HTTP was not suitable or not a great fit for two use cases: +With the arrival of advanced Web applications mimicking regular desktop applications with rich user interfaces, as well as mobile Web applications, it became +clear that HTTP was not suitable or not a great fit for two use cases: when the server wants to take the initiative and send the client a message when the client wants to send (many) (possibly asynchronous) short messages with little overhead -In the HTTP protocol, the server cannot take the initiative to send a message, the only workaround is for the client to do some form of polling. For short messages, the HTTP protocol adds quite a lot of overhead in the form of meta data headers. For many applications, the response (and the delay waiting for it) are not needed. Previously, Comet and Ajax were used as (partial) solutions to these use cases. +In the HTTP protocol, the server cannot take the initiative to send a message, the only workaround is for the client to do some form of polling. For short +messages, the HTTP protocol adds quite a lot of overhead in the form of meta data headers. For many applications, the response (and the delay waiting for it) +are not needed. Previously, Comet and Ajax were used as (partial) solutions to these use cases. -The WebSocket protocol defines a reliable communication channel between two equal parties, typically, but not necessarily, a Web client and a Web server, over which asynchronous messages can be send with very little overhead. Messages can be any String or ByteArray. Overhead is just a couple of bytes. There is no such thing as a direct reply or a synchronous confirmation. +The WebSocket protocol defines a reliable communication channel between two equal parties, typically, but not necessarily, a Web client and a Web server, over +which asynchronous messages can be send with very little overhead. Messages can be any String or ByteArray. Overhead is just a couple of bytes. There is no such +thing as a direct reply or a synchronous confirmation. -Using WebSockets, a server can notify a client instantly of interesting events, and clients can quickly send small notifications to a server, possibly multiplexing many virtual communications channels over a single network socket. +Using WebSockets, a server can notify a client instantly of interesting events, and clients can quickly send small notifications to a server, possibly +multiplexing many virtual communications channels over a single network socket. !!The WebSocket Protocol -Zinc WebSockets implements RFC 6455 http://tools.ietf.org/html/rfc6455, not any of the previous development versions. For an introduction to, both http://en.wikipedia.org/wiki/WebSocket and http://www.websocket.org are good starting points. +Zinc WebSockets implements RFC 6455 http://tools.ietf.org/html/rfc6455, not any of the previous development versions. For an introduction to, both +http://en.wikipedia.org/wiki/WebSocket and http://www.websocket.org are good starting points. -As a protocol, WebSocket starts with an initial setup handshake that is based on HTTP. The initiative for setting up a WebSocket lies with the client, who is sending a so called connection upgrade request. The upgrade request contains a couple of special HTTP headers. The server begins as a regular HTTP server accepting the connection upgrade request. When the request is conform the specifications, a 101 switching protocols response is sent. This response also contains a couple of special HTTP headers. From that point on, the HTTP conversation over the network socket stops and the WebSocket protocol begins. +As a protocol, WebSocket starts with an initial setup handshake that is based on HTTP. The initiative for setting up a WebSocket lies with the client, who is +sending a so called connection upgrade request. The upgrade request contains a couple of special HTTP headers. The server begins as a regular HTTP server +accepting the connection upgrade request. When the request is conform the specifications, a 101 switching protocols response is sent. This response also +contains a couple of special HTTP headers. From that point on, the HTTP conversation over the network socket stops and the WebSocket protocol begins. -WebSocket messages consist of one or more frames with minimal encoding. Behind the scenes, a number of control frames are used to properly close the WebSocket and to manage keeping alive the connection using ping and pong frames. +WebSocket messages consist of one or more frames with minimal encoding. Behind the scenes, a number of control frames are used to properly close the WebSocket +and to manage keeping alive the connection using ping and pong frames. !!Source Code -The code implementing Zinc WebSockets resides in a single package called 'Zinc-WebSocket-Core' in the Zinc HTTP Components repositories. There is also an accompanying 'Zinc-WebSocket-Tests' package containing the unit tests. The ==ConfigurationOfZincHTTPComponents== has a group called =='WebSocket'== that you can load separately. +The code implementing Zinc WebSockets resides in a single package called 'Zinc-WebSocket-Core' in the Zinc HTTP Components repositories. There is also an +accompanying 'Zinc-WebSocket-Tests' package containing the unit tests. The ==ConfigurationOfZincHTTPComponents== has a group called =='WebSocket'== that you can +load separately. !!!Using Client Side WebSockets @@ -44,7 +61,8 @@ ws://www.example.com:8088/my-app Two new schemes are defined, ==ws://== for regular WebSockets and ==wss://== for the secure (TLS/SSL) variant. The ==host:port== and path specification should be familiar. -Zinc WebSockets supports the usage of client side WebSockets of both the regular and secure variants (the secure variant requires Zodiac TLS/SSL). The API is really simple, once you open the socket, you use ==sendMessage:== and ==readMessage:== and finally ==close==. +Zinc WebSockets supports the usage of client side WebSockets of both the regular and secure variants (the secure variant requires Zodiac TLS/SSL). The API is +really simple, once you open the socket, you use ==sendMessage:== and ==readMessage:== and finally ==close==. Here is a client side example taking to a public echo service: @@ -56,11 +74,14 @@ webSocket := ZnWebSocket to: 'ws://echo.websocket.org'. readMessage ] ensure: [ webSocket close ]. ]]] -Note that ==readMessage:== is blocking. It always returns a complete ==String== or ==ByteArray==, possible assembled out of multiple frames. Inside ==readMessage:== control frames will be handled automagically. Reading and sending are completely separate and independent. +Note that ==readMessage:== is blocking. It always returns a complete ==String== or ==ByteArray==, possible assembled out of multiple frames. Inside +==readMessage:== control frames will be handled automagically. Reading and sending are completely separate and independent. -For sending very large messages, there are ==sendTextFrames:== and ==sendByteFrames:== that take a collection of ==Strings== or ==ByteArrays== to be sent as different frames of the same message. At the other end, these will be joined together and seen as a single message. +For sending very large messages, there are ==sendTextFrames:== and ==sendByteFrames:== that take a collection of ==Strings== or ==ByteArrays== to be sent as +different frames of the same message. At the other end, these will be joined together and seen as a single message. -In any non-trivial application, you will have to add your own encoding and decoding to messages. In many cases, JSON will be the obvious choice as the client end is often JavaScript. A modern, standalone JSON parser and writer is NeoJSON. +In any non-trivial application, you will have to add your own encoding and decoding to messages. In many cases, JSON will be the obvious choice as the client +end is often JavaScript. A modern, standalone JSON parser and writer is NeoJSON. To use secure web sockets, just use the proper URL scheme ==wss://== as in the following example: @@ -76,11 +97,15 @@ Of course, your image has to contain Zodiac and your VM needs access to the prop !!Using Server Side WebSockets -Since the WebSocket protocol starts off as HTTP, it is logical that a ZnServer with a special delegate is the starting point. ==ZnWebSocketDelegate== implements the standard ==handleRequest:== to check if the incoming request is a valid WebSocket connection upgrade request. If so, the matching 101 switching protocols response is constructed and sent. From that moment on, the network socket stream is handed over to a new, server side ==ZnWebSocket== object. +Since the WebSocket protocol starts off as HTTP, it is logical that a ZnServer with a special delegate is the starting point. ==ZnWebSocketDelegate== implements +the standard ==handleRequest:== to check if the incoming request is a valid WebSocket connection upgrade request. If so, the matching 101 switching protocols +response is constructed and sent. From that moment on, the network socket stream is handed over to a new, server side ==ZnWebSocket== object. - ==ZnWebSocketDelegate== has two properties. An optional prefix implements a specific path, like ==/my-ws-app==. The required handler is any object implementing ==value:== with the new web socket as argument. + ==ZnWebSocketDelegate== has two properties. An optional prefix implements a specific path, like ==/my-ws-app==. The required handler is any object implementing + ==value:== with the new web socket as argument. -Let's implement the echo service that we connected to as a client in the previous subsection. In essence, we should go in a loop, reading a message and sending it back. Here is the code: +Let's implement the echo service that we connected to as a client in the previous subsection. In essence, we should go in a loop, reading a message and sending +it back. Here is the code: [[[ ZnServer startDefaultOn: 1701. @@ -91,7 +116,9 @@ ZnServer default delegate: (ZnWebSocketDelegate handler: webSocket sendMessage: message ] repeat ]). ]]] -We start a default server on port 1701 and replace its delegate with a ==ZnWebSocketDelegate==. The ==ZnWebSocketDelegate== will pass each correct web socket request on to its handler. In this example, a block is used as handler. The handler is given a new connected ==ZnWebSocket== instance. For the echo service, we go into a repeat loop, reading a message and sending it back. +We start a default server on port 1701 and replace its delegate with a ==ZnWebSocketDelegate==. The ==ZnWebSocketDelegate== will pass each correct web socket +request on to its handler. In this example, a block is used as handler. The handler is given a new connected ==ZnWebSocket== instance. For the echo service, we +go into a repeat loop, reading a message and sending it back. Finally, you can stop the server using @@ -106,9 +133,11 @@ ConnectionTimedOut ConnectionClosed (or its more specific subclass ZnWebSocketClosed) ]]] -The ==readMessage== call blocks on the socket stream waiting for input until its timeout expires, which will be signaled with a ConnectionTimedOut exception. In most applications, you should just keep on reading, essentially ignoring the timeout for an infinite wait on incoming messages. +The ==readMessage== call blocks on the socket stream waiting for input until its timeout expires, which will be signaled with a ConnectionTimedOut exception. +In most applications, you should just keep on reading, essentially ignoring the timeout for an infinite wait on incoming messages. -This behavior is implemented in the ==ZnWebSocket>>runWith:== convenience method: it enters a loop reading messages and passing them to a block, continuing on timeouts. This simplifies our example: +This behavior is implemented in the ==ZnWebSocket>>runWith:== convenience method: it enters a loop reading messages and passing them to a block, continuing on +timeouts. This simplifies our example: [[[ ZnServer startDefaultOn: 1701. @@ -119,7 +148,9 @@ ZnServer default delegate: (ZnWebSocketDelegate handler: webSocket sendMessage: message ] ]). ]]] -That leaves us with the problem of ==ConnectionClosed==. This exception can occur at the lowest level when the underlying network connection closes unexpectedly, or at the WebSocket protocol level when the other end sends a close frame. In either case we have to deal with it as a server. In our trivial echo example, we can catch and ignore any ConnectionClosed exception. +That leaves us with the problem of ==ConnectionClosed==. This exception can occur at the lowest level when the underlying network connection closes unexpectedly, +or at the WebSocket protocol level when the other end sends a close frame. In either case we have to deal with it as a server. In our trivial echo example, we +can catch and ignore any ConnectionClosed exception. There is a handy shortcut method on the class side of ==ZnWebSocket== that helps to quickly set up a server implementing a WebSocket service. @@ -137,11 +168,15 @@ ZnWebSocket Don't forget to inspect the above code so that you have a reference to the server to close it, as this will not be the default server. -Although using a block as handler is convenient, for non-trivial examples a regular object implementing ==value:== will probably be better. You can find such an implementation in ==ZnWebSocketEchoHandler==. +Although using a block as handler is convenient, for non-trivial examples a regular object implementing ==value:== will probably be better. You can find such an +implementation in ==ZnWebSocketEchoHandler==. -The current process (thread) as spawned by the server can be used freely by the handler code, for as long as the web socket connection lasts. The responsibility for closing the connection lies with the handler, although a close from the other side will be handled correctly. +The current process (thread) as spawned by the server can be used freely by the handler code, for as long as the web socket connection lasts. The responsibility +for closing the connection lies with the handler, although a close from the other side will be handled correctly. -To test our echo service, you could connect to it using a client side web socket, like we did in the previous subsection. This is what the unit test ==ZnWebSocketTests>>testEcho== does. Another solution is to run some JavaScript code in a web browser. You can find the necessary HTML page containing JavaScript code invoking the echo service on the class side of ==ZnWebSocketEchoHandler==. The following setup will serve this code: +To test our echo service, you could connect to it using a client side web socket, like we did in the previous subsection. This is what the unit test +==ZnWebSocketTests>>testEcho== does. Another solution is to run some JavaScript code in a web browser. You can find the necessary HTML page containing JavaScript +code invoking the echo service on the class side of ==ZnWebSocketEchoHandler==. The following setup will serve this code: [[[ ZnServer startDefaultOn: 1701. @@ -164,7 +199,8 @@ http://localhost:1701/ws-echo-client The first one will connect to ==ws://echo.websocket.org== as a reference, the second one will connect to our implementation at ==ws://localhost:1701/ws-echo==. -Another simple example is available in ==ZnWebSocketStatusHandler== where a couple of Smalltalk image statistics are emitted every second for an efficient live view in your browser. In this scenario, the server accepts each incoming web socket connection and starts streaming to it, not interested in any incoming messages. Here is the core loop: +Another simple example is available in ==ZnWebSocketStatusHandler== where a couple of Smalltalk image statistics are emitted every second for an efficient live +view in your browser. In this scenario, the server accepts each incoming web socket connection and starts streaming to it, not interested in any incoming messages. Here is the core loop: [[[ ZnWebSocketStatusHandler>>value: webSocket @@ -179,7 +215,8 @@ ZnWebSocketStatusHandler>>value: webSocket self crLog: 'Stopping status streaming' ]]] -The last example, ==ZnWebSocketChatroomHandler==, implements the core logic of a chatroom: clients can send messages to the server who distributes them to all connected clients. In this case, the handler has to manage a collection of all connected client web sockets. Here is the core loop: +The last example, ==ZnWebSocketChatroomHandler==, implements the core logic of a chatroom: clients can send messages to the server who distributes them to all +connected clients. In this case, the handler has to manage a collection of all connected client web sockets. Here is the core loop: [[[ ZnWebSocketChatroomHandler>>value: webSocket @@ -241,9 +278,12 @@ Inside your Pharo image, you can also send chat messages, much like a moderator: !!A Quick Tour of the Implementation -All code resides in the =='Zinc-WebSocket-Core'== package. The wire level protocol, the encoding and decoding of frames can be found in ==ZnWebSocketFrame==. The key methods are ==writeOn:== and ==readFrom:== as well as the instance creation protocol. Together with the testing protocol and #printOn: these should give enough information to understand the implementation. +All code resides in the =='Zinc-WebSocket-Core'== package. The wire level protocol, the encoding and decoding of frames can be found in ==ZnWebSocketFrame==. +The key methods are ==writeOn:== and ==readFrom:== as well as the instance creation protocol. Together with the testing protocol and #printOn: these should give +enough information to understand the implementation. - ==ZnWebSocket== implements the protocol above frames, either from a server or a client perspective. The key methods are ==readMessage== and ==readFrame==, sending is quite simple. Client side setup can be found on the class side of ==ZnWebSocket==. Server side handling of the setup is implemented in ==ZnWebSocketDelegate==. + ==ZnWebSocket== implements the protocol above frames, either from a server or a client perspective. The key methods are ==readMessage== and ==readFrame==, + sending is quite simple. Client side setup can be found on the class side of ==ZnWebSocket==. Server side handling of the setup is implemented in ==ZnWebSocketDelegate==. Two exceptions, ==ZnWebSocketFailed== and ==ZnWebSocketClosed== and a shared ==ZnWebSocketUtils== class round out the core code. @@ -257,7 +297,8 @@ http://websocket.stfx.eu Have a look at ==ZnWebSocketDelegate class>>installExamplesInServer:== as a starting point to learn how this demo was set up. -Setting up a production demo is complicated by the fact that most proxies and load balancers, most notable market leader Apache, do not (yet) deal correctly with the WebSocket protocol. It is thus easiest to organize things so that your client talk directly to your Smalltalk image. +Setting up a production demo is complicated by the fact that most proxies and load balancers, most notable market leader Apache, do not (yet) deal correctly with +the WebSocket protocol. It is thus easiest to organize things so that your client talk directly to your Smalltalk image. The implementation of Zinc WebSockets as an add-on to Zinc HTTP Components was made possible in part through financial backing by Andy Burnett of Knowinnovation Inc. and ESUG.